Most people I had spoken about migrating from legacy DataStage to CP4D told me, that they don’t want to migrate just to stay in support and 11.7 yet has no end of support announced, so for them there is no reason to migrate. But DataStage on CP4D offers a lot of improvements, especially in the UI, compared to legacy which would justify migrating.
Find below some of my favourite improvements of CP4D, for some of those I have worked together with IBM.
Column metadata change propagation:
This is my favourite CP4D feature (I had called it designtime column propagation, when it was introduced). In a nutshell, if you add, drop or edit a column it is propagated downstream. So if you add 4 new columns to a flow, you do not have to drag them through all your stages, also if you got the datatype or nullability wrong when creating a flow (or it has been changed in the database), you now only have to change this once and not in every stage. It is important to know that you can switch this feature off on flow and link level, the later one is important for you don’t want to have it enabled e.g. on links that do logging. Also e.g. in the transformer stage metadata is not propagated if doing explicit or implicit conversions on a column (e.g. if you have VARCHAR(50) column on which you do a Left(40) to fit into the VARCHAR(40) target column, then increasing the input column size to VARCHAR(60) should not be propagated).
Expression editor in the Transformer stage:
In my last blog I had written about the new Transformer UI in CP4D which you can read here:
(https://community.ibm.com/community/user/blogs/ralf-martin/2025/05/08/new-features-in-cp4d-datastage-the-new-transformer)
I will not go into detail again, the enhancements in the expression editor are a great boost to productivity, with type ahead, the ability to add comment lines to your code, build in access to the documentation.
Filtering everywhere:
When starting to work with CP4D, you will notice that there will be a filtering box in nearly every context. I myself prefer filtering (you enter something in the filtering box and your result set is reduced to the items that match the filter) compared to a search, that does not reduce your items but bring you to the next item matching your search criteria.
An example:
In legacy the palette was grouped into topics, IBM asked me about this for CP4D and how I would arrange them, I gave them my input but also told them that in CP4D I do not really care about it, for I think that I will just use the Find palette nodes filter every time.
Instead of the legacy find column in a stage, you filter down to the columns matching your filter (the filter does not only apply by name, but also e.g. by data type), then you can e.g. select them all, drop them or bulkedit the metadata.
In the example below I would like to only see the columns named DIM_ORGA, then I could select them and e.g. change them from BIGINT to INTEGER or set them all to NOT NULL.
Or are you tired of selecting a key column in the Sort or Aggregator stage from a long list of columns? Also here you can filter your set of columns and improve your productivity.
The list of places where this is implemented is just to long to name them all.
Expression editor for the Modify stage:
As far as I know, it has been my idea to add this to CP4D, I always wanted to have an expression editor for the Modify stage, maybe because I was tired of looking into the documentation to check if the correct syntax is NOWARN or NO WARN (the first one is correct) or having to go to the input or output tab to copy paste my column names.
You now have all that you can add (that is available during design time) to an modify expression in a menu on the left (of course with the option to filter) and for e.g. functions you have the documentation in the lower right, just like in the Transformer expression editor, also like in the Transformer expression editor type ahead is implemented.
Adding and removing stages from a link:
As far as I know, this has been on the list of the DDSUG (German Speaking DataStage User Group) for longer than I work with the tool. You have two stages connected by a link and you want to add a new stage between them (e.g. a Peek or Filter stage), in CP4D you just drag the new stage onto the link, the link gets split in two with the new stage connecting it, also if deleting a stage, it is removed and the input and output links are merged to a single link. When this was first introduced, it was not as smooth as it is now (e.g. how to handle the partitioning of the split link), but now it feels like magic.
Asset browser:
The Asset browser is one of the features I liked the first time I used it. You drag the Asset browser like a stage from the palette to the canvas, then you are on a screen where you can select from your assets (Connections, Subflows, DataSets, Flat files …), select one or more of them and they get added to your flow.
Some highlights:
· When adding the Assets browser “Stage” on a link, then it knows that you are going to add a Subflow and only suggest this type of asset.
· Adding a table from a connection it will have the connection, table name and column metadata added. You can add several tables to the canvas in one action.
· When adding a Flat file, then you will have the option to preview and adjust the settings like in the legacy Import->Table Definitions->Sequential File Definitions->Define Sequential Meta Data. Your Flat file will be fully set up once it is on the canvas.
Improved undo/redo:
The undo and redo functionality has been greatly improved, you don’t have to click five times on the undo button to undo the last five steps (you still can do), but you get a nice drop down list, showing you your last actions and when they have been made, then you can decide to which point you want to go back.
Project connections:
There is a Data Connection asset in legacy, which I myself yet only used in one project, everywhere else, we where using parameters in the connector itself.
In CP4D you can still use local connections (i.e. set up everything in the connector) but you are more or less gently pushed towards using project connection. In my opinion the benefits of project connection are bigger than those of local connections and migrating from legacy to CP4D is the correct time to start using them. You set up a connection to e.g. a database once for a project and reuse this asset in your connections, the asset browser or to import column metadata. Also some attributes that where set on connector level in legacy are now specified in the connection, like e.g. SSL connection or Keep conductor connection alive.
When we started working with project connections, not everything was smooth (like you could not use a parameter for port, for it was specified as a numeric field and #Port# is a string), there are still some things IBM is working on to be a 100% great (still you cannot add a keytab file to a Kerberos connection as a parameter via the UI) for some connections, but even if migrating now, it should be a better experience than in legacy.
Conclusion
I think I have brought up some good points for considering migrating from legacy to CP4D, which you will have to do anyway in the future, unless you are planning on moving to a different platform. Depending on your current setup and use of functionality there is some work to be done.
· If you are still using server flows, then you have to migrate them to parallel flows, for they are not supported in CP4D.
· If you are a user of basic functions, then you have to replace them with e.g. shell scripts.
· Check if all your sources and targets work a 100% like they should in CP4D, I am currently struggling to get Impala sources working a 100% like they should.
· There is no Director client like it was in Legacy, there is something different, check if you can get along with this, if not then talk with IBM what they should change.
· As far as I know the GIT integration/Version control/Deployment is yet not a 100% like we need it (but soon will be), check if the current functionality is ok for you. IBM is working on this.