One of the biggest improvements of CP4D DataStage 5.1 last year, has been the introduction of the New Transformer UI, for which I had the honor to have a look at before it was officially available and contribute a bit to its development, since 5.1 it is now available as a beta feature that is still under development.

If you want to use it, open the Transformer stage in a DataStage flow and in the top right corner you will spot this switch

With this switch you can switch between the old and new CP4D Transformer UI, for in the end, it is just a different UI for the same engine, so while there is nothing that you can only achieve with the new UI, in my opinion it is much easier to use than the old one.

If we compare the old UI

to the new one

If you have been working with legacy DataStage like I did, it will look very familiar to you.

What is new compared to legacy?

·        In the action bar you find a search icon, with which you can search (or search and replace) for column names, derivation text or empty derivations

·        Also, in the action bar you now have an undo/redo for the scope of this transformer session. I think this is a very fine feature, if you are halfway through the editing of your transformer and do something horrible, you now don’t have to decide to cancel everything you have but can just undo this (or several) step.

·        The layout has changed.

o   In legacy I never liked that column definitions and derivations were in two separate places, especially for the input columns. Now you have column name, datatype and derivation in one place, only if you want to change more than the name of a column or its position, then you have to click on the Edit icon, which is in the top right of every box, to get to a column tearsheet, here you can add or drop columns or edit the definition.

o   Also, now the stage and loop variables are now located in the bottom left corner, which is something that I like.

o   The metadata of your stage/loop variables is now always visible, which is also a big improvement over legacy.

o   Derivations that target a stage/loop variable are displayed on the left side of the screen, while the ones targeting an output column, are located in the middle.

·        Like in all stages of CP4D DataStage, in the column tearsheet, you can easily filter and bulk edit your columns.

·        Also like in the rest of CP4D DataStage, column metadata change propagation (also known as auto column propagation) is available, which is one of my favorite features in CP4D. It can of course be turned off on flow or on link level, the latter being especially useful in the transformer stage, e.g. for links that handle error logging.

·        Especially when making small changes to a link constraint, you can now do this directly in the main transformer canvas, with the option to enter the derivation editor or enter a menu (by clicking on the chain icon) where all constrains are shown below each other, to better compare them.

·        For the whole transformer canvas, you can now freely scroll and zoom, which is something you first have to get used to. As of now I cannot judge how good or bad this is, it is certainly different (both to legacy and the old CP4D transformer).

·        The biggest and most positive change compared to legacy is the new expression editor, which improves working with DataStage in many ways

o   You can now add comments to your derivation or e.g. comment out lines, it is done C style with two slashes.

o   On the left side, you have all the stuff that you can add to your expressions (like input columns, stage variables and functions), which you can easily search to e.g. quickly get the timestamp conversion function you need.

o   If you select an asset on the left, then in the bottom right, it will show you all available information on it, for an Input column or stage variable this would be the definition, while for a function it would be the documentation of that function, so there is no need anymore to go to the help page for this.

o   Instead of using the assets on the left, you can use type ahead in the expression editor, which has about the same set of options available.

o   Also, as shown in the picture above, errors in your derivations will be flagged for the appropriate line, with the error text being shown if you move your curser to the error icon.

What is new compared to the old CP4D Transformer UI?

·        Like most people with which I talked about this, I never liked the layout of the old UI compared to the timeless design of Lee Scheffler form legacy DataStage. Especially the card design of the stage variables, which was not very useful if there were more than a handful of variables and that you had to switch between stage variables, input columns and output columns.

·        Overall, there is much less wasted space in the new UI. On the same display, in the old UI I can see about 6 columns/derivations, while (without zooming out) it is 21 in the new UI. Also, you can now easily zoom out and in, without adjusting your browser.

·        Like in legacy you can now display mapping links for your derivations, which is the feature that was missed most, as far as I can say.

·        Constraints are now easier to look at and edit, as described above.

·        Also as described above there is now an Undo/Redo on transformer session level available.

·        The Find and Replace is available for both UIs, it is just located somewhere else.

·        In addition to the Find and Replace, there is a derivation substitution option on link level that can use Regex. I don’t know why there are two places to find and replace expressions, this is a bit confusing to me.

·        Like in Legacy you can now drag and drop:

o   The position of your columns/variables

o   The position of your link ordering (in addition to editing the link ordering in the Stage properties)

o   An input/output column as new output column

o   An input/output column or stage/loop variable into a derivation (overwriting the existing derivation after confirmation)

o   And existing derivation into a derivation (overwriting the existing derivation after confirmation)

·        The column tearsheets offer new functionality, like adjusting the width of a column. As far as I know, this is going to be rolled out for other stages in the future.

Conclusion

From what I can judge, the new Transformer UI is a big step ahead, both bringing back features from legacy that where missed a lot (mappings links, drag and drop) while adding new cool functionality (expression editor, find and replace, column bulk editing). There is still work going on here and not everything is yet working a 100% (not all derivations are shown when mapping links are enabled, drag on drop does not work well for stage variables), but as I have seen how this has been half a year ago, I am sure this will be fine very soon.

When nowadays still working with the legacy transformer, I already miss the features that are implemented into CP4D and I am looking forward to the migration that we are currently preparing.