Data Integration

 View Only

Editing of Keys and Attributes in DataStage Legacy and Daas

By Ralf Martin posted Mon February 05, 2024 05:42 AM

  

This is what I had presented at the DDSUG (German speaking DataStage User Group) 38 last week in Munich.

In DataStage 11.7 there is a functionality to bulk edit keys and attributes which is a bit hidden. I will show some examples in this blog post and also show the current status of this in DaaS/CP4D. I had worked with DataStage for several years before discovering this funtionallity and it can save you a lot of effort if you know it.

I will demonstrate the functionallity for the following stages:

   Sort
   Column Export
   Aggregator
   Peek
   Change Capture

I have the following example Job

Sort Stage
On the Stage->Properties Tab, click on the red "Key = ?", then the Icon below the move up/down buttons should come to life, if you click on it, then you go to the bulk edit screen

As described on the annotation on the canvas of the job, I want to sort on the four key columns, doing a "don't sort previously sorted" on the first 3. So in the bulk edit screen I mark the 4 key columns with shift+click and click on the arrow to the right button

Now on the right side of the screen I mark the columns again

and click on the arrow to right on the right side of the screen to select Sort Key Mode for all selected columns

You'll then get the following result on the properties tab

The same then for the other stages

Column Export stage

Aggregator stage

Here you can bulkadd and -edit both Grouping Keys and Column for Calcluation

Grouping Keys

Column for Calculation

This is the finial result for the aggregator stage, unfortunately for the columns for Calculations the only bulkediting you can currently to is setting preserve type true/false, maybe in Nextgen DataStage you will be able to set the Aggregation type for several column

Peek Stage

Change Capture Stage
Here you can also edit the Keys and Values

Keys:

Values:

Now the same example flow in DaaS

Sort Stage in DaaS

In Daas also on  the Stage->Properties tab, click on edit next to the Sorting Keys

Then in the tear sheet, click on Add key +

For the stages where it is already implemented, bulkadding/-editing is directly build in (no somewhat hidden functionalliyt anymore), just click on the columns you want to add

and then edit the attributes and click on Apply

In our example I added the four key columns and applied "don't sort previously sorted" to all of them, so I have to remove this from the last key column later (or would have to add them without this attribute and bulk edit the first 3 columns afterwards)

This will then be the result

For the examples shown in DataStage Legacy the currently implemented state in DaaS is:
   Sort: Implemented
   Column Export: Not implemented yet
   Aggregator: Grouping keys implemented for DaaS (not yet in CP4D), Columns for Calculations not implemented yet
   Peek : Not implemented yet
   Change Capture: Implemented for Keys, not implemented yet for Values

IBM is working on this, when preparing the speech, Grouping keys in the Aggregator stage where not implemented, while a week later they had been added. They will also add the option to filter the columns you want to add via a search/filter box and have the option to check all of them with one click.

I hope the knowledge of this functionallity will help you to work with DataStage more efficiently.

KR Ralf

0 comments
7 views

Permalink