Data Integration

 View Only

 Use same etl for a different project with same source

  • IBMChampion
datastage dev's profile image
datastage dev posted Fri November 15, 2024 07:33 AM

Hi,

I have few ETLs built for a current  project. The next project also has the source tables same as current project.

I would like to use same ETLs rby just repointing them to different source on a different server rather than redeveloping the same logic again

Could you please let me know what guidelines to be followed apart from parameterization?

Ralf Martin's profile image
Ralf Martin IBM Champion posted Mon November 18, 2024 12:57 AM

Hi,

what is your setup? Are you on DataStage 11.7 or on CP4D? What is your source database? As far as I unterstand, your new project is still in the same DataStage project, the name and structure of your source tables is also the same, only the location of your database is different.

The following example would be for DB2. Depending on how you work in your environment I would advise to use a parameterset for the parameters of your connection. It should contain at least, DBNAME (if you use Legacy, in CP4D it currently would need to be HOST and PORT), USER, PASSWORD. Have a value file for each of your source databases. Have your database connector in your DataStage Job use the parameters of the parameterset, or on CP4D (if you are not using local connections) have a project connection use your parameterset.

HOW MING (Felix) YONG's profile image
HOW MING (Felix) YONG posted Mon November 18, 2024 07:09 PM

Hi

If the Source/Target Type is the same than you can leverage ParameterSet as per advise by Ralf.

If you are using ODBC Connector, you can leverage ParameterSets even if Source/Target Type will change.

If the Source/Target Type will change, (e.g. Oracle to Db2) than you can leverage Shared Container (DataStage V11.7.1.x) or SubFlow (DataStage on CPD). Most likely you will also be using Run-time Column Propagation (RCP) in your Shared-Container or SubFlow Design.

Documentation on SubFlow - https://www.ibm.com/docs/en/cloud-paks/cp-data/5.0.x?topic=assets-reusable-job-design-subflows

Documentation on Shared Container - https://www.ibm.com/docs/en/iis/11.7?topic=jobs-making-parts-your-job-design-reusable