DataStage - Data Integration

 View Only
  • 1.  DataStage Metadata usage upgrade

    Posted Thu June 30, 2022 09:49 AM
    Hello,
    I commonly use metadata management in DataStage for impact analysis for code changes.
    Currently I need to remove access to a dataset. For this I use the repository "find where used" function.
    After deleting access to my DataSet from my job and compiling it, the "find where used" is still refers to the deleted dataset.
    Is there a way to clean/regenerate the dependencies to have it working properly?

    ------------------------------
    Jerzy Konarski
    ------------------------------


  • 2.  RE: DataStage Metadata usage upgrade

    IBM Champion
    Posted Fri July 01, 2022 01:40 AM
    Have you deleted whole link? Maybe another link is copy of the previous one and column definition still referred to table definition of dataset.

    ------------------------------
    Marcin Molak
    ------------------------------



  • 3.  RE: DataStage Metadata usage upgrade

    Posted Fri July 01, 2022 02:59 AM
    Hi Marcin,
    Indeed the reference to the source that I no longer use is conveyed by the link. You can see it clearly in the export file. The incorrect source is referenced in the transformer and propagated to the output.
    Changing the column content in transformer doesn't change anything, the fake reference is still there. The only way to get rid of it is to remove the column from wherever it is referenced and recreate it. What is quite painful, the metadata must constitute an aid in the development and not a constraint.
    From my point of view it's a DataStage bug, at some point the references are no longer correct. In the absence of proper functioning, it would be necessary at least to have a function which makes it possible to regenerate the dependencies.
    Jerzy

    ------------------------------
    Jerzy Konarski
    ------------------------------



  • 4.  RE: DataStage Metadata usage upgrade

    IBM Champion
    Posted Fri July 01, 2022 03:22 AM
    I am aware of that. Nevertheless there is no option for DS to check what sort of metadata you put on link and I don't personally think that IBM will change it.

    ------------------------------
    Marcin Molak
    ------------------------------



  • 5.  RE: DataStage Metadata usage upgrade

    Posted Fri July 01, 2022 05:20 AM
    Metadata information is initialized by DataStage when I insert a datasource into the job.
    Afterwards this information is propagated by DataStage also.
    So when I change the data source for a column the software should be able to correct this information. Otherwise the information on the lineage is wrong and it is clearly a bug.
    I think so.

    ------------------------------
    Jerzy Konarski
    ------------------------------