Data Integration

Hello,
I commonly use metadata management in DataStage for impact analysis for code changes.
Currently I need to remove access to a dataset. For this I use the repository "find where used" function.
After deleting access to my DataSet from my job and compiling it, the "find where used" is still refers to the deleted dataset.
Is there a way to clean/regenerate the dependencies to have it working properly?

------------------------------
Jerzy Konarski
------------------------------

#DataIntegration

Have you deleted whole link? Maybe another link is copy of the previous one and column definition still referred to table definition of dataset.

------------------------------
Marcin Molak
------------------------------

Original Message:
Sent: Thu June 30, 2022 09:48 AM
From: Jerzy Konarski
Subject: DataStage Metadata usage upgrade

Hello,
I commonly use metadata management in DataStage for impact analysis for code changes.
Currently I need to remove access to a dataset. For this I use the repository "find where used" function.
After deleting access to my DataSet from my job and compiling it, the "find where used" is still refers to the deleted dataset.
Is there a way to clean/regenerate the dependencies to have it working properly?

------------------------------
Jerzy Konarski
------------------------------

#DataIntegration

Hi Marcin,
Indeed the reference to the source that I no longer use is conveyed by the link. You can see it clearly in the export file. The incorrect source is referenced in the transformer and propagated to the output.
Changing the column content in transformer doesn't change anything, the fake reference is still there. The only way to get rid of it is to remove the column from wherever it is referenced and recreate it. What is quite painful, the metadata must constitute an aid in the development and not a constraint.
From my point of view it's a DataStage bug, at some point the references are no longer correct. In the absence of proper functioning, it would be necessary at least to have a function which makes it possible to regenerate the dependencies.
Jerzy

------------------------------
Jerzy Konarski
------------------------------

Original Message:
Sent: Fri July 01, 2022 01:40 AM
From: Marcin Molak
Subject: DataStage Metadata usage upgrade

Have you deleted whole link? Maybe another link is copy of the previous one and column definition still referred to table definition of dataset.

------------------------------
Marcin Molak

Original Message:
Sent: Thu June 30, 2022 09:48 AM
From: Jerzy Konarski
Subject: DataStage Metadata usage upgrade

Hello,
I commonly use metadata management in DataStage for impact analysis for code changes.
Currently I need to remove access to a dataset. For this I use the repository "find where used" function.
After deleting access to my DataSet from my job and compiling it, the "find where used" is still refers to the deleted dataset.
Is there a way to clean/regenerate the dependencies to have it working properly?

------------------------------
Jerzy Konarski
------------------------------

#DataIntegration

I am aware of that. Nevertheless there is no option for DS to check what sort of metadata you put on link and I don't personally think that IBM will change it.

------------------------------
Marcin Molak
------------------------------

Original Message:
Sent: Fri July 01, 2022 02:59 AM
From: Jerzy Konarski
Subject: DataStage Metadata usage upgrade

Hi Marcin,
Indeed the reference to the source that I no longer use is conveyed by the link. You can see it clearly in the export file. The incorrect source is referenced in the transformer and propagated to the output.
Changing the column content in transformer doesn't change anything, the fake reference is still there. The only way to get rid of it is to remove the column from wherever it is referenced and recreate it. What is quite painful, the metadata must constitute an aid in the development and not a constraint.
From my point of view it's a DataStage bug, at some point the references are no longer correct. In the absence of proper functioning, it would be necessary at least to have a function which makes it possible to regenerate the dependencies.
Jerzy

------------------------------
Jerzy Konarski

Original Message:
Sent: Fri July 01, 2022 01:40 AM
From: Marcin Molak
Subject: DataStage Metadata usage upgrade

Have you deleted whole link? Maybe another link is copy of the previous one and column definition still referred to table definition of dataset.

------------------------------
Marcin Molak

Original Message:
Sent: Thu June 30, 2022 09:48 AM
From: Jerzy Konarski
Subject: DataStage Metadata usage upgrade

Hello,
I commonly use metadata management in DataStage for impact analysis for code changes.
Currently I need to remove access to a dataset. For this I use the repository "find where used" function.
After deleting access to my DataSet from my job and compiling it, the "find where used" is still refers to the deleted dataset.
Is there a way to clean/regenerate the dependencies to have it working properly?

------------------------------
Jerzy Konarski
------------------------------

#DataIntegration

Metadata information is initialized by DataStage when I insert a datasource into the job.
Afterwards this information is propagated by DataStage also.
So when I change the data source for a column the software should be able to correct this information. Otherwise the information on the lineage is wrong and it is clearly a bug.
I think so.

------------------------------
Jerzy Konarski
------------------------------

Original Message:
Sent: Fri July 01, 2022 03:21 AM
From: Marcin Molak
Subject: DataStage Metadata usage upgrade

I am aware of that. Nevertheless there is no option for DS to check what sort of metadata you put on link and I don't personally think that IBM will change it.

------------------------------
Marcin Molak

Original Message:
Sent: Fri July 01, 2022 02:59 AM
From: Jerzy Konarski
Subject: DataStage Metadata usage upgrade

Hi Marcin,
Indeed the reference to the source that I no longer use is conveyed by the link. You can see it clearly in the export file. The incorrect source is referenced in the transformer and propagated to the output.
Changing the column content in transformer doesn't change anything, the fake reference is still there. The only way to get rid of it is to remove the column from wherever it is referenced and recreate it. What is quite painful, the metadata must constitute an aid in the development and not a constraint.
From my point of view it's a DataStage bug, at some point the references are no longer correct. In the absence of proper functioning, it would be necessary at least to have a function which makes it possible to regenerate the dependencies.
Jerzy

------------------------------
Jerzy Konarski

Original Message:
Sent: Fri July 01, 2022 01:40 AM
From: Marcin Molak
Subject: DataStage Metadata usage upgrade

Have you deleted whole link? Maybe another link is copy of the previous one and column definition still referred to table definition of dataset.

------------------------------
Marcin Molak

Original Message:
Sent: Thu June 30, 2022 09:48 AM
From: Jerzy Konarski
Subject: DataStage Metadata usage upgrade

Hello,
I commonly use metadata management in DataStage for impact analysis for code changes.
Currently I need to remove access to a dataset. For this I use the repository "find where used" function.
After deleting access to my DataSet from my job and compiling it, the "find where used" is still refers to the deleted dataset.
Is there a way to clean/regenerate the dependencies to have it working properly?

------------------------------
Jerzy Konarski
------------------------------

#DataIntegration