Data Integration

Hi Folks,

We have a client requirement where we are planning to migrate almost 20k Datastage jobs from 11.7 to CPD. As we are building the automating to migrate the jobs, we figured out that easily available method is to migrate the jobs into CPD by using API provided for datastage. To implement these API , we can use it either by curl or implementing it thru Java/Python. 

While the curl command is working fine for us we wanted to use also the Java implementation to understand the pros and cons of the process. But the documentation provided is not sufficient , we are not able to write a workable code which works with Java. Any one has done any similar proof of concept and would like to share your views on this ?

Hi Pradhan,

what do yo want to automate? The migration is (besides the try and error in the beginning to get it right) a onetime process. You export 1 ISX per project either via ISM or ISTOOLS, then you import it in CP4D either via the UI (New Asset) or CPDCTL. It is designed by IBM not be an incremental process (like migrating 10% of a project from 11.7 to CP4D and later the next 10%).

KR Ralf

Hi Folks,

We have a client requirement where we are planning to migrate almost 20k Datastage jobs from 11.7 to CPD. As we are building the automating to migrate the jobs, we figured out that easily available method is to migrate the jobs into CPD by using API provided for datastage. To implement these API , we can use it either by curl or implementing it thru Java/Python. 

While the curl command is working fine for us we wanted to use also the Java implementation to understand the pros and cons of the process. But the documentation provided is not sufficient , we are not able to write a workable code which works with Java. Any one has done any similar proof of concept and would like to share your views on this ?

Thanks Ralf for the insights. After reading your message I tried importing assets using CPDCTL as I assume thru UI it would be again manual process to point to the right isx for import. I tried following the documentation for import from github link https://github.com/IBM/cpdctl/blob/master/README_command_reference.md but looks like to import asset we need to mandatorily have a space id . 

While calling the import api  , we needed only the project id and the asset path but it seems the same doesn't work for cpdctl. Would you be specific on which specific commands you used for isx import ?

As per the documentation On CPD 3.0.1 assets import is supported only in the context of a space.

Hi Pradhan,

what do yo want to automate? The migration is (besides the try and error in the beginning to get it right) a onetime process. You export 1 ISX per project either via ISM or ISTOOLS, then you import it in CP4D either via the UI (New Asset) or CPDCTL. It is designed by IBM not be an incremental process (like migrating 10% of a project from 11.7 to CP4D and later the next 10%).

KR Ralf

Hi Folks,

We have a client requirement where we are planning to migrate almost 20k Datastage jobs from 11.7 to CPD. As we are building the automating to migrate the jobs, we figured out that easily available method is to migrate the jobs into CPD by using API provided for datastage. To implement these API , we can use it either by curl or implementing it thru Java/Python. 

While the curl command is working fine for us we wanted to use also the Java implementation to understand the pros and cons of the process. But the documentation provided is not sufficient , we are not able to write a workable code which works with Java. Any one has done any similar proof of concept and would like to share your views on this ?

Hi Pradhan,

the basic command to import a 11.7 ISX to CP4D should be

cpdctl dsjob migrate -p #TARGET_PROJECT_NAME# -f FILE_NAME.isx 

KR Ralf

Thanks Ralf for the insights. After reading your message I tried importing assets using CPDCTL as I assume thru UI it would be again manual process to point to the right isx for import. I tried following the documentation for import from github link https://github.com/IBM/cpdctl/blob/master/README_command_reference.md but looks like to import asset we need to mandatorily have a space id . 

While calling the import api  , we needed only the project id and the asset path but it seems the same doesn't work for cpdctl. Would you be specific on which specific commands you used for isx import ?

As per the documentation On CPD 3.0.1 assets import is supported only in the context of a space.

Hi Pradhan,

what do yo want to automate? The migration is (besides the try and error in the beginning to get it right) a onetime process. You export 1 ISX per project either via ISM or ISTOOLS, then you import it in CP4D either via the UI (New Asset) or CPDCTL. It is designed by IBM not be an incremental process (like migrating 10% of a project from 11.7 to CP4D and later the next 10%).

KR Ralf

Hi Folks,

We have a client requirement where we are planning to migrate almost 20k Datastage jobs from 11.7 to CPD. As we are building the automating to migrate the jobs, we figured out that easily available method is to migrate the jobs into CPD by using API provided for datastage. To implement these API , we can use it either by curl or implementing it thru Java/Python. 

While the curl command is working fine for us we wanted to use also the Java implementation to understand the pros and cons of the process. But the documentation provided is not sufficient , we are not able to write a workable code which works with Java. Any one has done any similar proof of concept and would like to share your views on this ?