Hi Team, would like to get some help and input on the current issue I am facing in my project. We are trying to build a reusable job for loading into Cassandra. However we run into an issue wherein when running on RCP the key column check-box is not being carried over causing the load to fail. Below is the process.
Job1 - a transform job that creates a load ready dataset. This is NON-RCP job. The PK column(same as on the target table) is defined as a key on the output dataset(being the Key check box is checked).
Job2 - a reusable load job to Cassandra running on RCP. This job is failing because the Cassandra connector is looking for a primary key. This is the key defined on the Job1.
Test1 - I created a non reusable load job(NON-RCP) where the columns are defined and the key is checked and is working fine.
Source -> copy -> Cassandra Connector. This is working fine.
Test2 - I create another job with a combination of NON-RCP and RCP stagest. The last 3 stages are:
copy stage(NON-RCP, column are defined together with the key) -> copy(RCP, no columns defined -> Cassandra Connector. This test is failing as Cassandra connector is looking for the key
Is this an RCP thing or a Cassandra connector issue? Can anyone help me on this issue if you have run into this problem as well.
------------------------------
Ajie LIM
------------------------------