Cloud Pak for Data

 View Only

  • 1.  CP4D v3 - Missing chunking feature for Hierarchical Data

    IBM Champion
    Posted Mon July 13, 2020 10:23 AM
    Hi

    I've setup the DataStage Enterprise Plus service in my Cloud Pak for Data v3.0.1 environment - and discovered what appears to be missing functionality.

    I'm working through this tutorial to test out the environment: 
     https://www.ibm.com/support/knowledgecenter/SSZJPZ_11.7.0/com.ibm.swg.im.iis.ds.stages.xml.core.usage.doc/topics/example2_parent_composingjsondata.html

    When I get to there step where I create views on the schema, there is no way to define that a schema item should be "chunked" in the view. This chunking capability is essential as without it there is no way to may composed JSON fragments to a chunk of the schema.

    This is what I'd see in DataStage...


    This is what I see in the Cloud Pak...


    In DataStage there is no "Chunk" column for the schema items.

    Is this really missing? I can't process my JSON without it!

    I've referred to an IBM tutorial above, but I have real jobs in DS 11.7 with this capability that will not work in the Cloud Pak without it.

    Any help much appreciated.






    ------------------------------
    Greg Hodgkinson
    CTO
    Prolifics
    ------------------------------

    #CloudPakforDataGroup


  • 2.  RE: CP4D v3 - Missing chunking feature for Hierarchical Data
    Best Answer

    IBM Champion
    Posted Mon July 13, 2020 03:56 PM
    If anyone else hits this issue it looks as if this is by design - and might actually be a simplification of how things work.

    I've managed to implement the tutorial without chunking. Instead of composing a JSON chunk as a string in one compose step and then mapping that string to a chunk in a subsequent step, you can now do away with the earlier step and just map directly in a single step.

    I've implemented the full tutorial in CP4D v3 and will upload it to GitHub if anyone is interested in the solution.

    ------------------------------
    Greg Hodgkinson
    CTO
    Prolifics
    ------------------------------

    Original Message


  • 3.  RE: CP4D v3 - Missing chunking feature for Hierarchical Data

    Posted Tue July 14, 2020 03:36 AM
    Hi,

    Please share the GitHub link.

    Thanks

    ------------------------------
    TOMASZ HANUSIAK
    ------------------------------

    Original Message


  • 4.  RE: CP4D v3 - Missing chunking feature for Hierarchical Data

    IBM Champion
    Posted Tue July 14, 2020 03:52 AM
    I've uploaded the code here: https://github.com/ghodgkinson/json_hjoin_example-in-CP4D-v3

    ------------------------------
    Greg Hodgkinson
    CTO
    Prolifics
    ------------------------------

    Original Message