Data Integration

Data Integration

Connect with experts and peers to elevate technical expertise, solve problems and share insights.


#Data
#Data
#Dataintegration
#Artificialintelligence
 View Only

  • 1.  Hierarchical stage - Flow Designer - json libraries resource file structure?

    Posted 10/19/23 11:04 AM
      |   view attached

    I am working with the Hierarchical stage in Datastage flow designer to access SharePoint lists via the RestAPI. I have written some powershell to generate a json schema file from the fields output of the list. However the Hierarchical stage does not like this resource file. All documentation I can find on this states to use the Schema Library Manager, which is flash based and therefore not available to me. I feel like I should be able to manage this through the Flow Designer, rather than the IBM® InfoSphere® DataStage® and QualityStage® Designer, I just need to know what json file should look like. Below is the error that I get when importing my json file.

    Json Resource File Import Error
    I have also attached my attempt at the json schema file. Any help would be greatly appreciated.


    ------------------------------
    Jackson Eyton
    ------------------------------

    Attachment(s)

    json
    SP_TestList_Schema.json   34 KB 1 version


  • 2.  RE: Hierarchical stage - Flow Designer - json libraries resource file structure?

    Posted 10/20/23 04:30 PM
    Hello,
    Which version of Information Server are you using?
     
    There were a couple of known issues with DFD returning the generic "RestApiResponseUnsuccessful 200" error in some situations, because it is missing the default directory or the default project. Please see the links below. The issue that you raised might be related,  and the issues have been fixed.
     
    JR61096: THE DEFAULT DFD DIRECTORY ._DFDESIGNER IN ENGINE TIER WAS NOT CREATED DURING IIS INSTALLATION.
    https://www.ibm.com/support/pages/apar/JR61096
     
    JR61281: DFD API IS LOOKING INTO DEFAULT "DSTAGE1" PROJECT WHEN NO PROJECT IS PASSED TO THE API.
    https://www.ibm.com/support/pages/apar/JR61281
     
    That said, according to this link 
    https://www.ibm.com/support/pages/node/1282666 
    the Datastage flow designer (DFD) approach for the Hierarchical Data stage is for simple use cases.
    Simple use cases can be
    • Parsing and Composing XMLs which can be just imported/referred (cannot apply schema chunk operations).
    Working scenarios:
    • JSON Parser and Composer
    • XML Composer
    • REST 
    • The following transformation steps work fine:
      OrderJoin (change in usability) 
    Non-Working scenarios:
    • XML Parser
    • The transformation steps: Regroup, Switch.

    The error message in your screen shot says there is an Unexpected character. Can you check your json file with a hex editor for any kind of garbage or unprintable characters?

    Regards.


    ------------------------------
    Emily White
    ------------------------------



  • 3.  RE: Hierarchical stage - Flow Designer - json libraries resource file structure?

    Posted 10/24/23 09:12 AM

    Hello again,

    You might be seeing the issue that was fixed by this APAR:
    JR64973: Hierarchical Stage: REST step throws invalid xml error when xml response body contains less than 4 characters in the first line.
    The above APAR is fixed in IIS v11.7.1.4.

    Here is a paste from the README file in that patch:

    PROBLEM DETAILS:
===============
In Hierarchical Data stage, REST Step throws invalid XML error when the received XML content contains 
less than 4 characters in the first line of the body content. This happens because REST Step uses 
the first four characters before a new line for determining the charset.

Following is the error message
CDIER0925E: The REST step found invalid XML data in an HTTP body.

RESOLUTION:
==========
A new java property xmlcc.ignore.invalidXMLError has been added, setting this to true will by pass the
check and assume the charset to be UTF-8 and proceed. Please note that this does not guarantee that 
the resulting XML will be valid and may not be parsed in the XML parser step.


    Regards.



    ------------------------------
    Emily White
    ------------------------------

    Original Message


Related Content