IBM Sterling Transformation Extender

Sterling Transformation Extender

Come for answers, stay for best practices. All we're missing is you.

 View Only
  • 1.  Best practices for output specification

    Posted Wed August 23, 2023 03:55 PM
    Edited by Nout Hoozemans Wed August 23, 2023 04:02 PM

    Good morning!

    I have some questions cq. opinions regarding (input and) output specification for ITX maps.

    Given a landscape of fairly simple transformations, where for instance one or two layer XML is transformed into a similarly layered JSON for POSTing to a RESTful APIs, and supposing that we get output specifications from our back end suppliers in the form of model JSON documents (eg. comprehensive/representative sample messages), I would say that it's best for version control and maintenance to keep as close to the source of truth as possible, with as few layers of specification between that source and the implementation as possible, and use the JSON as a type doc *directly*, rather than produce an MTT for that JSON.

    I know that there may be cases that necessitate producing an MTT, but I'd say that using JSON docs is a pretty solid *guideline*. You keep close to the RFC, you eliminate complexity of maintenance by limiting sources of truth, and improve transparency and maintainability (especially in the context of version control) by not using a binary as type specification.

    That was the opinion part. Here comes the question part:

    Am I wrong, and if so, why?

    Thanks for your response!



    ------------------------------
    Nout Hoozemans
    ------------------------------



  • 2.  RE: Best practices for output specification

    Posted Thu August 24, 2023 10:24 AM

    Let me make sure I understand.  The idea is to use a JSON Template or JSON Schema (relatively new) in the input or output card instead of a Type Tree (.mtt).   I agree with this approach.

    As you know, you can use XML XSD in the input or output card directly instead in the last few years.  This eliminates the effort to Import the XSD as a type tree.  When there is a change in XSD, it will be reflected in the map.

    The same technique can also be applied to JSON Template or JSON Schema in v10.x (JSON Schema requires the latest release 10.1.x).  No importer is required.  When there is a change in the Template or Schema, it will be reflected in the map.

    Just to clarify what is a JSON Template.  It's sample data..  The Template however should consider all JSON objects, not just some objects.  This is so that the map can parse any object in the real data.



    ------------------------------
    Rex Chan
    ------------------------------



  • 3.  RE: Best practices for output specification

    Posted Thu August 24, 2023 10:46 AM
    Edited by Nout Hoozemans Thu August 24, 2023 10:48 AM

    Hi Rex,

    Thanks for responding!

    The idea, as you say, is to use JSON Templates -- a JSON sample message, fully populated with representative values, but with only a single item populating each array, as the type specification.

    I'm considering three options for specifying JSON messages:

    1. Manually create an MTT (or manually modifying an MTT generated by importing a JSON)
      My concerns here are:
      1. Changes are difficult to track, as the MTT is binary. You could export the MTT as XML, but then you'd have another separate document, even further removed from the source doc.
      2. There is a million different approaches for specifying the same JSON structures in an MTT. Each developer will have a different take on how to encode them.
      3. Complexity. No matter how fluently you speak MTT, it's still more complex and less transparent than a simple JSON message.
      4. The type spec is too far removed from the source document.
    2. Generating an MTT by importing a JSON, and using this without modification.
      1. and c. as above. In fact, the MTTs generated by importing JSON are not always a joy to work with.
    3. Using JSON Template messages directly
      1. Possible performance issues? I'm not sure how performant these generated MTXs are compared to either generated or manually constructed MTT's.

    Are there any other considerations pertaining to these three options I could put in this summary?

    Thanks!



    ------------------------------
    Nout Hoozemans
    ------------------------------



  • 4.  RE: Best practices for output specification

    Posted Thu August 24, 2023 11:57 AM
    Edited by Rex Chan Thu August 24, 2023 11:58 AM

    Performance should be the same regardless whether you are using JSON Template or imported Type Tree.

    I can't comment on manually created Type Tree as I probably won't recommend it for JSON or XML.

    Do make sure the JSON Template is not a large file as it could impact the map size.  JSON Template should only be sample data that represents all the possible objects.  I believe you need at least 2 repeating groups/element for multiple occurrences as it would affect the group Range (like 1:s).



    ------------------------------
    Rex Chan
    ------------------------------



  • 5.  RE: Best practices for output specification

    Posted Thu August 24, 2023 12:08 PM

    Hi Rex,

     I believe you need at least 2 repeating groups/element for multiple occurrences as it would affect the group Range (like 1:s).

    So far I haven't had any problems with JSONs such as this

    {
        "myList" : [
            {
                "fieldA" : "",
                "fieldB" : 0,
                "fieldC" : false
            }
        ]
    }

    The resulting map serializes and deserializes multiple iterations (or none) of the object in <myList> no problem.

    Thanks!



    ------------------------------
    Nout Hoozemans
    ------------------------------



  • 6.  RE: Best practices for output specification

    Posted Thu August 24, 2023 01:12 PM

    Not wrong.  It is a matter of scenario as to which approach might be best for your business application. I like to use the schema or the json directly instead of building a type tree when possible for the exact reasons you mention.

    One reason to NOT do this is the abilty to add restarts for some automatic error handling within the map using ISERROR() or CONTAINSERRORS() which only work with the restart function AND you can only add the restart on a type tree vs using the xml/json schema itself.  If you are using Launcher or Flow Server - the alternative is to use a simple map that uses VALIDATE() to handle / kick out malformed content vs handling within the translation map itself if you use the direct schema/doc spec instead.

    Hope this helps!



    ------------------------------
    Lisa Edwards
    Software Engineer / Subject Matter Expert
    Rainbow Data Systems, Inc
    ------------------------------



  • 7.  RE: Best practices for output specification

    Posted Thu August 24, 2023 02:06 PM

    I believe we support Restart in JSON now.  This is captured from my 10.0.3 Design Studio.

    Before applying Restart:

    Applying Restart

    Restart applied (the icon changes)

    Can somebody write a simple map to check this feature out?

    It should work with 10.0.3 and above.



    ------------------------------
    Rex Chan
    ------------------------------



  • 8.  RE: Best practices for output specification

    Posted Tue August 29, 2023 09:45 PM

    Yes, I understand.  I was telling Nout that the preference to using a type tree generated from the JSON schema vs using the schema itself would be the ability to add appropriate restarts on the TYPE TREE for error handling if needed.  You cannot add a restart on a JSON or XML schema used directly.  So depending on error handling needs may drive whether he creates the type tree or uses the schema directly.



    ------------------------------
    Lisa Edwards
    Software Engineer / Subject Matter Expert
    Rainbow Data Systems, Inc
    ------------------------------