Originally posted by: MarieWallace

I've been working with ITX for a few months processing a variety of input formats (EDI,CSV, ...) and transforming into output JSON. During this time I've run into issues with invalid JSON being generated by ITX, and the worst part of this is that it doesn't tell you it had an issue so you don't know that the map failed until the consuming application validates the output JSON and identifies that it's invalid. Are these known issues/bugs, and/or is there a workaround or different approach I should take in mapping to JSON?

1. Input data with special characters results in invalid JSON.
   I can in theory workaround this issue by "Using item restrictions to escape special characters" (thanks to PaulBrett), but it's hugely time consuming to add checks into every single field in my input and to keep track of all the types of character combinations that invalidate JSON. Is there a better way of ensuring ITX generates valid JSON?
    
2. Input data with empty fields, where ITX doesn't properly close the output JSON. This specifically happens if I have embedded objects, specifically if you've an array which frequently doesn't get properly closed. For example;
   "organizations" : [
       {
           "profile" : {
               "id" : 1457329351,
               "name" : {
                   "firstname" : "John",
                   "lastname" : "Doe"
               }
           }
       }
   ]
   This is a really horrible problem as it's really hard to find. I've tried to use PRESENT(...) conditionals all over my maps and to put "dummy - always there" objects into the output JSON which I can use to force array to close.

Has anyone else run into either of these issues? Are there fixes or other ways of addressing these issues?

Originally posted by: PaulBrett

Would you say the attached accurately shows the issue you are describing?

If not, please alter this testcase to show the issue, and re-post it.

Thank you.

Paul

Follow me on Twitter

Attachment(s)

JSON.zip   7 KB 1 version

Originally posted by: MarieWallace

I'm happy to generate a sample map/type system; I'll upload something tomorrow. However, just to let you know, the JSON that your map has generated is invalid JSON.

Originally posted by: MarieWallace

Thanks so much for your help with this Paul, I really appreciate it. Here is an updated example folder that reproduces the problem; I've also fixed the JSON issue in the original folder, it just required a minor tweak of the JSON schema.

Thanks again, you are a life saver :-) 

Attachment(s)

JSON.Marie.zip   9 KB 1 version

Originally posted by: PaulBrett

I thought perhaps the input Typetree was wrong, and if I made the components optional (range 0:1) then we would see the correct output.  Then I tried some EITHER() rules, but no dice.

I think you should raise this as a Case through your IBM Support contract (if you have one).

Thank you.

Paul

Follow me on Twitter

Originally posted by: MarieWallace

Thanks for taking a look at it Paul, it was totally wrecking my head and I was completely stumped, so I'm glad that it's not just me and this is more than a simple mistake in my type system. Have a good day, Marie