IBM Sterling Transformation Extender

 View Only
  • 1.  Reading XML file with special characters, XSD vs Typetree

    Posted Fri May 03, 2024 11:46 AM

    Hi!

    I am trying to read a XML file with special characters. I have an XSD, and from that XSD i created a Typetree. 
    My problem is that when I read the file using the TypeTree definition, the file will not validate. I get an "One or more inputs are invalid" error. 
    When I read the file using only the XSD in my Input Card, the file validates and processes. 

    Is it anyone who knows if there are any issues / differences when using Typetree instead of XSD? 

    I'll be happy for any input that can point me in the right directon! For as you might have understood I would really like to use Typetree, rather than the XSD.  

    If more info is needed, just ask! I'll be happy to provide! 

    Possibly useful information:

    - The XML tag that's causing the trouble contains something like this: <Nm>Branåsøra AS -> MRK:130 + firma</Nm>
       So as you can see both Norwegian special characters and an ">" inside the Nm-tag
    - I use ISO-8859-1 (Latin1) as charset in my Typetree and "Schema Validation" in DocVerification->XML/JSON in Input Card properties
    - ITX version 10.1.0.



    ------------------------------
    Jan Thomas Flaate
    ------------------------------


  • 2.  RE: Reading XML file with special characters, XSD vs Typetree

    Posted Fri May 03, 2024 11:55 AM

    Hi,

    XML has a special set of characters that cannot be used in normal XML strings.

    These characters are: 

    & - &amp; 
    < - &lt; 
    > - &gt; 
    " - &quot; 
    ' - &#39; 

    Try to replace the > with &gt;  and see if the data validates.


    Gertie. 



    ------------------------------
    Gertie Kearney
    ------------------------------



  • 3.  RE: Reading XML file with special characters, XSD vs Typetree

    Posted Wed May 08, 2024 06:58 AM

    Hi Gertie! Thanks for helping out! 

    Your solution works. But in my case the in-file is part of a value chain that I don't have control over. I have no possibility to change the characters before the file is processed by my mapping. 
    I did make a solution where I read the file with XSD, instead of a TypeTree, and writes it as a string. This actually did what you suggested. I got a file with "&xxx" instead of the special characters. The drawback with this solution is that some of the files are big(!) and it actually takes time to process the data two times. (...and this approach introduced some other problems too. So I discarded this solution after some testing).
     



    ------------------------------
    Jan Thomas Flaate
    ------------------------------



  • 4.  RE: Reading XML file with special characters, XSD vs Typetree

    Posted Fri May 03, 2024 11:57 AM
    Edited by Rex Chan Fri May 03, 2024 12:01 PM

    The direction of processing XML is to use native XSD.  It's been available for several years. 

    It's the same for JSON, use native JSON template or schema.

    In addition, the XML error message is so much better than the regular trace.  It tells you exactly what's wrong and gives you the location of the error.  Usually I see them at the end of the trace.  There is also map function to externalize the XML trace in the map rule.  This would allow you to put the XML error in an output card.

    ------------------------------
    Rex Chan
    ------------------------------



  • 5.  RE: Reading XML file with special characters, XSD vs Typetree
    Best Answer

    Posted Mon May 06, 2024 10:13 AM

    Hi,

    may be you can try this.
    While importing the XSD to create typetree try importing via xerces parser instead of classic  parser and keep xsd in the workspace.
    when xsd is imported via classic parser i see that the item properties are defined with initiators and terminators with > tagname and </tagname

    when xsd is imported via xerces parser i see its not the case.



    ------------------------------
    Bheeshma P
    ------------------------------



  • 6.  RE: Reading XML file with special characters, XSD vs Typetree

    Posted Wed May 08, 2024 06:44 AM

    Hi Bheeshma!
    I tried your  solution, and it seems to work! Thanks! 

    I'm not so familiar with XML-parsing, so if you, or anyone, could explain to me what is the difference between Classic and Xerces parsing I would be happy. 
    As you commented Bheeshma, the Xercses parser seem to treat tags in a different way, but I also notice that my new Xerces Typetree has a slightly different structure than the Classic one. Probably there are more differences, but that's what I have detected so far.

    Until now we have always used Classic when importing XSD's to TypeTrees in my company. Not sure why we have done so, maybe default behavior in ITX!?
    Would it be better to use Xerces in the future? 



    ------------------------------
    Jan Thomas Flaate
    ------------------------------



  • 7.  RE: Reading XML file with special characters, XSD vs Typetree

    Posted Wed May 08, 2024 09:38 AM

    Hello,

    Good to hear that it worked.
    below details might help you. these i got from WTX 8.2 REDBOOK.

    XML parsing WTX diffreces

    Also more detailed explanation present in REDBOOK below. refer to page numbers 384 to 391 and 76.

    https://www.redbooks.ibm.com/redbooks/pdfs/sg247693.pdf



    ------------------------------
    Bheeshma P
    ------------------------------