webMethods

webMethods

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
Back to discussions
Expand all | Collapse all

Process Large Volume XML File

  • 1.  Process Large Volume XML File

    Posted Mon September 06, 2010 07:28 AM

    Hi All , I want to process very high volume xml file say 2 GB in flow service/java service . The requirement is :-

    Fetch the input file from location , transform it and sftp to target system .

    Now the problem is even though if we use NodeIterator to iterate over the nodes 1 by 1 , if we get say 10000 child tag list in it …then flow service takes a long time to loop over 10000 child tag and many time fail…

    Can you please help me better solution to handle large volume of xml ??

    thanks in advance
    Rohit Gupta
    rohitgupta.it@gmail.com


    #webMethods
    #Integration-Server-and-ESB
    #Flow-and-Java-services


  • 2.  RE: Process Large Volume XML File

    Posted Tue September 07, 2010 08:24 AM

    Hello Rohit,

    How are you planning to load this XML? Try loading the xml in stream instead of bytes. Use XML node iterator to iterate over the tags. And while mapping to target format use segment mapping, instead of writing the whole mapping logic in one service.

    Search this forum for large xml handling…

    Cheers,
    Sasanka


    #webMethods
    #Flow-and-Java-services
    #Integration-Server-and-ESB


Related Content