IBM webMethods Hybrid Integration

IBM webMethods Hybrid Integration

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
Expand all | Collapse all

Handling Large Flat file Urgent

webMethods Community Member

webMethods Community MemberMon June 20, 2011 07:38 AM

  • 1.  Handling Large Flat file Urgent

    Posted Sat September 11, 2004 09:16 PM

    Guys,
    I am trying to parse Large Flat file. I am using convertToValues in REPEAT loop to parse the file. I understand that I need to EXIT the repeat when ffIterator variable is null. I am not able to set this condition.

    Can someone help me how to check ffIterator for null and then EXIT the repeat.

    Thanks,
    Brain.


    #Integration-Server-and-ESB
    #webMethods
    #Flow-and-Java-services


  • 2.  RE: Handling Large Flat file Urgent

    Posted Sun September 12, 2004 01:29 AM

    Brain,

    How much large is your flatfile?
    Since i have parsed 28-30MB with out any problems using wm.flatfile:convertToValues,i didnt set ffIterator param at all.

    Just schedule the service or directly run the service with out step thru.

    HTH,
    RMG


    #webMethods
    #Integration-Server-and-ESB
    #Flow-and-Java-services


  • 3.  RE: Handling Large Flat file Urgent

    Posted Sun September 12, 2004 09:07 PM

    Hi,
    My fine is about 20-25 MB. ffIterator is an object. How do you compare if the object is null.

    Thanks,
    Brain.


    #webMethods
    #Flow-and-Java-services
    #Integration-Server-and-ESB


  • 4.  RE: Handling Large Flat file Urgent

    Posted Mon September 13, 2004 12:25 AM

    you can ignore setting ffIterator in the convertToValues,25MB and more size file will straight away can parse…

    HTH,
    RMG


    #Integration-Server-and-ESB
    #Flow-and-Java-services
    #webMethods


  • 5.  RE: Handling Large Flat file Urgent

    Posted Tue September 14, 2004 04:47 PM

    Brain,

    You don’t need to explicitly set ffIterator. After your pub.flatFile:convertToValues, set up a BRANCH on ffIterator. Underneath the BRANCH, put in an EXIT with (Label = $null), (Exit from = $loop), and (Signal = SUCCESS).

    Make sure your REPEAT is set to Repeat On Success with a count of blank or -1 and your convertToValues has iterate set to “true.”

    Dave


    #Integration-Server-and-ESB
    #Flow-and-Java-services
    #webMethods


  • 6.  RE: Handling Large Flat file Urgent

    Posted Tue September 14, 2004 05:00 PM


  • 7.  RE: Handling Large Flat file Urgent

    Posted Tue September 14, 2004 05:00 PM


  • 8.  RE: Handling Large Flat file Urgent

    Posted Thu January 03, 2008 05:19 PM

    Thanks Dave. I had the same problem in utilizing the ffIterator properly. One look at this thread and your reply has solved it.


    #Integration-Server-and-ESB
    #Flow-and-Java-services
    #webMethods


  • 9.  RE: Handling Large Flat file Urgent

    Posted Mon June 20, 2011 07:38 AM


  • 10.  RE: Handling Large Flat file Urgent

    Posted Fri August 19, 2011 04:38 PM

    Hi Dave,

    I am doing exactly same what you have suggested. But in this way, I am missing to process last record of the file as it is exiting from the flow as ffIterator comes null when convertToValues pull last record of the file. How should I fix it? Please suggest.

    TIA,
    Shweta


    #Flow-and-Java-services
    #Integration-Server-and-ESB
    #webMethods


  • 11.  RE: Handling Large Flat file Urgent

    Posted Fri August 19, 2011 05:24 PM

    Either process the last record after the loop is exited, or change your loop logic to put the test to exit after processing the current record.


    #Flow-and-Java-services
    #Integration-Server-and-ESB
    #webMethods


  • 12.  RE: Handling Large Flat file Urgent

    Posted Fri August 19, 2011 05:29 PM

    Yeah, I was also thinking of processing that record after exit, just wanted to be sure if there is any better way of doing it.

    Thanks for your reply! :slight_smile:


    #Flow-and-Java-services
    #Integration-Server-and-ESB
    #webMethods


  • 13.  RE: Handling Large Flat file Urgent

    Posted Thu March 01, 2012 03:16 PM

    Hi RMG,

    Can you elaborate when you say "directly run the service with out step thru.


    #webMethods
    #Integration-Server-and-ESB
    #Flow-and-Java-services


  • 14.  RE: Handling Large Flat file Urgent

    Posted Thu March 01, 2012 07:41 PM

    I meant via scheduled job instead of manual running service/debug mode from developer:

    HTH,
    RMG


    #Flow-and-Java-services
    #webMethods
    #Integration-Server-and-ESB