IBM webMethods Hybrid Integration

IBM webMethods Hybrid Integration

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
Expand all | Collapse all

IS crashed while handling Large FlatFile (40MB)

  • 1.  IS crashed while handling Large FlatFile (40MB)

    Posted Tue October 20, 2015 11:20 AM

    Hello RMG/Experts

    We have build a framework to handle large flat file(40MB).But we are facing below problems.

    The first couple Of time are successful and later Automatically restarting itself (2nd time when we put the file in FTP ).

    Could you please suggest me on this. What are the changes I needs to be done.


    #webMethods
    #Integration-Server-and-ESB


  • 2.  RE: IS crashed while handling Large FlatFile (40MB)

    Posted Tue October 20, 2015 01:17 PM

    Are you trying to FTP that large file using pub.client.ftp* services?

    Since this is a huge file, make sure you are transferring it as stream and also try to compress the file along with adjusting the time out settings.

    Assuming your IS MEM settings is capable to handle this large file processing.

    HTH,
    RMG


    #webMethods
    #Integration-Server-and-ESB


  • 3.  RE: IS crashed while handling Large FlatFile (40MB)

    Posted Tue October 20, 2015 03:08 PM

    Yes,I am trying with FTP service and I am transferring Stream data.
    Could you please let me know if the below properties are wrong .

    watt.server.keepAliveTimeOut=15000(C:\SoftwareAG82\IntegrationServer\bin\server.bat).
    tn.BigDocThreshold=
    tn.xml.xqlthreshold=
    tn.space.location=
    tn.tspace.max=52,428,00.


    #Integration-Server-and-ESB
    #webMethods


  • 4.  RE: IS crashed while handling Large FlatFile (40MB)

    Posted Tue October 20, 2015 04:14 PM

    What did you have set this currently and some of these are TN related.

    tn.BigDocThreshold=
    tn.xml.xqlthreshold=
    watt.server.tspace.location =
    watt.server.tspace.timeToLive=

    Basically you need to tune the values pertaining to your environment and IS memory/Resource settings.

    HTH,
    RMG


    #Integration-Server-and-ESB
    #webMethods


  • 5.  RE: IS crashed while handling Large FlatFile (40MB)

    Posted Wed October 21, 2015 11:47 AM

    Can you share some details around the service that actually consumes the file? Are you using an iterator?

    Percio


    #webMethods
    #Integration-Server-and-ESB


  • 6.  RE: IS crashed while handling Large FlatFile (40MB)

    Posted Wed October 21, 2015 12:03 PM

    Hi Percio,

    Yes I am using an Iterator. The service invokes when the file has been put in FTP server and We are transferring data in streams.

    Please help me out on this.


    #webMethods
    #Integration-Server-and-ESB


  • 7.  RE: IS crashed while handling Large FlatFile (40MB)

    Posted Wed October 21, 2015 12:07 PM

    Thanks for the reply,

    Could you please let me know what are the necessary settings I need to check in IS memory and Resource settings.Plese give me some light on this.


    #Integration-Server-and-ESB
    #webMethods


  • 8.  RE: IS crashed while handling Large FlatFile (40MB)

    Posted Wed October 21, 2015 12:11 PM

    Laxman,

    Are you able to share your code and a sample (small) file with us so we can take a look? Also, do you see any errors written to the server.log, wrapper.log, or nohup file prior to the crash?

    Percio


    #webMethods
    #Integration-Server-and-ESB


  • 9.  RE: IS crashed while handling Large FlatFile (40MB)

    Posted Wed October 21, 2015 12:39 PM

    Percio,

    I am sorry,I could not share the code as I am connecting to client machine. And I dont see any errors server log.


    #Integration-Server-and-ESB
    #webMethods


  • 10.  RE: IS crashed while handling Large FlatFile (40MB)

    Posted Wed October 21, 2015 01:15 PM

    Do you see any core/heap dumps in the filesystem folder IntegrationServer/instances/default (if you are on 9.x) or IntegrationServer after the IS crashed?

    HTH,
    RMG


    #Integration-Server-and-ESB
    #webMethods


  • 11.  RE: IS crashed while handling Large FlatFile (40MB)

    Posted Wed October 21, 2015 01:22 PM

    Hi RMG,

    I am working on webMehtods 82 version. Can we see Core/Heap dumps in the filesystem of webMethods 82 ?


    #Integration-Server-and-ESB
    #webMethods


  • 12.  RE: IS crashed while handling Large FlatFile (40MB)

    Posted Wed October 21, 2015 01:23 PM

    Hello Percio,

    I am also facing the same problem.

    Where would I find the “wrapper.log” (it is not in the same directory (\webMethods82\IntegrationServer\logs) as the server.log

    what is a “nohup” file…and how would I find, or generate it ?

    Could you please suggest me.

    Thanks,
    Prem.


    #Integration-Server-and-ESB
    #webMethods


  • 13.  RE: IS crashed while handling Large FlatFile (40MB)

    Posted Wed October 21, 2015 07:31 PM

    Prem,

    I can’t remember if the IS was already running inside the Tanuki wrapper in 8.2. I think it was. I’m running 9.6 currently and the wrapper.log file is located under \profiles\IS_default\logs, but you won’t find yours there. Just run a search for wrapper.log under the Software AG home folder.

    As for nohup, I’m guessing based on your backslashes that you’re running on Windows, right? In that case, it doesn’t apply to you. If you’re still interested though, there’s a ton of info online about when and how to use the nohup command.

    Percio


    #Integration-Server-and-ESB
    #webMethods


  • 14.  RE: IS crashed while handling Large FlatFile (40MB)

    Posted Thu October 22, 2015 07:33 AM

    Prem – You can find nohup file in the same location from where you kick-off the script in general. Which OS are you using.

    Thanks,


    #webMethods
    #Integration-Server-and-ESB