IBM webMethods Hybrid Integration

IBM webMethods Hybrid Integration

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
  • 1.  Flow service performance issue

    Posted Wed December 31, 2008 11:21 PM

    Hi All,

       My service selects records from DB (in this case 50K) which are PGP encrypted. I then loop over each record, decrypt and append each record to a file. Once all records are appended ...the file is PGP encrypted.
    

    This design is slowing down my process.

    Alternatives considered:

    Select all records and publish each to broker. Trigger invokes concurrent threads which decrypt and append each record to a file (not sure if it leads to file locking problems).

    Any suggestions would be appreciated.

    Regards
    Dave


    #Integration-Server-and-ESB
    #Flow-and-Java-services
    #webMethods


  • 2.  RE: Flow service performance issue

    Posted Thu January 01, 2009 06:46 PM

    How are you performing the PGP decryption? In the past this needed a call to a command shell. Doing so for each record will be slow. Perhaps you could devise a means of decrypting batches of records to avoid the overhead of the command shell invocation.

    Not sure which database you are using. Is there a way to use its stored procedures to perform the PGP decryption?

    Mark


    #Integration-Server-and-ESB
    #webMethods
    #Flow-and-Java-services


  • 3.  RE: Flow service performance issue

    Posted Fri January 02, 2009 05:42 AM

    Hi Mark,

    Thanks for your reply. I am new to PGP, please correct me if i am wrong.
    

    (pub.openpgp:decrypt) service is being used and i did notice that the service is referring to command shell. We are using SQL DB and yes we have an option of Stored procedure encryption/decryption, but we want to see if we can improve wM performance by changing our design. Would you please brief me with the process of decrypting batches of records. Also as i mentioned earlier i have to append all decrypted records to a file and encrypt the whole fine again.

    Do we have problem with decrypt service as i am looping for each record (status,error and output variables) of decrypt service keep adding up in the pipeline for each iteration. Should i be explicitly dropping these variables ?


    #Flow-and-Java-services
    #webMethods
    #Integration-Server-and-ESB


  • 4.  RE: Flow service performance issue

    Posted Fri January 02, 2009 05:11 PM

    I would think that decrypting a batch of records using a stored procedure would be much faster than iterating over the same number of records making a call to the pub.openpgp:decrypt service for each.

    You would need to read up on the PGP command line interface to see how best to decrypt multiple strings in a single call.

    Mark


    #webMethods
    #Flow-and-Java-services
    #Integration-Server-and-ESB


  • 5.  RE: Flow service performance issue

    Posted Fri January 02, 2009 06:34 PM

    Hi Mark,

        Thank's for the advise.
    

    Regards
    Dave


    #webMethods
    #Flow-and-Java-services
    #Integration-Server-and-ESB


  • 6.  RE: Flow service performance issue

    Posted Sat January 03, 2009 10:55 AM

    Save file to hard drive is very slow operation. Please consider using local variables to store decrypted records and save it to file on the end of service.

    You can also consider to use java services with has better performance than webMethods.

    What do you mean about that? Can’t you check how it is working with dropping and without? Then you will know what is better :wink:

    Additional thing that can help you: you can log times (e.g. to server logs) after each step and then check which step takes the longest time

    Regards,
    Lukasz Konkol


    #Integration-Server-and-ESB
    #webMethods
    #Flow-and-Java-services