IBM webMethods Hybrid Integration

IBM webMethods Hybrid Integration

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
Expand all | Collapse all

Parallel Processing in Webmethods Developer

  • 1.  Parallel Processing in Webmethods Developer

    Posted Mon February 03, 2014 06:31 AM

    Hello,

    I am currently using webMethods developer. I have a parent flow service and 2 child services are invoked sequencially from this parent service. I have a requirement to invoke these 2 child services parallely since I need not wait for the response of 1st service to invoke the second service. Also there will be performance issue if invoked sequentially. Could you please help me how can I invoke these 2 services parallely from parent flow service?

    Any help on this would be greatly appreciated.

    Regards,
    Rashmi


    #Integration-Server-and-ESB
    #webMethods
    #Flow-and-Java-services


  • 2.  RE: Parallel Processing in Webmethods Developer

    Posted Mon February 03, 2014 10:49 AM

    You can invoke a parallel using a seperate thread using service.doThreadInvoke() check this service API…

    Or create a custom JS for invoke call as below:

    IDataCursor pipelineCursor = pipeline.getCursor();
    String svc = IDataUtil.getString( pipelineCursor, “svc” );
    IData input = IDataUtil.getIData( pipelineCursor, “input” );
    if ( input != null)
    {
    input = IDataUtil.clone(input);
    }else
    {
    input = IDataFactory.create();
    }
    ServiceThread svcThread = Service.doThreadInvoke(NSName.create(svc),input,0);
    IDataUtil.put(pipelineCursor, “svcThread”, svcThread );
    pipelineCursor.destroy();

    HTH,
    RMG


    #Flow-and-Java-services
    #Integration-Server-and-ESB
    #webMethods


  • 3.  RE: Parallel Processing in Webmethods Developer

    Posted Wed March 18, 2015 11:14 AM

    Hi RMG,

           Thanks for the code. Please let me know how to get the Results of the invoked services i.e., child services to be returned to the Main service i.e., Parent service. Thanks in advance.
    

    Regards,
    vinod kumar.


    #webMethods
    #Integration-Server-and-ESB
    #Flow-and-Java-services


  • 4.  RE: Parallel Processing in Webmethods Developer

    Posted Fri April 10, 2015 10:57 AM

    Call your child service as ‘transformer’ in map step. All services that you define in transformers runs parallely and returns the output to parent service.

    -Senthil


    #Flow-and-Java-services
    #Integration-Server-and-ESB
    #webMethods


  • 5.  RE: Parallel Processing in Webmethods Developer

    Posted Fri April 10, 2015 07:23 PM

    I think this is not correct. Services called via transformers in a map step are called in an unpredictable order, but still in one thread, i.e. sequentially. To implement a true parallel processing you should implement your own threading etc. Read up on the internet (Java resources) about how to do that. Or, if you don’t have to remain within the IS, you could create a BPM process with parallel branches.


    #Flow-and-Java-services
    #Integration-Server-and-ESB
    #webMethods


  • 6.  RE: Parallel Processing in Webmethods Developer

    Posted Mon April 13, 2015 03:09 AM

    Thanks Guys for the reply.

    I found a solution for my query. Its worked for me. Please have a look at the following code.

    try
    {
    IDataCursor pipelineCursor = pipeline.getCursor();
    String a = IDataUtil.getString(pipelineCursor,“a”);
    String b = IDataUtil.getString(pipelineCursor,“b”);
    IData results = null;
    com.wm.app.b2b.server.ServiceThread svcThread =Service.doThreadInvoke(“VINODPOC.services”,“javaService_A”,results);
    IData results1 = null;
    com.wm.app.b2b.server.ServiceThread svcThread1 =Service.doThreadInvoke(“VINODPOC.services”,“javaService_B”,results1);
    results = svcThread.getIData(); // Getting Results from The Invoked Child Service 1.
    results1 = svcThread1.getIData(); // Getting Results from The Invoked Child Service 2.
    String c = a + b;
    IDataUtil.put( pipelineCursor,“c”,c);
    IDataUtil.put(pipelineCursor, “svcThread”, results );
    IDataUtil.put(pipelineCursor, “svcThread1”, results1 );
    pipelineCursor.destroy();
    }
    catch(Exception e)
    {

    }

    javaService_A:

    try
    {

                     IDataCursor pipelineCursor = pipeline.getCursor();
    com.wm.util.JournalLogger.log( com.wm.util.JournalLogger.INFO,
    com.wm.util.JournalLogger.FAC_FLOW_SVC,
    com.wm.util.JournalLogger.DEBUG,
    "Method in service A before SLEEP" );
    Thread.sleep(10000);
    com.wm.util.JournalLogger.log( com.wm.util.JournalLogger.INFO,
    com.wm.util.JournalLogger.FAC_FLOW_SVC,
    com.wm.util.JournalLogger.DEBUG,
    "Method in service A AFTER SLEEP" );
    
    IDataUtil.put( pipelineCursor,"Result A","Result A");
    pipelineCursor.destroy();
    

    }
    catch(Exception e)
    {

    }

    javaService_B:

    try
    {

                     IDataCursor pipelineCursor = pipeline.getCursor();
    com.wm.util.JournalLogger.log( com.wm.util.JournalLogger.INFO,
    com.wm.util.JournalLogger.FAC_FLOW_SVC,
    com.wm.util.JournalLogger.DEBUG,
    "Method in service B Before Sleep" );
    Thread.sleep(10000);
    com.wm.util.JournalLogger.log( com.wm.util.JournalLogger.INFO,
    com.wm.util.JournalLogger.FAC_FLOW_SVC,
    com.wm.util.JournalLogger.DEBUG,
    "Method in service B AFTER SLEEP" );
    IDataUtil.put( pipelineCursor,"Result B","Result B");
    pipelineCursor.destroy();
    

    }
    catch(Exception e)
    {

    }

    Both the services will run parallel and we will get the results back to main service.

    Please correct me if i am wrong any where.

    Thanks & Regards,
    vinodkumar v.


    #webMethods
    #Flow-and-Java-services
    #Integration-Server-and-ESB


  • 7.  RE: Parallel Processing in Webmethods Developer

    Posted Mon April 13, 2015 03:44 AM

    hi fml2,

    You said, unpredictable order, but sequential. Both are contradicting, isn’t it? :slight_smile:

    There is no guarantee what order transformers will execute. This can potentially change in every execution also.
    Transformers will execute only if output of the service is mapped to pipeline
    Transformers only copies specific variables in their inputs, they don’t take complete copy of pipeline.
    Services invoked inside transformers are independant of each other, thus they would run parallel and join back the parent service execution

    -Senthil


    #Flow-and-Java-services
    #webMethods
    #Integration-Server-and-ESB


  • 8.  RE: Parallel Processing in Webmethods Developer

    Posted Mon April 13, 2015 07:46 AM

    Hi Senthil,

    I think fml2 is correct. Even i thought if you invoke child services as a tranformer from main service they can execute in parallel. But it didnt happen. More over the pipeline is not getting passed to child services when i rum them. But when i debug ( like using stepInto ) then the pipeline is getting passed to child services. Please correct me if i am wrong any where. Thanks for your support in advance.

    Thanks & Regards,
    vinodkumar v.


    #Integration-Server-and-ESB
    #Flow-and-Java-services
    #webMethods


  • 9.  RE: Parallel Processing in Webmethods Developer

    Posted Mon April 13, 2015 08:59 AM

    The Tundra package includes a service that lets you asynchronously invoke other services: tundra.service:invoke with a $mode = “asynchronous” invokes a given service on another thread, and returns the ServiceThread object which is invoking the service immediately.

    To retrieve the results of the threaded invoke you can then call tundra.service:join, passing it the ServiceThread, and it will block until the threaded invocation has completed.

    For your use case, I would suggest asynchronously invoking the first (or slowest) of your two services and then invoking the second directly in your parent service. When the second invocation has completed, then call tundra.service:join to wait for the first service to finish if it hasn’t already finished.


    #Integration-Server-and-ESB
    #webMethods
    #Flow-and-Java-services


  • 10.  RE: Parallel Processing in Webmethods Developer

    Posted Mon April 13, 2015 10:04 AM

    Vommi, fml2,
    Apologies for my mistake… You are right…

    Transformer services are executed in sequence though you cannot determine the sequence in which they are called…

    1. I tried this in two ways… by having child services sleep for some time, and calculated total time taken by parent service.

    2. Printed 1000 msgs from both child services, and noticed in server log, the printed messages were not overlapping with each other instead, it printed all msgs of child service 1, followed by child service 2.

    One of the benefit of using transformers is that, if the pipeline is having huge data, while executing child services in transformers, it doesn’t pass the entire pipeline to child, rather pass only the required variables.

    One more typical place where it can be used is, if your services are cache enabled…

    -Senthil


    #Flow-and-Java-services
    #webMethods
    #Integration-Server-and-ESB


  • 11.  RE: Parallel Processing in Webmethods Developer

    Posted Mon April 13, 2015 02:38 PM

    Hi Vinodkumar,

    This is correct. When a transformer is executed, it only takes the variables from the pipeline, which are declared in its input, and it only returns those declared in its output.

    If inputs are modified inside the transformer but are not declared to be outputs, these modifications are not reflected in the resulting pipeline after the transformer. There will be still the same value as before the transformer.

    Transformers are only executed when at least one of their outputs is mapped to the pipeline.

    Regards,
    Holger


    #webMethods
    #Integration-Server-and-ESB
    #Flow-and-Java-services


  • 12.  RE: Parallel Processing in Webmethods Developer

    Posted Mon April 13, 2015 04:46 PM

    It’s good to minimize the usage of transformers in our services.


    #Integration-Server-and-ESB
    #webMethods
    #Flow-and-Java-services