DataPower

DataPower

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
Expand all | Collapse all

Datapower Post XML Parallel calls

  • 1.  Datapower Post XML Parallel calls

    Posted Wed July 12, 2023 02:16 PM

    I have a requirement where 

    1. I receive a JSON request from Front end and I am sending as GET Query params to first backend, Here I am using dp-open url and calling the backend.
    2. From the First backend I receive tokens, Consider if I receive ten tokens then I have to call second backend ten times parallelly  with xml post body  

    I am able to call first backend and get tokens and then I am forming xml but can you please let me know how to post xml body in parallel call?



    ------------------------------
    Sagar Muktha
    ------------------------------


  • 2.  RE: Datapower Post XML Parallel calls

    Posted Wed July 12, 2023 02:24 PM

    When you say parallel, do you mean asynchronous, 10 times, to the second back end?



    ------------------------------
    Joseph Morgan
    ------------------------------



  • 3.  RE: Datapower Post XML Parallel calls

    Posted Thu July 13, 2023 03:03 AM

    Yes, Asynchronous.



    ------------------------------
    Sagar Muktha
    ------------------------------



  • 4.  RE: Datapower Post XML Parallel calls

    Posted Thu July 13, 2023 03:04 AM

    Yes, Asynchronous.



    ------------------------------
    Sagar Muktha
    ------------------------------



  • 5.  RE: Datapower Post XML Parallel calls

    Posted Thu July 13, 2023 03:05 AM

    Yes, Asynchronous.



    ------------------------------
    Sagar Muktha
    ------------------------------



  • 6.  RE: Datapower Post XML Parallel calls

    Posted Thu July 13, 2023 07:14 AM
    Edited by Hermann Stamm-Wilbrandt Thu July 13, 2023 07:15 AM

    With XSLT you cannout do parallel dp:url-open().
    But you could execute 10 asynchronous xform actions with one XSLT sending one request to backend each.

    Easier to do with GatewayScript:
    https://www.ibm.com/docs/en/datapower-gateway/10.0.1?topic=apis-urlopen-module#urlopen.openforcommunicationwithservers


    urlopen.open() is asynchrounous function, returns immediately.
    So you can start as many as you want in say a for loop.

    Each specifies a callback function that deals with the results from backend. 



    ------------------------------
    Hermann Stamm-Wilbrandt
    Compiler Level 3 support & Fixpack team lead
    IBM DataPower Gateways (⬚ᵈᵃᵗᵃ / ⣏⠆⡮⡆⢹⠁⡮⡆⡯⠂⢎⠆⡧⡇⣟⡃⡿⡃)
    https://stamm-wilbrandt.de/en/blog/
    ------------------------------



  • 7.  RE: Datapower Post XML Parallel calls

    Posted Thu July 13, 2023 09:46 AM

    As Hermann mentioned, you're not going to be able to do it in raw XSLT.   There is a way in GatewayScript via looping, which is functionally more straigtforward, but you have to be careful, especially if you must capture each response from the asynchronous calls from the loop.  

    It sounds like you may be sending the JSON request via XML dp:url-open in the original call.  Since you are receiving JSON, it is likely much better to process the original request with GatewayScript so when the response with tokens comes, you can more easily loop through it to call the 2nd back end.   If you need the responses from each of the back end token calls, however, it gets a little more tricky in GatewayScript.

    However, if you must stick with using XSLT via a transform, you can output the response from the transform action, and then use the for-each action, invoking what you need (say a Results action) to the back end for each token. This can then be "wrapped" with an event sync action to handle the various responses.

    It really is just a matter of how your policy is currently configured, if you have full control over changing it, and what you know about using the various built-in actions vs writing code.



    ------------------------------
    Joseph Morgan
    ------------------------------



  • 8.  RE: Datapower Post XML Parallel calls

    Posted Mon July 17, 2023 03:20 AM

    I am able to do parallel calls dynamically with XSLT and event sync action(If I receive twenty tokens from FirstBackend then I am able to call twenty times second back end parallely with post body/XML). When I am triggering more than twenty times to the second back end parallely I am receiving Java socket time out exception or 504 error code.

    Can you please let me know if there is any limitation to do parallel calls in data power?



    ------------------------------
    Sagar Muktha
    ------------------------------



  • 9.  RE: Datapower Post XML Parallel calls

    Posted Mon July 17, 2023 10:37 AM

    There are several things that can be going on here. 

    Where is this Java socket timeout coming from? 

    How much time occurs before the timeout?

    First, make sure the back end can handle what DataPower is sending to it.  That is, it could get bogged down by too many rapid requests.   One way to find out is to not make asynchronous calls and see if the same thing happens when 20 or more requests are made.

    Check DataPower memory, cpu, and load when a large batch of requests are being made.   I doubt this is the problem, but it is worth checking.



    ------------------------------
    Joseph Morgan
    ------------------------------



  • 10.  RE: Datapower Post XML Parallel calls

    Posted Mon July 17, 2023 02:36 PM

    >  I am receiving Java socket time out exception 
    >
    DataPower has no Java, so is Java running on your backend?



    ------------------------------
    Hermann Stamm-Wilbrandt
    Compiler Level 3 support & Fixpack team lead
    IBM DataPower Gateways (⬚ᵈᵃᵗᵃ / ⣏⠆⡮⡆⢹⠁⡮⡆⡯⠂⢎⠆⡧⡇⣟⡃⡿⡃)
    https://stamm-wilbrandt.de/en/blog/
    ------------------------------