DataPower

DataPower

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
Expand all | Collapse all

Calling Backed REST API without manipulating request in IBM Data power

  • 1.  Calling Backed REST API without manipulating request in IBM Data power

    Posted Wed July 10, 2024 02:15 AM
    Edited by Anil Nayak Thu July 11, 2024 05:11 AM

    Dears Team ,

    I need a help for this . I have requirement, i need to build a https request in data power which will call the backend REST API without manipulating the request header and data . Its just like a passthrough , when client called the data power https URL , then datapower just need to send to backend API dynamically as it is , without manipulating the request.

    The backed API is REST API with https protocol . 

    Client <----> Data power <---> Backend API

    Thanks

    Dear Morgan,

    Thank you for your response . I am not using any TLS or TCP proxy . I am using MPGW  as HTTPS handler  and backend URL is dynamic ally calling using urlopen.open module , but the challenge for me is how to pass the "application/x-www-form-urlencoded" body parameter in "data" field . I try to use "apim.getvariable('request.body')" but did not work . Please find my gateway script

    var urlopen = require ('urlopen');
    var apim = require ('apim');
    var hm = require('header-metadata');
    var sm = require('service-metadata');
    var query = require('querystring')
    var list = sm.list();
    var fullHeaders = hm.current.get();
    // Retrieve a json object with all headers in another form
    var fullHeaders = hm.current.headers;
    console.debug("Full Header : "+JSON.stringify(fullHeaders));
    console.debug("Full Service Data : "+JSON.stringify(list));
    console.debug("Protocol Method: "+sm.protocolMethod);
    console.debug("Protocol URL: "+sm.getVar('var://service/URL-in'));
    // define the urlopen options
    var options = {
        
        // if target is https, supply a sslProxyProfile
    target: "https://xxxxxx"+sm.URI,
        sslClientProfile: "client-client-dp-profile",
        method: sm.protocolMethod,
        //headers: fullHeaders,
    accept: 'application/json',
    contentType: 'application/x-www-form-urlencoded',
    xcorrelationId: '45678210',
        timeout: 60,
        //data: apim.getvariable('request.body')
    data: "grant_type=urn:ietf:params:oauth:grant-type:jwt-bearer&assertion=<Need to put the paramter coming from request>"
    };
    session.output.write("Request Body: "+options);
    console.debug("Request Body: "+JSON.stringify(options));
    // open connection to target and send data over
    urlopen.open (options, function (error, response) {
        if (error) {
            // an error occurred during request sending or response header parsing
    console.debug("urlopen connect error: " +JSON.stringify(error));
            session.output.write ("urlopen connect error: " + JSON.stringify(error));
        } else {
            // read response data
            // get the response status code
            var responseStatusCode = response.statusCode;
    var responseReasonPhrase = response.reasonPhrase;
    console.debug("responseStatusCode: " +responseStatusCode);
            if (responseStatusCode == 200) {
                response.readAsBuffer(function(error, responseData) {
                    if (error) {
                        // error while reading response or transferring data to Buffer
                        session.output.write("readAsBuffer error: " + JSON.stringify(error));
                    } else {
                        session.output.write(responseData);
                    } 
                });
            } else {

    //var hm = require('header-metadata');
    var contentType = hm.current.get('content-type');
    // Retrieve a json object with all headers
    var allHeaders = hm.current.get();
    // Retrieve a json object with all headers in another form
    var allHeaders = hm.current.headers;
    session.output.write({
    code: responseStatusCode,
    message: responseReasonPhrase,
    });           
            }
        }
    }); 
    apim.output('application/json');


    ------------------------------
    Anil Nayak
    ------------------------------



  • 2.  RE: Calling Backed REST API without manipulating request in IBM Data power

    Posted Wed July 10, 2024 08:59 AM

    Have you considered just using a TCP Proxy or TLS Proxy?



    ------------------------------
    Joseph Morgan
    ------------------------------



  • 3.  RE: Calling Backed REST API without manipulating request in IBM Data power

    Posted Thu July 11, 2024 05:16 AM
    Edited by Anil Nayak Thu July 11, 2024 05:18 AM

    Dear Morgan,

    Thank you for your response . I am not using any TLS or TCP proxy . I am using MPGW  as HTTPS handler  and backend URL is dynamic ally calling using urlopen.open module , but the challenge for me is how to pass the "application/x-www-form-urlencoded" body parameter in "data" field . I try to use "apim.getvariable('request.body')" but did not work . Please find my gateway script

    var urlopen = require ('urlopen');
    var apim = require ('apim');
    var hm = require('header-metadata');
    var sm = require('service-metadata');
    var query = require('querystring')
    var list = sm.list();
    var fullHeaders = hm.current.get();
    // Retrieve a json object with all headers in another form
    var fullHeaders = hm.current.headers;


    console.debug("Full Header : "+JSON.stringify(fullHeaders));
    console.debug("Full Service Data : "+JSON.stringify(list));
    console.debug("Protocol Method: "+sm.protocolMethod);
    console.debug("Protocol URL: "+sm.getVar('var://service/URL-in'));


    // define the urlopen options
    var options = {
        
        // if target is https, supply a sslProxyProfile
    target: "https://xxxxxx"+sm.URI,
        sslClientProfile: "client-client-dp-profile",
        method: sm.protocolMethod,
        //headers: fullHeaders,
    accept: 'application/json',
    contentType: 'application/x-www-form-urlencoded',
    xcorrelationId: '45678210',
        timeout: 60,
        //data: apim.getvariable('request.body')
    data: "grant_type=urn:ietf:params:oauth:grant-type:jwt-bearer&assertion=<Need to put the paramter coming from request>"


    };


    session.output.write("Request Body: "+options);


    console.debug("Request Body: "+JSON.stringify(options));


    // open connection to target and send data over
    urlopen.open (options, function (error, response) {
        if (error) {
            // an error occurred during request sending or response header parsing
    console.debug("urlopen connect error: " +JSON.stringify(error));
            session.output.write ("urlopen connect error: " + JSON.stringify(error));
        } else {
            // read response data
            // get the response status code
            var responseStatusCode = response.statusCode;
    var responseReasonPhrase = response.reasonPhrase;
    console.debug("responseStatusCode: " +responseStatusCode);
            if (responseStatusCode == 200) {
                response.readAsBuffer(function(error, responseData) {
                    if (error) {
                        // error while reading response or transferring data to Buffer
                        session.output.write("readAsBuffer error: " + JSON.stringify(error));
                    } else {
                        session.output.write(responseData);
                    } 
                });
            } else {


    //var hm = require('header-metadata');


    var contentType = hm.current.get('content-type');
    // Retrieve a json object with all headers
    var allHeaders = hm.current.get();
    // Retrieve a json object with all headers in another form
    var allHeaders = hm.current.headers;
    session.output.write({
    code: responseStatusCode,
    message: responseReasonPhrase,
    });           
            }
        }
    }); 
    apim.output('application/json');

    Call from postman


  • 4.  RE: Calling Backed REST API without manipulating request in IBM Data power

    Posted Thu July 11, 2024 09:09 AM

    It seems you've changed the conditions of the original question.  At first you said:

    >> I have requirement, i need to build a https request in data power which will call the backend REST API without manipulating the request header and data 

    But now you've added:

    >> ...but the challenge for me is how to pass the "application/x-www-form-urlencoded" body parameter in "data" field

    Are you putting this into the request because it is not originally present?

    Honestly speaking, if you truly need to pass it through unmodified, the TCP Proxy is, by far, the most simple of these.  However, if you need to terminate and renegotiate TLS because the certificate on the back end won't be validated by the client, then a TLS Proxy is necessary.   Neither will change anything about the headers or data.

    If you do, in fact, need to change the data, wouldn't the "application/x-www-form-urlencoded" value be found in the Content-Type header of the client request?



    ------------------------------
    Joseph Morgan
    ------------------------------



  • 5.  RE: Calling Backed REST API without manipulating request in IBM Data power

    Posted Fri July 12, 2024 10:38 AM

    Dear Morgan,

    I try with TSL Proxy , but its giving me connection error to backend API , because in TCP Proxy i can not define UserAgent to enable the proxy , to call the backend API i need to enable proxy . So can you please help me how I can do using MPGW . For your information using gateway script its working fine , but one challenge is to read application/x-www-form-urlencoded , currently I hardcode the JWT token , so its working , but I need to do as passThru using MPGW . Kindly help me on this



    ------------------------------
    Anil Nayak
    ------------------------------



  • 6.  RE: Calling Backed REST API without manipulating request in IBM Data power

    Posted Fri July 12, 2024 01:28 PM

    Hi Anil,
    I'm jumping into the conversation a few days late, so let me make sure I understand the problem.

    You have a MPGW that you effectively want to pass thru everything to your API.  I'd assume you're having the MPGW handle security before passing the request on to the API?

    But the problem in the API is that your code that is doing the urlopen.open is trying to set the data option to the inbound request, and that isn't working.  This may be an API issue, not a MPGW issue.  I don't have your yaml file, but by default the inbound request is streamed.  That means that a context.get('request.body'), for which the apim "v5 compatibility module" in v10 APIC will proxy that call and then potentially parse the payload so it would return a v5 like response, will get a null as the GatewayScript policy you're executing may be done in parallel with the inbound message being received.  Try putting a parse policy prior to your GatewayScript policy.  This will read the stream until completion, parse the payload, even if it is a "binary" parse, but given the stream has been read now request.body will have a buffer and the apim.getvariable should return the value of that payload.  Assuming you'll have the form-encoded payload, that function should return a buffer still.  Another mechanism of getting the stream read would be to enable buffering for the API, in which case the request would be fully read before the assembly policies start to execute.

    Best Regards,

    Steve Linn



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 7.  RE: Calling Backed REST API without manipulating request in IBM Data power

    Posted Sun July 14, 2024 07:40 AM

    Dear Steve ,

    Thank you for your help , I figure it out  how to call backend API without manipulating the request, its working now . I did it without wring any XSLT or Gateway script , Did it using MPGW



    ------------------------------
    Anil Nayak
    ------------------------------



  • 8.  RE: Calling Backed REST API without manipulating request in IBM Data power

    Posted Fri July 12, 2024 03:43 PM

    The TCP and TLS proxies use the "default" XMLManager, and thus "default" UserAgent within the configuration domain to handle outbound connectivity (a flaw in DP, IMHO), but it is what it is.  Therefore, if you do use this path, be very precise in the settings you modify.



    ------------------------------
    Joseph Morgan
    ------------------------------



  • 9.  RE: Calling Backed REST API without manipulating request in IBM Data power

    Posted Sun July 14, 2024 07:42 AM

    Dear Morgan,

    Thank you for your help , I figure it out  how to call backend API without manipulating the request, its working now . I did it without wring any XSLT or Gateway script , Did it using MPGW



    ------------------------------
    Anil Nayak
    ------------------------------



  • 10.  RE: Calling Backed REST API without manipulating request in IBM Data power

    Posted Sun July 14, 2024 07:12 PM

    Great news Anil!  Congrats.



    ------------------------------
    Joseph Morgan
    ------------------------------