API Connect

 View Only
Expand all | Collapse all

How to invoke target-url / api with x-www-form-urlencoded parameters

  • 1.  How to invoke target-url / api with x-www-form-urlencoded parameters

    Posted Sat February 05, 2022 07:04 AM
    I want to invoke target url with some x-www-form-urlencoded parameters,
    I tried to work around with some gateway scripts didn't worked out in favor.
    Also is it possible to use curl command using gateway script.




    ------------------------------
    Anurag Gupta
    ------------------------------


  • 2.  RE: How to invoke target-url / api with x-www-form-urlencoded parameters

    Posted Wed February 09, 2022 02:06 PM
    Hi Anurag,
    The parameters are simply a posted body of the form
    param1=value1[&param2=value2&etc ...]​

    where you could simply write a GatewayScript policy to construct this string and write it to message.body by using an apim.setvariable ('message.body', yourstring) (v5/v5c) or for the api gateway, context.message.body.write(yourstring).  Don't forget to also set your content type using apim.output(`x-www-form-urlencoded`) (v5/v5c) or context.message.header.set('Content-Type', `x-www-form-urlencoded`) in the api gateway.  The subsequent invoke policy will consume message.body and will send it to your target url.

    Regards,
    Steve



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 3.  RE: How to invoke target-url / api with x-www-form-urlencoded parameters

    Posted Thu February 10, 2022 02:31 AM
    Hi Anurag,

    for the form part, you might use the fonction encodeURIComponent as follows:

    var details = {
        'userName': 'test@gmail.com',
        'password': 'myPwd',
        'grant_type': 'password'
    };
    
    var formBody = [];
    for (var property in details) {
      var encodedKey = encodeURIComponent(property);
      var encodedValue = encodeURIComponent(details[property]);
      formBody.push(encodedKey + "=" + encodedValue);
    }
    formBody = formBody.join("&");
    
    context.message.body.write(JSON.stringify(formBody));​


    ------------------------------
    Pierre Richelle
    IBM Hybrid Cloud Integration Specialists
    IBM
    +32474681892
    ------------------------------



  • 4.  RE: How to invoke target-url / api with x-www-form-urlencoded parameters

    Posted Fri April 29, 2022 02:17 PM
    Is there a way in the API Manager to allow a x-www-form-urlencoded?

    1) I do not want to force my users to change from x-www-form-urlencoded to json
    2) I want to be able to test this in the APIM Assembly (sandbox) but there is no way to add a generic body even if not required is not checked.

    ------------------------------
    Ray Jennings
    ------------------------------



  • 5.  RE: How to invoke target-url / api with x-www-form-urlencoded parameters

    Posted Fri April 29, 2022 05:44 PM
    Hi Ray,
    The original question in this thread was how to send a urlencoded message to a backend.  As I noted earlier in this thread, you can build a simple string in the form
    p1=value1&p2=test%24%24test

    by building your string and then using the encodeURIComponent() function to properly encode the appropriate characters requiring encoding.  Set that string into message.body, set the content type to application/x-www-form-urlencoded in your gateway script.  The subsequent invoke policy will send that to your backend .

    If your question is will the gateway accept a request of this type, I believe you can do this with both v5 and v10.  In v5 request.body should be the inbound form string which invoke should be able to consume.  If you have created a message.body in your API via some previous policy, invoke only uses message.body so you may need to copy request.body to message.body via some gateway script.  Perhaps your issue is with the APIM test tool not sending this request properly?  If you can provide some more detail of what you're encountering that would be helpful.

    Regards,

    Steve



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 6.  RE: How to invoke target-url / api with x-www-form-urlencoded parameters

    Posted Tue May 10, 2022 01:26 PM

    Steve, Thank you for your reply. I am trying to copy the "request.body" to "message.body" (which is just a string: "username=xxx&password=yyy")

    But I am getting an error of: (as shown in dpod)

    Failed to parse 'request.body' as content type 'application/json;charset=UTF-8'.

    My initial API Gateway script to test this is:

    var apim=require('apim');

    apim.setvariable('message.headers.Content-Type', 'application/x-www-form-urlencoded');

    var body = apim.getvariable('request.body');  # Failure is here

    var str_body = body.toString();

    apim.setvariable('message.body', str_body);



    ------------------------------
    Ray Jennings
    ------------------------------



  • 7.  RE: How to invoke target-url / api with x-www-form-urlencoded parameters

    Posted Tue May 10, 2022 03:57 PM

    Hi Ray,

    In v10, the apim module is what I call the "v5 compatibility" module.  The v10 API Gateway context is different than v5 and also has the data in a particular data type (in this case a string), and the apim module's getvariable function will attempt to return to you a response like v5 would, which for a *.body would attempt to return a parsed JSON object if the content type was a JSON type.  The error would indicate that your request Content-Type header was application/json; charset=UTF-8 which obviously doesn't match the content type of the data that the client is providing.  The request Content-Type header should be application/x-www-form-urlencoded.  Two options here:

    1. Have your client provide the proper content-type request header to match the data type of the payload being sent. In this case a payload should be returned from the compatibility module getvariable function, but depending upon the APIC release you are using, it may be a string (which was not how v5 would have returned it) or it will be in a "XML Binary Node" (a recent fix to the compatibility module to return what v5 would have returned for a non-XML / non-JSON payload.  To convert a binary node to a string and make your code safe regardless of the APIC version you're using, you can use the following:

    // is the return from apim.getvariable a binary node, convert it to a string

    if ((body.item && body.length > 0 && body.item(0).nodeType === 13)) {
    body = body.item(0).toBuffer().toString();
    }

    2. Change the GatewayScript to use the native context.get('request.body') or context.request.body.readAsBuffer(function(error, buffer) {});  The latter like many GatewayScript functions has an asynchronous callback function which would receive the data in the buffer variable where you could convert that buffer to a string and then pass it to a function that would continue execution from within the callback.

    Note, in either case, by default, the API Gateway Service streams the request payload which means that the processing of the assembly will start in parallel with the consuming of the request.body, so in your GatewayScript, the request.body may not be available.  To force it to be available, especially with a payload so small where streaming isn't going to provide a tangible performance benefit, you can either a) set the x-ibm-configuration.buffering: true option in your API b) you can provide as your first policy in the assembly a "parse" policy.  Given your content type request header isn't correct, don't use the "Use Content Type" in the parse policy, so the policy will inspect the payload to see the string can't be JSON or XML and will simply read in all of the data before continuing.

    Hope this helps!

    Regards,

    Steve



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 8.  RE: How to invoke target-url / api with x-www-form-urlencoded parameters

    Posted Thu August 11, 2022 11:06 AM

    Hi Pierre,

    I used your script example to propagate form URL-encoded parameters to a backend, because our API Connect is still dealing with this issue (we haven´t upgraded yet): https://www.ibm.com/support/pages/apar/IT38371.
    The formBody variable is constructed correctly, however I still get an error from the backend saying that I do not supply the parameters. 
    formBody variable looks like this:

    'username=XYZ&password=PW&grant_type=password&remote_scheme=XYZ&scope=XYZ&client_id=XYZ'


    This gets sent to the message body via the last line in your script:

    context.message.body.write(JSON.stringify(formBody));​
    


    Is it possible to log the context variable message.body to see what is sent to the backend? How would the correct syntax look like for this?
    And do I need to send anything in my original request body?

    My goal is to supply the url-encoded parameters in formBody to the backend. 

    Thank you for your help

    Henri
    API Connect version: 10.0.1.5.



    ------------------------------
    Henri Unruh
    ------------------------------



  • 9.  RE: How to invoke target-url / api with x-www-form-urlencoded parameters

    Posted Thu August 11, 2022 01:17 PM
    I figured it out myself. Had to remove the JSON.stringify method and just write the formBody variable directly to the message body as in:

                context.message.body.write(formBody);​


    This thread helped nontheless a lot. Thank you!

    ------------------------------
    Henri Unruh
    ------------------------------



  • 10.  RE: How to invoke target-url / api with x-www-form-urlencoded parameters

    Posted Thu August 11, 2022 07:31 PM

    Hi Henri,

    Makes sense, formBody is a string, so a JSON.stringify of a string will add double quotes around that string which the backend wouldn't like.  

    Regards,
    Steve



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 11.  RE: How to invoke target-url / api with x-www-form-urlencoded parameters

    Posted Wed October 12, 2022 04:53 PM
    Edited by Gabriel Marte Blanco Thu October 13, 2022 09:35 AM
    Hi there:

    Is there a way of doing this, in v5 with multipart form-data?

    I tried using this gateway script, all I'm getting is Invalid or unexpected token error.

    var apim = require('./apim.custom.js');  
     
    apim.setvariable('message.headers.Content-Type','multipart/form-data; boundary=--------------------------948993005467454460911999');
    
    apim.setvariable('message.headers.accept','application/json;odata=verbose');
    
    var grant_type = apim.getvariable('api.properties.property1');
    var client_id = apim.getvariable('api.properties.property2');
    var client_secret = apim.getvariable('api.properties.property3');
    var resource = apim.getvariable('api.properties.property4');
    
    var paramAsBlob = '--------------------------948993005467454460911999\n'+'Content-Disposition: form-data; name="grant_type"\n\n'+grant_type+'\n--------------------------948993005467454460911999\nContent-Disposition: form-data; name="client_id"\n\n'+client_id+'\n--------------------------948993005467454460911999\nContent-Disposition: form-data; name="client_secret"\n\n'+client_secret+'\n--------------------------948993005467454460911999\nContent-Disposition: form-data; name="resource"\n\n'+resource+'\n--------------------------948993005467454460911999--' ;
    
    
    var paramAsString = paramAsBlob.toString();
    
    var longitud = paramAsString.length;
    
    apim.setvariable('message.headers.Content-Length',longitud);
    
    apim.setvariable('message.body', paramAsString,'set');​


    ------------------------------
    Joan Hache
    ------------------------------



  • 12.  RE: How to invoke target-url / api with x-www-form-urlencoded parameters

    Posted Fri October 14, 2022 10:02 AM

    Hi Joan, 

    Please see my reply in your other thread.

    Regards,

    Steve



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 13.  RE: How to invoke target-url / api with x-www-form-urlencoded parameters

    Posted Fri October 21, 2022 09:44 AM
    Hi Steve.. I was try from API Connect 10 and v5, but this not working.. I created the gws, and invoque.. I see in the header and body the information, but when I try to test is not answer well. 

    All works fine from curl and postman .  And I put the same key/values from postman.  Is anything else that I need to do?

    var details = {
    'grant_type': 'client_credentials',
    'client_id': 'xxx_clientid',
    'client_secret': 'xxx_clientsecret',
    'resource' : 'xxx_resource'

    };

    var formBody = [];
    for (var property in details) {
    var encodedKey = encodeURIComponent(property);
    var encodedValue = encodeURIComponent(details[property]);
    formBody.push(encodedKey + "=" + encodedValue);
    }
    formBody = formBody.join("&");

    context.message.header.set('Content-Type', `x-www-form-urlencoded`);
    context.message.body.write(formBody);

    ------------------------------
    Oscar Gonzalez
    ------------------------------



  • 14.  RE: How to invoke target-url / api with x-www-form-urlencoded parameters

    Posted Fri October 21, 2022 10:57 AM

    Hi Oscar,

    The content-type should be application/x-www-form-urlencoded.   Otherwise I don't see anything in the above GatewayScript that would be problematic.  If you were testing this in v5 with apim.setvariable instead of the context.message.* you're using for v10, the statement order may be important where you would set the Content-Type header after updating the body.  What type of error are you getting from your backend server?

    Best Regards,

    Steve



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 15.  RE: How to invoke target-url / api with x-www-form-urlencoded parameters

    Posted Tue October 25, 2022 09:29 AM

    Hi Oscar, 

    I received your direct reply via email, as noted above, your content-type is incorrect, yet the API's gatewayscript you sent to me still has

    context.message.header.set('Content-Type', `x-www-form-urlencoded`);

    That value should be changed to application/x-www-form-urlencoded.  You indicated "I need this API to work on V5" but just a friendly reminder that V5 is out of service as of April 30 of this year and will be fully out of service for those with extended service contracts at year end.  Hopefully you were indicating v5 compatible in v10?  If so, then also as noted, your v5 API would of course be using apim.setvariable instead of context.get

    apim.setvariable('message.body', formBody);
    apim.setvariable('message.headers.Content-Type', `application/x-www-form-urlencoded`);
    

    If that still isn't working, please advise what type of error you are getting?

    Best Regards,

    Steve



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 16.  RE: How to invoke target-url / api with x-www-form-urlencoded parameters

    Posted Tue October 25, 2022 02:37 PM

    Hi Oscar,

    I received your reply that you're still being directed to a login page even after changing the content-type header to application/x-www-form-urlencoded.  I ran your GatewayScript code  in a test API and see that message.body is updated appropriately

    body: "grant_type=client_credentials&client_id=xxx_clientid&client_secret=xxx_clientsecret&resource=xxx_resource"

    so message.body does look like a legitimate form post body.  Not knowing exactly what your API is doing, and assuming you've confirmed that posting this same payload using postman will return a successful result, perhaps the backend can help you understand why your request is being rejected.  If you have access to the DataPower appliance, in your v10 apiconnect domain you can also enable the API Probe Settings object.  Then invoke your API using the Test tab of your API.  The response that is returned to the client will be shown, but on the trace tab, you will also be able to see what the context looks like (which is how I provided the message.body above) and message.headers at the beginning of each policy.

    Best Regards,
    Steve



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 17.  RE: How to invoke target-url / api with x-www-form-urlencoded parameters

    Posted Tue October 25, 2022 03:58 PM

    Hi Oscar,

    I'm not sure how you're replying to my post but they are not seen in this thread and those reading it are looking at a one way conversation :-) , but you're now asking how to enable the API probe settings object in DataPower.  Enter probe in the navigation of the WebGui in your apiconnect domain, select the API probe settings link in the navigation, select the enable radio button and apply.  Don't forget to save the configuration too.  You said "I see in the trace tab, that the context looks good I see the message.body and the message headers in the policy." so that sounds like the API probe settings object is already enabled for you??  Are you seeing for the trace something like the following?

    Sample API Probe Trace pageBest Regards,
    Steve


    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------