DataPower

DataPower

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
  • 1.  How to handle the incoming base-64 data in extra filed in Datapower and send as an attachment to backend

    Posted Tue September 19, 2023 10:55 AM

    Hi,

    I've a requirement like in datapower we need to add one extra filed like<attachmentfile>(filed is not in WSDL which is shared by backend team ) to my request  and will share it with front end team(front end service is a rest service) and they will send an attachment in base-64 in that filed send it to datapower. Moreover, within datapower we need to extract that base-64 content and send as an attachment(out of body as backend will have a performance issues) to backend vice versa.

    Any prompt help would be greatly appreciated!

    Thanks,

    Venkateswarlu K.



    ------------------------------
    kandukuri venkateswarlu
    ------------------------------


  • 2.  RE: How to handle the incoming base-64 data in extra filed in Datapower and send as an attachment to backend

    Posted Wed September 20, 2023 02:48 AM

    Hi,

    can you share a bit more details? So consumers are calling DataPower through a REST API? And the backend service is a SOAP Web Service? And if so does the backend expect the attachments as MTOM or SwA? 



    ------------------------------
    Hermanni Pernaa
    ------------------------------



  • 3.  RE: How to handle the incoming base-64 data in extra filed in Datapower and send as an attachment to backend

    Posted Wed September 27, 2023 04:09 PM

    Hi Kandukuri,

    Based upon the email you sent me directly for this, I'm providing my answer here in the forum.

    Its difficult to say what your issue is without also seeing your JSON payload whose schema must match the code you've written, but just a few notes I made in evaluating your code you provided:

    1. Typically the root body of the multipart form is the SOAP message.   Looks like you have the SOAP message as the last part which to me is unusual.
    2. I don't see an ending boundary string with the suffix of two hyphens that follow to indicate the end of attachments after the last part of the form.
    3. As for the attachments themselves, again I don't have the sample request JSON to look at to compare to what the code is doing, but for the following code

                                    var attachmentList = inputJson.ELRASubmitDetailsRequest.AttachmentList.Attachment;

                                    var v2 = '';

                                    if ( attachmentList && attachmentList.length > 0) {

    The accuracy of this code will depend on what the input JSON schema is.   Is the property "AttachmentList" a parent object of property "Attachment" that is an array of objects, and within each Attachment object are properties that are named Attachment* including AttachmentNotes?  The code

    v2 += '<v1: >' + attachment.AttachmentNotes + '</v1:AttachmentNotes>' + crlf;

    is missing the local name in the start tag.  That appears fixed based on the screen shot you provided, but the code you provided isn't correct.  Assuming you fixed that and provided a back level code sample, then I'd expect you'd either get <v1:AttachmentNotes>undefined</v1:AttachmentNotes> or an element with the actual value filled in.  Given you have nothing in your output, it is as if the input property value is an empty string.  What is AttachmentNotes anyway? Is that just a string value which could be an empty string?  If so the code should check for the presence of this property to ensure that you don't get a value of undefined when the property doesn't exist.  The code appears to indicate that the actual base-64 content is not in attachment.AttachmentNotes but in attachment.AttachmentContent.  Typically, I'd expect you would base64 decode this before placing it in the multipart form, but I don't see that happening either.   I see the following:

                                                                                    //v2 += '<v1:AttachmentContent>' + attachment.AttachmentContent + '</v1:AttachmentContent>' + crlf;

    So you've commented this out for now, and you aren't getting the base64 content put into your output for now, but you probably will want to also base-64 decode this.  Given the need to do the decode to place the actual file payload is required, for a .pdf it will most probably have binary data and you may need to use buffer objects instead of strings as the concatenation of the content would take any binary content, even if in a buffer, and would change the encoding of that content to be utf-8 which would change your binary data characters and make the pdf unusable, so just be warned when you get to that point.

    So overall, I'm just speculating on the code since I don't know the actual schema of your input JSON data, but the above are just some observations from what you've provided.

    Since you're in a MPGW, the best friend of a GatewayScript code developer is the GatewayScript debugger.  Please see more information on it at https://www.ibm.com/docs/en/datapower-gateway/10.5.0?topic=gatewayscript-debuggers.  To enable, you must have in your GatewayScript code a debugger; statement.  This will be your first breakpoint to pause your code execution that must be in the main execution path of the code.   You must also have enabled the GatewayScript debugger for the GatewayScript action you are executing.   Then the actual debugging is done in one of two ways

    1. Using the DataPower CLI. You must be in your application domain in configuration mode, issue the show debug command and using the session id in that result, issue a debug <sessionid> command.
    2. Using the remote debugger which requires the remote debugger to be enabled and using the chrome browser, configuring chrome://inspect/#devices to listen for GatewayScript code that is paused in the debugger. Selecting the transaction from the list shown will provide you the code, list of variables to that point, etc.

    Either way, you can set breakpoints, change values of variables in your code before you execute code that would use that variable if you see that the variable value is currently incorrect which would allow you to continue debugging despite a code defect you know you need to fix but don't want to terminate your debugging session, step into and out of functions, etc.   I'd strongly recommend that you get very familiar with the debugger to help you.

    Best Regards,
    Steve



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 4.  RE: How to handle the incoming base-64 data in extra filed in Datapower and send as an attachment to backend

    Posted Wed September 27, 2023 04:44 PM

    Hi Kandukuri,

    And actually, looking through the DataPower documentation, instead of doing all of the form creation manually, there appears to be new functionality, which quite frankly I have not tried personally, that will allow you to build this without manually creating the multipart form yourself.  See https://www.ibm.com/docs/en/datapower-gateway/10.5.0?topic=services-apis-manipulate-attachments-specific-context.  Effectively, your output context of your GatewayScript action would need to have a root body part, typically your XML document, which would be written with a session.output.write  function, but you can add each attachment to that output context using the session.output.attachment.create function, thus once your GatewayScript is completed, your output context will have a payload and 1..n attachments.  If that output context is either OUTPUT or your intermediary context is the input to a Results action to OUTPUT, then the MPGW will build  the multipart form FOR YOU!!! That would greatly simply your code.  The presence of attachments in your context and your context payload would let DataPower build the multipart form including the boundaries, etc for you instead of all of the string manipulation you're using now.  Looks like a great enhancement!  I was aware this had been done for the API Gateway service, but I see it is also available with the session object which would make it available for the MPGW too. Something to do a prototype of for sure!
    Best Regards,

    Steve



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 5.  RE: How to handle the incoming base-64 data in extra filed in Datapower and send as an attachment to backend

    Posted Thu September 28, 2023 07:08 AM

    Hi Kandukuri,

    Thanks for sending the sample input data which has ...

            "AttachmentList": {
                "Attachment": [
                    {
                        "AttachmentID": 57,
                        "AttachmentRequestId": 10,
                        "AttachmentLinkID": 49,
                        "AttachmentNotes": "",
                        "AttachmentType": "PDF",
                        "AttachmentContent":"base64 encoded content"
                        
                    },
                    {
                        "AttachmentID": 58,
                        "AttachmentRequestId": 11,
                        "AttachmentLinkID": 50,
                        "AttachmentNotes": "",
                        "AttachmentType": "JPG"
                    }
                ]
            },

    Indeed AttachmentNotes is an empty string which is why your XML is created for that element as <v1:AttachmentNotes></v1:AttachmentNotes> which is correct.  If that XML element isn't required, you could test for that JSON property to not be undefined and not an empty string before adding the element to your XML string.

    Otherwise my previous comments still apply and I'd suggest you investigate the new attachment functions link that I provided in my previous post as it will save you the work of building the multipart form yourself.  I don't have cycles to do a prototype of those functions myself, but we've had numerous threads in this and the API Connect forum concerning building multipart forms and these functions were an enhancement to meet that requirement.

    Best Regards,

    Steve



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 6.  RE: How to handle the incoming base-64 data in extra filed in Datapower and send as an attachment to backend

    Posted Thu September 28, 2023 02:29 PM

    Hi,

    just adding to Steve's comprehensive explanation that the new feature which makes it possible for the DataPower to create the multipart automatically really works according to my experience. In addition to GatewayScript you can use for example Results action and attachment protocol to decode and insert the attachments to your context that is going out to the backend.



    ------------------------------
    Hermanni Pernaa
    ------------------------------



  • 7.  RE: How to handle the incoming base-64 data in extra filed in Datapower and send as an attachment to backend

    Posted Fri September 29, 2023 10:29 AM

    Hi Kandukuri,

    Just reading your original post again and I have some addition comments and question.
    1. If DataPower is receiving a multi-part form as its request body, ensure the MPGW's request settings for attachments is set to "allow".  Upon reading the request, the root body of the multipart form will become the INPUT context of the rule, the content type header will be the content-type header of the root body part, and the other parts will become attachments.  Note there is a service variable var://service/original-content-type that would have your multipart content type.
    2. If your MPGW request rule is sending traffic to some backend url, the presence of the attachments will cause the multipart form to be rebuilt, using the OUTPUT message body as the root body part with the part Content-Type header coming from the Content-Type header, the attachments will be added as additional parts, and the outbound Content-Type header will be another multipart form Content-Type.
    3. The new feature capability will allow you to get, remove, add or modify attachments.  It will be the presence of attachments that will drive whether a multipart form is sent out at the end of the request rule.  No attachments, then what will be sent out will be whatever is in OUTPUT with the current Content-Type protocol header.

    Your code is looking through a JSON object with the desire to build a multipart form. My question based on the above is do you have a use case where you are being sent a multipart form and you need to add a new attachment to it?  Either way the new feature would be the recommended path to take.

    Best Regards,

    Steve



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------