API Connect

Please provide your inputs.

My backend API has 2 operations .
1)Operation A-This operation accepts content-type of multipart/form-data and only accepts one file as an input. So we can only send input to the backend as one file attached.

 2)Operation B-This operation accepts content-type of Octet-Stream.

My API in APICONNECT has below operation. 
1)Operation 1-which accepts multipart/formdata with multiple attachments. How can I iterate through the attachments to send one file at a time to backend api to operation A as mentioned above(is it even possible?)
If iterating  through attachments and sending one file at a time would not work ,can we convert multipart/formdata request with 2 attachments into octet/Stream content-type and send it Backend of operation B.

@steve Linn can you please provide any inputs here.

Please provide your inputs.

My backend API has 2 operations .
1)Operation A-This operation accepts content-type of multipart/form-data and only accepts one file as an input. So we can only send input to the backend as one file attached.

 2)Operation B-This operation accepts content-type of Octet-Stream.

My API in APICONNECT has below operation. 
1)Operation 1-which accepts multipart/formdata with multiple attachments. How can I iterate through the attachments to send one file at a time to backend api to operation A as mentioned above(is it even possible?)
If iterating  through attachments and sending one file at a time would not work ,can we convert multipart/formdata request with 2 attachments into octet/Stream content-type and send it Backend of operation B.

Hi Vijaya,
Perhaps https://chrisphillips-cminion.github.io/apiconnect/2024/07/30/multipart-API-a.html and https://chrisphillips-cminion.github.io/apiconnect/2024/07/31/multipart-API-example.html that I worked on with @Chris Phillips will be helpful.  Note the content type is multipart/related, not multipart/form-data.  With the related multipart content, the parse policy will do a nice job of moving the root body part to message.body and the root body part content-type header to message.headers.content-type, and will also take the remaining parts and make them addressable attachments in the message object.  The example simply shows using the message attachments as an array that is directly indexed, but you can always iterate over an array in Javascript to do what you need to do.  So the question would be if you have control over the inbound content-type your API will accept.  If so, simply use multipart/related and do a parse before your GatewayScript.  If not, then you try changing the content-type header to multipart/related in message.headers.content-type, taking care to not change the other attributes such a boundary, then do a parse policy before looking at the attachments.  I've not tried this myself, and the question I have is whether the parse policy will have the payload buffer in message.body so you could set the message.header.content-type header.  You might need to set x-ibm-configuration.buffering to true to ensure that as a parse policy that uses request will still have the request header asis in message as you cannot modify a request header outside of a pre-request global policy.  Some documentation that will be useful for attachment processing can be found at https://www.ibm.com/docs/en/api-connect/10.0.5.x_lts?topic=gateway-manipulating-attachments-in-message-object

If working directly with multipart/form-data directly, in this case the parse policy will simply parse the entire multipart payload as a binary payload and it would be up to you do parse out the individual attachments.  Anything is possible, a simple matter of programming as the old saying goes, where you could get the boundary from the content-type header (you might need to find that in request.original.headers.content-type after a parse policy is used) and using the boundary from the content-type header, doing a buffer.indexOf to find each boundary, then another to find the <cf><lf> that would mark the end of the part headers to get to the payload, and another to search for the -- and boundary to find then end of the part to extract the payload.  All this has to be done with buffers as any conversion to string will encode your string and change any binary data. So it is doable, but it takes some development.  I did something like this years ago in the v5 days and used the GatewayScript debugger to step through my code to get it working, so although the process seems straightforward enough, the good GDB debugger sped things up significantly.

Hope this helps!
Steve Linn

@steve Linn can you please provide any inputs here.

Please provide your inputs.

My backend API has 2 operations .
1)Operation A-This operation accepts content-type of multipart/form-data and only accepts one file as an input. So we can only send input to the backend as one file attached.

 2)Operation B-This operation accepts content-type of Octet-Stream.

My API in APICONNECT has below operation. 
1)Operation 1-which accepts multipart/formdata with multiple attachments. How can I iterate through the attachments to send one file at a time to backend api to operation A as mentioned above(is it even possible?)
If iterating  through attachments and sending one file at a time would not work ,can we convert multipart/formdata request with 2 attachments into octet/Stream content-type and send it Backend of operation B.

Hi Steve,

We tried to use the split() function and able to print the boundary values in the log. However, when we are testing with multiple files using any test client like ARC, in response we are seeing both the file seperated by boundaries but in log when we are printing context.get('message.body'), it is not printing the whole response instead only one attachement we can see.

Also, not able to seperate the file details in the payload depending upon the boundary values using indexOf(). Can you please help on this?

------------------------------
Nirmalya Mukherjee

Original Message:
Sent: Fri November 01, 2024 03:34 PM
From: Steve Linn
Subject: Converting Multipart/form-data to octet-stream content-type

Hi Vijaya,
Perhaps https://chrisphillips-cminion.github.io/apiconnect/2024/07/30/multipart-API-a.html and https://chrisphillips-cminion.github.io/apiconnect/2024/07/31/multipart-API-example.html that I worked on with @Chris Phillips will be helpful.  Note the content type is multipart/related, not multipart/form-data.  With the related multipart content, the parse policy will do a nice job of moving the root body part to message.body and the root body part content-type header to message.headers.content-type, and will also take the remaining parts and make them addressable attachments in the message object.  The example simply shows using the message attachments as an array that is directly indexed, but you can always iterate over an array in Javascript to do what you need to do.  So the question would be if you have control over the inbound content-type your API will accept.  If so, simply use multipart/related and do a parse before your GatewayScript.  If not, then you try changing the content-type header to multipart/related in message.headers.content-type, taking care to not change the other attributes such a boundary, then do a parse policy before looking at the attachments.  I've not tried this myself, and the question I have is whether the parse policy will have the payload buffer in message.body so you could set the message.header.content-type header.  You might need to set x-ibm-configuration.buffering to true to ensure that as a parse policy that uses request will still have the request header asis in message as you cannot modify a request header outside of a pre-request global policy.  Some documentation that will be useful for attachment processing can be found at https://www.ibm.com/docs/en/api-connect/10.0.5.x_lts?topic=gateway-manipulating-attachments-in-message-object

If working directly with multipart/form-data directly, in this case the parse policy will simply parse the entire multipart payload as a binary payload and it would be up to you do parse out the individual attachments.  Anything is possible, a simple matter of programming as the old saying goes, where you could get the boundary from the content-type header (you might need to find that in request.original.headers.content-type after a parse policy is used) and using the boundary from the content-type header, doing a buffer.indexOf to find each boundary, then another to find the <cf><lf> that would mark the end of the part headers to get to the payload, and another to search for the -- and boundary to find then end of the part to extract the payload.  All this has to be done with buffers as any conversion to string will encode your string and change any binary data. So it is doable, but it takes some development.  I did something like this years ago in the v5 days and used the GatewayScript debugger to step through my code to get it working, so although the process seems straightforward enough, the good GDB debugger sped things up significantly.

Hope this helps!
Steve Linn

------------------------------
Steve Linn
Senior Consulting I/T Specialist
IBM

Original Message:
Sent: Wed October 30, 2024 09:34 AM
From: vijaya k
Subject: Converting Multipart/form-data to octet-stream content-type

@steve Linn can you please provide any inputs here.

------------------------------
vijaya k

Original Message:
Sent: Tue October 29, 2024 01:19 PM
From: vijaya k
Subject: Converting Multipart/form-data to octet-stream content-type

Please provide your inputs.

My backend API has 2 operations .
1)Operation A-This operation accepts content-type of multipart/form-data and only accepts one file as an input. So we can only send input to the backend as one file attached.

 2)Operation B-This operation accepts content-type of Octet-Stream.

My API in APICONNECT has below operation. 
1)Operation 1-which accepts multipart/formdata with multiple attachments. How can I iterate through the attachments to send one file at a time to backend api to operation A as mentioned above(is it even possible?)
If iterating  through attachments and sending one file at a time would not work ,can we convert multipart/formdata request with 2 attachments into octet/Stream content-type and send it Backend of operation B.

------------------------------
vijaya k
------------------------------

Steve,

Below is the code which Nirmalaya is talking about, we are testing multipart/form-data  request with 2 attachments.

below is the request, we would like to seperate each file content when we receive and send it as an octet/stream or each attachment as multipartform-data request to backend

request

Below is the screenshot for console.error("reqbody is"+reqbody);

 screenshot of console.error("indexofpayload456"+indexofpayload1);  
  

------------------------------
vijaya k

Original Message:
Sent: Thu November 21, 2024 04:47 AM
From: Nirmalya Mukherjee
Subject: Converting Multipart/form-data to octet-stream content-type

Hi Steve,

We tried to use the split() function and able to print the boundary values in the log. However, when we are testing with multiple files using any test client like ARC, in response we are seeing both the file seperated by boundaries but in log when we are printing context.get('message.body'), it is not printing the whole response instead only one attachement we can see.

Also, not able to seperate the file details in the payload depending upon the boundary values using indexOf(). Can you please help on this?

------------------------------
Nirmalya Mukherjee

Original Message:
Sent: Fri November 01, 2024 03:34 PM
From: Steve Linn
Subject: Converting Multipart/form-data to octet-stream content-type

Hi Vijaya,
Perhaps https://chrisphillips-cminion.github.io/apiconnect/2024/07/30/multipart-API-a.html and https://chrisphillips-cminion.github.io/apiconnect/2024/07/31/multipart-API-example.html that I worked on with @Chris Phillips will be helpful.  Note the content type is multipart/related, not multipart/form-data.  With the related multipart content, the parse policy will do a nice job of moving the root body part to message.body and the root body part content-type header to message.headers.content-type, and will also take the remaining parts and make them addressable attachments in the message object.  The example simply shows using the message attachments as an array that is directly indexed, but you can always iterate over an array in Javascript to do what you need to do.  So the question would be if you have control over the inbound content-type your API will accept.  If so, simply use multipart/related and do a parse before your GatewayScript.  If not, then you try changing the content-type header to multipart/related in message.headers.content-type, taking care to not change the other attributes such a boundary, then do a parse policy before looking at the attachments.  I've not tried this myself, and the question I have is whether the parse policy will have the payload buffer in message.body so you could set the message.header.content-type header.  You might need to set x-ibm-configuration.buffering to true to ensure that as a parse policy that uses request will still have the request header asis in message as you cannot modify a request header outside of a pre-request global policy.  Some documentation that will be useful for attachment processing can be found at https://www.ibm.com/docs/en/api-connect/10.0.5.x_lts?topic=gateway-manipulating-attachments-in-message-object

If working directly with multipart/form-data directly, in this case the parse policy will simply parse the entire multipart payload as a binary payload and it would be up to you do parse out the individual attachments.  Anything is possible, a simple matter of programming as the old saying goes, where you could get the boundary from the content-type header (you might need to find that in request.original.headers.content-type after a parse policy is used) and using the boundary from the content-type header, doing a buffer.indexOf to find each boundary, then another to find the <cf><lf> that would mark the end of the part headers to get to the payload, and another to search for the -- and boundary to find then end of the part to extract the payload.  All this has to be done with buffers as any conversion to string will encode your string and change any binary data. So it is doable, but it takes some development.  I did something like this years ago in the v5 days and used the GatewayScript debugger to step through my code to get it working, so although the process seems straightforward enough, the good GDB debugger sped things up significantly.

Hope this helps!
Steve Linn

------------------------------
Steve Linn
Senior Consulting I/T Specialist
IBM

Original Message:
Sent: Wed October 30, 2024 09:34 AM
From: vijaya k
Subject: Converting Multipart/form-data to octet-stream content-type

@steve Linn can you please provide any inputs here.

------------------------------
vijaya k

Original Message:
Sent: Tue October 29, 2024 01:19 PM
From: vijaya k
Subject: Converting Multipart/form-data to octet-stream content-type

Please provide your inputs.

My backend API has 2 operations .
1)Operation A-This operation accepts content-type of multipart/form-data and only accepts one file as an input. So we can only send input to the backend as one file attached.

 2)Operation B-This operation accepts content-type of Octet-Stream.

My API in APICONNECT has below operation. 
1)Operation 1-which accepts multipart/formdata with multiple attachments. How can I iterate through the attachments to send one file at a time to backend api to operation A as mentioned above(is it even possible?)
If iterating  through attachments and sending one file at a time would not work ,can we convert multipart/formdata request with 2 attachments into octet/Stream content-type and send it Backend of operation B.

------------------------------
vijaya k
------------------------------