BPM, Workflow, and Case

 View Only
Expand all | Collapse all

Reading Octetstream file content response via javascript using Apache HTTP client

  • 1.  Reading Octetstream file content response via javascript using Apache HTTP client

    Posted Thu March 25, 2021 08:34 AM
    Hello All,

    Good morning!. I have a requirement to read a file content which streams the output in Octet-stream format through an REST-API call and to encode the received response in encodedBase64 format as it needs to be sent to another system in upstream.

    So I have implemented this and partly successful- which includes calling the RESTAPI via Javascript using Apache HTTP client methods and I got the 200 response back and also I am able to encode the same in base64 format and pass it to the upstream system. The challenge i face here is when the file size streamed is of approximately anything above 13KB size and I get the below error
    Internal Script error: java.io.IOException: chunked stream ended unexpectedly.

    Note:
    The system which is streaming the file content can be of any type and any size. there is no upper limit set but in real time scenario it won't exceed 15-20 MB approx.
    I tried 2 approaches till now. both of them are giving the same errors. 
    first approach
    var output=op.getResponseBodyAsStream();                //to invoke the response body
    rd= new Packages.java.io.BufferedReader(new Packages.java.io.InputStreamReader(output));   // to buffer the read contents
    response= Packages.org.apache.commons.io.IOUtils.toString(rd);     //finally collecting the response

    second approach
    var inputStream= op.getResponseBodyAsStream();
    var outputStream= new Packages.org.apache.commons.io.output.ByteArrayOutputStream();     //instead of stream reader here i used bytearraystream
    response= Packages.org.apache.commons.io.IOUtils.copy(inputStream,outputStream);                  //finally copying the input to outputstream

    I just highlighted the main piece of code where it is going wrong. Let me know if you need any other details . Please advise how to overcome this obstacle. 
    currently this needs to be implemented in BPM7.5 Process designer but the runtime environment is in BAW 19.0.0.3. I also tried the same logic in BAW 19.0.0.3 Web process designer thinking that Apache library version could be a problem but there also I am getting the same error.

    Regrads,
    Bharanee
    IBM BAW Developer


    ------------------------------
    Bharaneedharan Muralidharan
    IBM BAW Developer
    ------------------------------


  • 2.  RE: Reading Octetstream file content response via javascript using Apache HTTP client

    Posted Fri March 26, 2021 06:41 AM
    Does it fail with the same error in first approach and second approach? 
    Can you provide a full stack trace of an error? 
    Why not do this in Java vs JS Liveconnect?
    Seems like you are not able to get the response body because the content sent by the server appears malformed (incorrectly chunk-encoded). HttpClient
    always attempts to read responses in their entirety in order to be able to reuse the connection. If a response is not fully consumed the connection used to transmit it cannot be reused for obvious reasons.
    You may need to enable httpclient debug logs to understand why it's failing this way.

    ------------------------------
    Sergei Malynovskyi
    ------------------------------



  • 3.  RE: Reading Octetstream file content response via javascript using Apache HTTP client

    Posted Thu June 24, 2021 04:43 AM
    Thanks Sergei Malynovskyi  for your suggestions and coming back quickly. you are correct about the point (Seems like you are not able to get the response body because the content sent by the server appears malformed (incorrectly chunk-encoded).)
     
    We are able to figure out the problem like you said there is no problem in reading the response by httpclient . there was a configuration problem at the upstream system where they were not able to stream the file content beyond a certain limit. once they rectified, my function worked perfectly.
     
    Regards,
    Bharanee
    IBM Process Workflow
    Phone: +91 9791220711