We get an incomplete body in response from pub.client:http.
Json is cuted at a random location.

No matter if it’s long. It can be 1200 lines and will be cuted at 1000, or it can have 500 lines and be cuted at 340. There is no rule. This is nondeterministic. I will add that there is a lot of processing on the IS at the time when this truncation occurs.

No error is logged. The connection has been checked from the network side. TCPDUMP has not shown that the truncation is on the network side.

I am asking for help in locating the problem.

Is there any buffer to set to the size of all incoming messages?
Anyone had a similar case?

Hi @Leszek_Zaczyk
This is a tricky problem , hence will need more information.

1. You’ve mentioned that there is a lot of processing on the IS at the time truncation occurs - Can you mention what kind of processing you are referring to? Other requests should not interfere with this request.
   You can run the pub.client:http alone in a flow and check for the response headers -specifically the Content-Length , this should be in the header->lines->Content-Length of the output.
2. You can then check the data from the body to confirm if there is any missing data.
3. If the pub.client:http cannot be run separately , you can invoke pub.flow:tracePipeline just after the pub.client:http invocation , this should the response headers in the logs where you can check the content length.
4. I’m assuming that you are calling the same endpoint from pub.client:http which return different lengths of data . Are there any endpoints that you can call from pub.client:http that return data comparable to the current service that you are getting data from? Other formats too…

-NP

Hi @Nagendra_Prasad
Thank you for your response

ad 1
The data synchronization interface is running.
We put about 2,000 events on the UM queue.
2 IS’s fetches them in 5 threads and processes them.
During one processing we call about 100 adapters (REST) to download the necessary information.
It is one of them that the truncated answer appears.

There is no Content-Length in the header
ad 2
We have the answer in bytes.
We use pub.io:bytesToStream then pub.json: jsonStreamToDocument - throws an exception because checking bytes - json truncated there

ad3
pub.flow: tracePipeline does not print Content-Length

ad4
Not. I have to use this particular endpoint

Finally, I would like to repeat the question:
Is there any buffer to set to the size of all incoming messages?

This looks this needs a detailed investigation , best to raise a support incident.

Also modified tags to include UM to reach wider audience.

-NP