The fact that the system successfully processes small PDFs but fails with larger files suggests that the issue might be related to network limitations. I suspect that the 404 errors are not originating from API Connect itself, but rather from load balancers or firewalls that might be redirecting requests to an error page. However, this is just one possible explanation and further investigation is required.
To effectively diagnose and resolve this issue, it's crucial to collaborate with all relevant teams to gather information and analyze the problem comprehensively.
Original Message:
Sent: Tue December 31, 2024 10:03 PM
From: Joseph Morgan
Subject: Issue with Uploading Large PDF Files from Web Application to DataPower-APIC (HTTP 404)
Ahmad,
Why would network bandwidth problems result in a 404? Does API Connect respond with a 404 on a timeout?
Why would API Connect respond with a 404 in any case of a file being too large?
The 404 is the mystery here. I could understand a 504, which actually makes perfect sense if it is a network bandwidth problem. A 413 is called for if the message is too large. Even a 415 if the content encoding is not correct. But 404? What, per se, is not being found?
------------------------------
Joseph Morgan
CEO - Independent
Dallas TX
Original Message:
Sent: Tue December 31, 2024 09:04 AM
From: ahmad taha
Subject: Issue with Uploading Large PDF Files from Web Application to DataPower-APIC (HTTP 404)
Dear Kumar,
You've mentioned that you're encountering an issue when attempting to pass large PDF files through DataPower to the API Connect Gateway. While the transfer of small files appears successful, larger files are not being processed.
To assist you effectively, please provide more details:
- Data Transfer Method: Could you clarify the method or protocol used to transfer the PDF files? Are you using SOAP with MTOM, Base64 encoding, or another method?
- Rationale for DataPower: Could you explain the rationale behind using DataPower in front of API Connect? Why not consider alternative solutions such as SFTP, Aspera, leveraging DataPower as SFTP Proxy in the MPG Service, or direct HTTP/SOAP transfers with Base64 encoding within the request?
- Infrastructure Readiness: Have you assessed the readiness of the network infrastructure to handle the transfer of large PDF files?
- Short-term Workaround: As a temporary solution, you can increase the maximum message size in the DataPower's XML Manager and in the API Connect API definition -using parse Assembly-. However, this is not a sustainable long-term solution as it can significantly impact the performance of DataPower, API Connect, and the network.
Further Investigation:
- Network Bandwidth:Thoroughly evaluate the available network bandwidth to ensure it can accommodate the transfer of large PDF files.
- Firewall Rules:Check if any firewall rules are blocking or restricting the data transfer.
- I recommend a comprehensive review of the current architecture and a thorough investigation of the underlying cause of the issue.
. Best regards,
------------------------------
Ahmad Taha
Enterprise Architect | IBM | Coach | TOGAF
Original Message:
Sent: Mon December 30, 2024 11:52 AM
From: Kumar .
Subject: Issue with Uploading Large PDF Files from Web Application to DataPower-APIC (HTTP 404)
Hello All,
There is a web application that allows users to upload PDF files, and DataPower receives these PDF files, ranging from 1MB to a maximum of 25MB. DataPower is expected to forward these files to API Connect. However, the request fails between DataPower and API Connect with an HTTP status code 404.
Smaller files, such as those around 4MB or less, are successfully processed and transmitted from DataPower to API Connect. We have configured the XML Manager to handle payloads up to 30MB.
We saved the 10MB file in our local directory, referenced its path in the SOAP request and tested from SOAP UI, and sent it to DataPower. DataPower successfully received and forwarded this file to API Connect without issues.
The problem arises only when uploading through the web application, specifically with larger files, resulting in a 404 error. What could be the cause of this? We are utilizing chunked transfers in the API Connect Invoke , however the request(10MB) is not being forwarded to API Connect. When inspecting the headers in the DataPower "Show Probe," the page hangs with a memory-related error, possibly due to the size of the payload.
We would appreciate any insights or suggestions to resolve this issue.
------------------------------
Kumar
------------------------------