I have an IIB message flow that looks like this:
HTTP Input Node -> File Read Node -> HTTP Reply Node.
(1) HTTP Input Node - flow is started by a user initiating an https request. (2) File Read Node - message flow gets a file from our ftp server (local file directory, file name, sftp host and credentials, and sftp server directory all temporarily hardcoded just for ease in troubleshooting a solution, but in real life this info would get extracted from the http request). (3) HTTP Reply Node - reply to user with contents of file retrieved from the ftp server.
The problem is this: The message flow causes a "java/lang/OutOfMemoryError" heap dump when the File Read Node reads large files (I've been testing by running multiple instances of the message flow with a 124 MG file).
Background: This is a legacy ASP/VBScript application that was rewritten in IIB. I've been tasked with fixing the memory error.
More Info: From reading, I understand why the committed memory gets all used up. I've read all I can on large message processing but because this application's main purpose is to allow users to pick up files from our ftp server via https, I seem to be tied to that HTTP Reply Node and thus the entire file has to be available in the MBMessageAssembly object (and therefore in memory) when the flow hits the "in" terminal of the HTTP Reply Node.
My question is this: Is there a technique for "streaming" to the HTTP Reply Node? Or, is it possible to remove the HTTP Reply Node altogether and do the SFTP and the HTTP Reply in a Java Compute Node (I can't use a Compute Node/ESQL since we aren't licensed for that)? Or, any other ideas?