How do I publish a large xml file . I have gone through couple of articles in this forum. This is how my flow service looks:

Get xml file
Convert xml string to node…loading as stream
specifying a node iterator
Getxml node iterator
repeat
GetnextXMLnode
Branch on next node:
node to doc
document to string
append to string list

String list to documentlist
Documentlist to document
Publish

Am sure there is a better approach to this and wanted some help.

Thanks,

Hi,
You can adopt multi publish strategy. Publish around 1000 records each time till the total records for a particular transaction or set of record is published. On the Subscriber side, the records can be processed based on some flag set for that particular set of records. Flag should eb set in such a way that indicates if there are more records coming for this lot/set of rec…

Cheers,
KK,

Another approach would be to not publish the large document at all. Instead, place the large document in a commonly accessible place and publish an event that essentially says “there’s a new big file over there.”

Publishing large documents (which usually are driven by batch processes) is contrary to the notion of publishing things as they happen (near real-time integration) through a broker. A broker isn’t really intended to handle large documents (though the ceiling keeps rising).

Be careful too that you don’t keep the entire target document in memory–no sense treating an incoming document as large if you’re just going to load it all into memory in the target format.

I have got my code modified. It loops for all the nodes and when the number of records reaches 20 it publishes a document. A trigger is in places that subscribes to this document and inserts it into a db.

If I want to use the same code to receive 3 different type of xml, How can I do this.

Thanks