hi

when i am setting movingwindow flag on nested iterators the 2nd getNextIterator is throwing error saying it should be only true when iterating on nodes. :shock:

Ex: xml structure is like xmltag1->xmltag2. Passing tag1’s iterable node to tag2’s getNextIterator causes this error.

Does any one have faced this situation and resolved it? :roll:

thanks
vijay

Hi Vijay,
Are you talking about “pub.xml:getXMLNodeIterator” service.
If so,then I think you no need to pass movingWindow values.If you have more then one node then pass criteria.Then use “pub.xml:getNextXMLNode” in some repeated step to get the next node.

Please let us know if you face any difficulty in implementing the same or you can pass your exact xml structure.

Thanks
Baharul Islam

We use moving window input while handling large xml file.

Can you attach your xml, for your reference reference the link below

Hi Mahesh

The file is huge and cannnot attach, I can give a sample example as below

<rootnode>
<nodeparent>
<parentname>ParentOfNode_1_2</parentname>
<childnode>
<name>Node 1</name>
<price>100</price>
</childnode>
<childnode>
<name>Node 2</name>
<price>23</price>
</childnode>
</nodeparent>
<nodeparent>
<parentname>ParentOfNode_3_4</parentname>
<childnode>
<name>Node 3</name>
<price>100</price>
</childnode>
<childnode>
<name>Node 4</name>
<price>23</price>
</childnode>
</nodeparent>
<childnode>
<name>Node 5</name>
<price>4353</price>
</childnode>
<childnode>
<name>Node 6</name>
<price>23.4</price>
</childnode>
<childnode>
<name>Node 7</name>
<price>12.4</price>
<resources>
<resource type="test">This will be in the output</resource>
<resource type="test1">This won't be in the output</resource>
</resources>
</childnode>
</rootnode>

As can be seen above, the nodeparent tag is repeating and I want to iterate on this and subsequently get the iterator of childnode under it and process each childnode individually.

However for the service getXMLNodeIterator if I give criteria nodeparent and childnode then it loads all “childnode” under each “nodeparent” while iterating on nodeparent, which doesn’t make sense as I am telling to iterate on childnode also but it loads all childnode at once while iterating on nodeparent. So I have to explicitly get each iterator one by one by calling getXMLNodeIterator twice using parent child relationship.

During the first call of getXMLNodeIterator with criteria nodeparent and movingWindow to true, I get first node of this iterator and pass it to get the second iterator with criteria childnode and movingWindow set to true again then it throws an error saying that this flag can only be used once.

Hi Mahesh

The file is huge and cannnot attach, I can give a sample example as below

<rootnode>
<nodeparent>
<parentname>ParentOfNode_1_2</parentname>
<childnode>
<name>Node 1</name>
<price>100</price>
</childnode>
<childnode>
<name>Node 2</name>
<price>23</price>
</childnode>
</nodeparent>
<nodeparent>
<parentname>ParentOfNode_3_4</parentname>
<childnode>
<name>Node 3</name>
<price>100</price>
</childnode>
<childnode>
<name>Node 4</name>
<price>23</price>
</childnode>
</nodeparent>
<childnode>
<name>Node 5</name>
<price>4353</price>
</childnode>
<childnode>
<name>Node 6</name>
<price>23.4</price>
</childnode>
<childnode>
<name>Node 7</name>
<price>12.4</price>
<resources>
<resource type="test">This will be in the output</resource>
<resource type="test1">This won't be in the output</resource>
</resources>
</childnode>
</rootnode>

As can be seen above, the nodeparent tag is repeating and I want to iterate on this and subsequently get the iterator of childnode under it and process each childnode individually.

However for the service getXMLNodeIterator if I give criteria nodeparent and childnode then it loads all “childnode” under each “nodeparent” while iterating on nodeparent, which doesn’t make sense as I am telling to iterate on childnode also but it loads all childnode at once while iterating on nodeparent. So I have to explicitly get each iterator one by one by calling getXMLNodeIterator twice using parent child relationship.

During the first call of getXMLNodeIterator with criteria nodeparent and movingWindow to true, I get first node of this iterator and pass it to get the second iterator with criteria childnode and movingWindow set to true again then it throws an error saying that this flag can only be used once.

Hi Mahesh

The file is huge and cannnot attach, I can give a sample example as below

<rootnode>
<nodeparent>
<parentname>ParentOfNode_1_2</parentname>
<childnode>
<name>Node 1</name>
<price>100</price>
</childnode>
<childnode>
<name>Node 2</name>
<price>23</price>
</childnode>
</nodeparent>
<nodeparent>
<parentname>ParentOfNode_3_4</parentname>
<childnode>
<name>Node 3</name>
<price>100</price>
</childnode>
<childnode>
<name>Node 4</name>
<price>23</price>
</childnode>
</nodeparent>
<childnode>
<name>Node 5</name>
<price>4353</price>
</childnode>
<childnode>
<name>Node 6</name>
<price>23.4</price>
</childnode>
<childnode>
<name>Node 7</name>
<price>12.4</price>
<resources>
<resource type="test">This will be in the output</resource>
<resource type="test1">This won't be in the output</resource>
</resources>
</childnode>
</rootnode>

As can be seen above, the nodeparent tag is repeating and I want to iterate on this and subsequently get the iterator of childnode under it and process each childnode individually.

However for the service getXMLNodeIterator if I give criteria nodeparent and childnode then it loads all “childnode” under each “nodeparent” while iterating on nodeparent, which doesn’t make sense as I am telling to iterate on childnode also but it loads all childnode at once while iterating on nodeparent. So I have to explicitly get each iterator one by one by calling getXMLNodeIterator twice using parent child relationship.

During the first call of getXMLNodeIterator with criteria nodeparent and movingWindow to true, I get first node of this iterator and pass it to get the second iterator with criteria childnode and movingWindow set to true again then it throws an error saying that this flag can only be used once.

I see your sample xml is not correct. Just have a close look at line 24 it is starting with - Is it normal

Should it be like this: inside a root tag. If yes I can share you the code. Contact me on my email.

    <nodeparent>
<parentname>ParentOfNode_1_2</parentname>
<childnode>
<name>Node 1</name>
<price>100</price>
</childnode>
<childnode>
<name>Node 2</name>
<price>23</price>
</childnode>
</nodeparent>
<nodeparent>
<parentname>ParentOfNode_3_4</parentname>
<childnode>
<name>Node 3</name>
<price>100</price>
</childnode>
<childnode>
<name>Node 4</name>
<price>23</price>
</childnode>
</nodeparent>

I see your sample xml is not correct. Just have a close look at line 24 it is starting with - Is it normal

Should it be like this: inside a root tag. If yes I can share you the code. Contact me on my email.

    <nodeparent>
<parentname>ParentOfNode_1_2</parentname>
<childnode>
<name>Node 1</name>
<price>100</price>
</childnode>
<childnode>
<name>Node 2</name>
<price>23</price>
</childnode>
</nodeparent>
<nodeparent>
<parentname>ParentOfNode_3_4</parentname>
<childnode>
<name>Node 3</name>
<price>100</price>
</childnode>
<childnode>
<name>Node 4</name>
<price>23</price>
</childnode>
</nodeparent>

It’s just a sample file to explain abt the issue . Don’t worry too much about the semantics of the file. If you really could solve the issue email me the code to kalyan.cdev@gmail.com I will take a look at it.

Yes, I know it might be a typo or copy paste error I want you to confirm on the same and I am not worrying about it

Check the code sample I have attached it here.
X.zip (6.77 KB)

The code can look simpler if you use moving window=false. You can try it out. Ask me if any questions.

The code can look simpler if you use moving window=false. You can try it out. Ask me if any questions.

Thanks for sharing the code…the 3 steps just after the first branch will consume memory and will not work in my case. I have attached the scrshot for ur reference.

https://higherlogicdownload.s3.amazonaws.com/IMWUC/webMethods/Files3/4fe99fcf943b3d68a7791f01dde6c2d12700b0ea_2_1035x580.jpeg 1.5x, https://higherlogicdownload.s3.amazonaws.com/IMWUC/webMethods/Files/4fe99fcf943b3d68a7791f01dde6c2d12700b0ea.jpeg 2x" data-dominant-color="D2D1D9">

Untitled.jpg1366×768 276 KB

Yes, let me see if I find any alternative solution. What is your xml file size?

consider it as large xml ~100mb

consider it as large xml ~100mb

Good deal and interesting. I will keep you posted if I find any alternate solution. Let me know if you find something.

For handling large xml file I assume you should even take in to effect of your JVM MAX and MIN memory assigned to IS? Is it?

Good deal and interesting. I will keep you posted if I find any alternate solution. Let me know if you find something.

For handling large xml file I assume you should even take in to effect of your JVM MAX and MIN memory assigned to IS? Is it?

yep ofcourse its all about memory when handling large files…i will let u know if i find anything that works…appreciate ur efforts on this

thanks

yep ofcourse its all about memory when handling large files…i will let u know if i find anything that works…appreciate ur efforts on this

thanks

Ok, few more points on this subject. It might help you I guess

1> If you are loading the large xml via FS using getFile Svc load the file as stream data instead of bytes for better performance
2> You can pass the stream data directly to xmlStirng to Node service
3> Also you can do a performance test with movingWindow=false.
4> What is your JVM min and max memory maintained in prod environment. As I would recommend you to test with the same JVM memory on your dev/test environments.

I already tried it but of no use. Looks like the movingWindow is the key here, if thats not set then it accumulates all the previous nodes and is causing out of memory error.

I tried just one iterator with this flag set to true and it works fine with small memory footprint.

If I use nested iterators then it fails with memory heap space error. :-?

I think I some how managed to code it as per your requirement with moving window=true. Install the package and just run once and see the server.log file.

X.zip (8.26 KB)

I think I some how managed to code it as per your requirement with moving window=true. Install the package and just run once and see the server.log file.

X.zip (8.26 KB)

this one doesn’t quite meet the req. as the second iterator is independent of first one since u r passing main_node try passing next/node and see if it works.

i tried all the possible things, could not make this work in webM IS, its a shame that webMethods is unable to handle this

LOL Lets keep trying, did you talk to support (not sure if they can help). And we can also try using the enhanced xml parsing if you are on version 9 (but we need to try if this works or not).

LOL Lets keep trying, did you talk to support (not sure if they can help). And we can also try using the enhanced xml parsing if you are on version 9 (but we need to try if this works or not).

LOL Lets keep trying, did you talk to support (not sure if they can help). And we can also try using the enhanced xml parsing if you are on version 9 (but we need to try if this works or not).

LOL Lets keep trying, did you talk to support (not sure if they can help). And we can also try using the enhanced xml parsing if you are on version 9 (but we need to try if this works or not).

i don’t have access to empower site. so have to take help of someone who has access and raise an incident about this issue…

Hi Mahesh

Is it possible for you to open a support ticket with SAG regarding this issue? I would really appreciate it if you can do this.

Thanks,
Vijay

Nope, I don’t have access privileges to raise incident against SAG. Only my onsite counterpart has this access.

Can you get in-touch with your onsite team. They can do it for your just like that.

Nope, I don’t have access privileges to raise incident against SAG. Only my onsite counterpart has this access.

Can you get in-touch with your onsite team. They can do it for your just like that.

Nope, I don’t have access privileges to raise incident against SAG. Only my onsite counterpart has this access.

Can you get in-touch with your onsite team. They can do it for your just like that.