Hi community,

when we migrated from 9.1.6 our application took quite a performance hit of taking about twice as long as before, which is sadly not sufficient in the given use case.

There is one task in particular taking up most of the time. It consists of ~75 rulesets each of which fills a particular field with the method experimentally found to work best to get the data. Some rules to differentiate between subtypes of that document, but nothing complicated.

The rrs-logs show no single action taking even a whole second - so the hit is somewhat spread out :/

To be honest, i am a bit lost of how to proceed. I don't think there is much room to tune the rules themselves. Is there something like a delay for the Rulerunner between actions/functions/rules, that can be adjusted? What would be the trade-off?

What are your experiences with datacap 9.1.9 performance? Is this just normal? I saw Shaun had massive (more than x2) problems in DcDesktop, but those got alleviated by installing iFix  04, which is not the case here. Any ideas/input/help would be much appreciated :)

Regards and thanks,

Julian

------------------------------
Julian Fiegenbaum | ISR Information Products AG | Consultant | Germany
------------------------------

The RRS log show start time as  to 13:23:06 and end time at 13:23:31.  That's a total of 25 seconds.   No time gap and they are all in milliseconds.  
These are expensive task meaning require processing time.  

Rule Runner are background processor.  A true test is to run it with thick client such as "dcDesktop".  See if you are seeing 25 seconds to process the recogFiled Task.
Rule Runner could take longer because of the share location of your batch or if the rrs log are huge and it's writing to a NAS or share location impacted by network bandwidth. 

Go to Taskmaster Application Manager.  In 9.19 you can separate your rrs log from the batch.

What speed are you seeing in 9.16?  Same set of document? 

------------------------------
Blue Devil

Original Message:
Sent: Wed October 25, 2023 07:34 AM
From: Julian Fiegenbaum
Subject: Datacap 9.1.9 iFix 04 performance after migration from 9.1.6 and tuning

Hi community,

when we migrated from 9.1.6 our application took quite a performance hit of taking about twice as long as before, which is sadly not sufficient in the given use case.

There is one task in particular taking up most of the time. It consists of ~75 rulesets each of which fills a particular field with the method experimentally found to work best to get the data. Some rules to differentiate between subtypes of that document, but nothing complicated.

The rrs-logs show no single action taking even a whole second - so the hit is somewhat spread out :/

To be honest, i am a bit lost of how to proceed. I don't think there is much room to tune the rules themselves. Is there something like a delay for the Rulerunner between actions/functions/rules, that can be adjusted? What would be the trade-off?

What are your experiences with datacap 9.1.9 performance? Is this just normal? I saw Shaun had massive (more than x2) problems in DcDesktop, but those got alleviated by installing iFix  04, which is not the case here. Any ideas/input/help would be much appreciated :)

Regards and thanks,

Julian

------------------------------
Julian Fiegenbaum | ISR Information Products AG | Consultant | Germany
------------------------------

Hi Blue Devil,

thank your for taking the time to look at my problem! Formerly the task took (depending on the document) 11-19s and now it takes about 20-30s. Which does not seem like that much, but is relevant to our client's requirements. 
But yeah - the log left me a little hopeless. There is no single cause and each individual action takes milliseconds. That's why i came here for ideas ;)

I have tried to disable rrs-logging entirely, which did not have a noteworthy impact. We enquired about any differences in specs, including how the system is integrated in the intranet, but were assured that these are exactly the same. I will ask again and be more specific about the share.

The difference in time is notable in DcStudio as well, but maybe that was just my expectations. I will measure the time so i can confirm or refute a RuleRunner specific problem. 

Thank you so much for your suggestions,

Julian

The RRS log show start time as  to 13:23:06 and end time at 13:23:31.  That's a total of 25 seconds.   No time gap and they are all in milliseconds.  
These are expensive task meaning require processing time.  

Rule Runner are background processor.  A true test is to run it with thick client such as "dcDesktop".  See if you are seeing 25 seconds to process the recogFiled Task.
Rule Runner could take longer because of the share location of your batch or if the rrs log are huge and it's writing to a NAS or share location impacted by network bandwidth. 

Go to Taskmaster Application Manager.  In 9.19 you can separate your rrs log from the batch.

What speed are you seeing in 9.16?  Same set of document? 

------------------------------
Blue Devil

Original Message:
Sent: Wed October 25, 2023 07:34 AM
From: Julian Fiegenbaum
Subject: Datacap 9.1.9 iFix 04 performance after migration from 9.1.6 and tuning

Hi community,

when we migrated from 9.1.6 our application took quite a performance hit of taking about twice as long as before, which is sadly not sufficient in the given use case.

There is one task in particular taking up most of the time. It consists of ~75 rulesets each of which fills a particular field with the method experimentally found to work best to get the data. Some rules to differentiate between subtypes of that document, but nothing complicated.

The rrs-logs show no single action taking even a whole second - so the hit is somewhat spread out :/

To be honest, i am a bit lost of how to proceed. I don't think there is much room to tune the rules themselves. Is there something like a delay for the Rulerunner between actions/functions/rules, that can be adjusted? What would be the trade-off?

What are your experiences with datacap 9.1.9 performance? Is this just normal? I saw Shaun had massive (more than x2) problems in DcDesktop, but those got alleviated by installing iFix  04, which is not the case here. Any ideas/input/help would be much appreciated :)

Regards and thanks,

Julian

------------------------------
Julian Fiegenbaum | ISR Information Products AG | Consultant | Germany
------------------------------

dStudio is very forgiving and better client to use for testing is thick client "dcDestkop".  If you can isolate that the speed is acceptable with dcDesktop as in 9.16, then you can proceed to Rulerunner as the next suspect. 

This all goes down to the amount of thread(s) and task(s) configured in RuleRunner from batch processing.  This will impact the speed over all.  Best to just have one thread and one task aka  your recogFiled Task to see if there is a different between dcDesktop vs processing in RuleRunner.  Please make sure you are comparing apple to apple meaning it's the same set of rules and action library in 9.16 you are using. 
If some ruleset or action library that's available in 916 but depreciated in 9.19 however a replacement is being used, that will also make a difference. 

------------------------------
Blue Devil

Original Message:
Sent: Tue October 31, 2023 10:40 AM
From: Julian Fiegenbaum
Subject: Datacap 9.1.9 iFix 04 performance after migration from 9.1.6 and tuning

Hi Blue Devil,

thank your for taking the time to look at my problem! Formerly the task took (depending on the document) 11-19s and now it takes about 20-30s. Which does not seem like that much, but is relevant to our client's requirements. 
But yeah - the log left me a little hopeless. There is no single cause and each individual action takes milliseconds. That's why i came here for ideas ;)

I have tried to disable rrs-logging entirely, which did not have a noteworthy impact. We enquired about any differences in specs, including how the system is integrated in the intranet, but were assured that these are exactly the same. I will ask again and be more specific about the share.

The difference in time is notable in DcStudio as well, but maybe that was just my expectations. I will measure the time so i can confirm or refute a RuleRunner specific problem. 

Thank you so much for your suggestions,

Julian

------------------------------
Julian Fiegenbaum | ISR Information Products AG | Consultant | Germany

Original Message:
Sent: Mon October 30, 2023 03:33 PM
From: Blue Devil
Subject: Datacap 9.1.9 iFix 04 performance after migration from 9.1.6 and tuning

The RRS log show start time as  to 13:23:06 and end time at 13:23:31.  That's a total of 25 seconds.   No time gap and they are all in milliseconds.  
These are expensive task meaning require processing time.  

Rule Runner are background processor.  A true test is to run it with thick client such as "dcDesktop".  See if you are seeing 25 seconds to process the recogFiled Task.
Rule Runner could take longer because of the share location of your batch or if the rrs log are huge and it's writing to a NAS or share location impacted by network bandwidth. 

Go to Taskmaster Application Manager.  In 9.19 you can separate your rrs log from the batch.

What speed are you seeing in 9.16?  Same set of document? 

------------------------------
Blue Devil

Original Message:
Sent: Wed October 25, 2023 07:34 AM
From: Julian Fiegenbaum
Subject: Datacap 9.1.9 iFix 04 performance after migration from 9.1.6 and tuning

Hi community,

when we migrated from 9.1.6 our application took quite a performance hit of taking about twice as long as before, which is sadly not sufficient in the given use case.

There is one task in particular taking up most of the time. It consists of ~75 rulesets each of which fills a particular field with the method experimentally found to work best to get the data. Some rules to differentiate between subtypes of that document, but nothing complicated.

The rrs-logs show no single action taking even a whole second - so the hit is somewhat spread out :/

To be honest, i am a bit lost of how to proceed. I don't think there is much room to tune the rules themselves. Is there something like a delay for the Rulerunner between actions/functions/rules, that can be adjusted? What would be the trade-off?

What are your experiences with datacap 9.1.9 performance? Is this just normal? I saw Shaun had massive (more than x2) problems in DcDesktop, but those got alleviated by installing iFix  04, which is not the case here. Any ideas/input/help would be much appreciated :)

Regards and thanks,

Julian

------------------------------
Julian Fiegenbaum | ISR Information Products AG | Consultant | Germany
------------------------------

Hi Julian / Blue Devil,

I've followed this discussion with interest as we have recently done an upgrade (as you know) to Datacap 9.1.9 from Datacap 8.1. The difference in performance between the two is like night and day, in Rulerunner and DStudio. We had a PMR open because the user-facing function was taking too long to execute under Datacap 9.1.9 ifix 03. IBM Support carried out some tests using the same ruleset and document in both versions and confirmed that there had been a degradation in performance. They then tested with ifix 04 and noticed an improvement so suggested that we test with ifix 04 when it was released.

We did test and although Rulerunner performance improved very slightly, there was a big difference noted in the Find Details performance. 

We never did find out what had changed or why the Rulerunner performance was degraded in recent versions (we also tested 9.1.8 and it has the same issues). In my mind, I ascribed it to the new 64-bit actions, but I've nothing scientific to prove that.

We did notice a lot of these entries in the RRS Log, which you don't seem to have (I believe because you have upgraded to ifix 04):

10:25:11.814 (0)	 DocCount 2,  PageCount 310:25:11.820 (0)	nDefaultRtn default: 110:25:11.821 (0)	cOnRuleEnd: OVERRIDEABLE:False10:25:11.821 (0)	 rrRuleEnd OVERRIDEABLE:False10:25:11.821 (0)	 Rule esec='0.01953'

It's only a few milliseconds each time but in one RRS Log file, we counted around 2200 of these and it does add up. I asked Support what was causing this as it was absent in 8.1 with the same level of logging, but never got a satisfactory answer.

We did notice that these entries seemed to have been removed in iFix04, but I suspect that whatever was causing them is still present.

Shaun

------------------------------
Shaun McDowall

Original Message:
Sent: Tue October 31, 2023 12:10 PM
From: Blue Devil
Subject: Datacap 9.1.9 iFix 04 performance after migration from 9.1.6 and tuning

dStudio is very forgiving and better client to use for testing is thick client "dcDestkop".  If you can isolate that the speed is acceptable with dcDesktop as in 9.16, then you can proceed to Rulerunner as the next suspect. 

This all goes down to the amount of thread(s) and task(s) configured in RuleRunner from batch processing.  This will impact the speed over all.  Best to just have one thread and one task aka  your recogFiled Task to see if there is a different between dcDesktop vs processing in RuleRunner.  Please make sure you are comparing apple to apple meaning it's the same set of rules and action library in 9.16 you are using. 
If some ruleset or action library that's available in 916 but depreciated in 9.19 however a replacement is being used, that will also make a difference. 

------------------------------
Blue Devil

Original Message:
Sent: Tue October 31, 2023 10:40 AM
From: Julian Fiegenbaum
Subject: Datacap 9.1.9 iFix 04 performance after migration from 9.1.6 and tuning

Hi Blue Devil,

thank your for taking the time to look at my problem! Formerly the task took (depending on the document) 11-19s and now it takes about 20-30s. Which does not seem like that much, but is relevant to our client's requirements. 
But yeah - the log left me a little hopeless. There is no single cause and each individual action takes milliseconds. That's why i came here for ideas ;)

I have tried to disable rrs-logging entirely, which did not have a noteworthy impact. We enquired about any differences in specs, including how the system is integrated in the intranet, but were assured that these are exactly the same. I will ask again and be more specific about the share.

The difference in time is notable in DcStudio as well, but maybe that was just my expectations. I will measure the time so i can confirm or refute a RuleRunner specific problem. 

Thank you so much for your suggestions,

Julian

------------------------------
Julian Fiegenbaum | ISR Information Products AG | Consultant | Germany

Original Message:
Sent: Mon October 30, 2023 03:33 PM
From: Blue Devil
Subject: Datacap 9.1.9 iFix 04 performance after migration from 9.1.6 and tuning

The RRS log show start time as  to 13:23:06 and end time at 13:23:31.  That's a total of 25 seconds.   No time gap and they are all in milliseconds.  
These are expensive task meaning require processing time.  

Rule Runner are background processor.  A true test is to run it with thick client such as "dcDesktop".  See if you are seeing 25 seconds to process the recogFiled Task.
Rule Runner could take longer because of the share location of your batch or if the rrs log are huge and it's writing to a NAS or share location impacted by network bandwidth. 

Go to Taskmaster Application Manager.  In 9.19 you can separate your rrs log from the batch.

What speed are you seeing in 9.16?  Same set of document? 

------------------------------
Blue Devil

Original Message:
Sent: Wed October 25, 2023 07:34 AM
From: Julian Fiegenbaum
Subject: Datacap 9.1.9 iFix 04 performance after migration from 9.1.6 and tuning

Hi community,

when we migrated from 9.1.6 our application took quite a performance hit of taking about twice as long as before, which is sadly not sufficient in the given use case.

There is one task in particular taking up most of the time. It consists of ~75 rulesets each of which fills a particular field with the method experimentally found to work best to get the data. Some rules to differentiate between subtypes of that document, but nothing complicated.

The rrs-logs show no single action taking even a whole second - so the hit is somewhat spread out :/

To be honest, i am a bit lost of how to proceed. I don't think there is much room to tune the rules themselves. Is there something like a delay for the Rulerunner between actions/functions/rules, that can be adjusted? What would be the trade-off?

What are your experiences with datacap 9.1.9 performance? Is this just normal? I saw Shaun had massive (more than x2) problems in DcDesktop, but those got alleviated by installing iFix  04, which is not the case here. Any ideas/input/help would be much appreciated :)

Regards and thanks,

Julian

------------------------------
Julian Fiegenbaum | ISR Information Products AG | Consultant | Germany
------------------------------

dStudio is very forgiving and better client to use for testing is thick client "dcDestkop".  If you can isolate that the speed is acceptable with dcDesktop as in 9.16, then you can proceed to Rulerunner as the next suspect. 

This all goes down to the amount of thread(s) and task(s) configured in RuleRunner from batch processing.  This will impact the speed over all.  Best to just have one thread and one task aka  your recogFiled Task to see if there is a different between dcDesktop vs processing in RuleRunner.  Please make sure you are comparing apple to apple meaning it's the same set of rules and action library in 9.16 you are using. 
If some ruleset or action library that's available in 916 but depreciated in 9.19 however a replacement is being used, that will also make a difference. 

Hi Blue Devil,

thank your for taking the time to look at my problem! Formerly the task took (depending on the document) 11-19s and now it takes about 20-30s. Which does not seem like that much, but is relevant to our client's requirements. 
But yeah - the log left me a little hopeless. There is no single cause and each individual action takes milliseconds. That's why i came here for ideas ;)

I have tried to disable rrs-logging entirely, which did not have a noteworthy impact. We enquired about any differences in specs, including how the system is integrated in the intranet, but were assured that these are exactly the same. I will ask again and be more specific about the share.

The difference in time is notable in DcStudio as well, but maybe that was just my expectations. I will measure the time so i can confirm or refute a RuleRunner specific problem. 

Thank you so much for your suggestions,

Julian

The RRS log show start time as  to 13:23:06 and end time at 13:23:31.  That's a total of 25 seconds.   No time gap and they are all in milliseconds.  
These are expensive task meaning require processing time.  

Rule Runner are background processor.  A true test is to run it with thick client such as "dcDesktop".  See if you are seeing 25 seconds to process the recogFiled Task.
Rule Runner could take longer because of the share location of your batch or if the rrs log are huge and it's writing to a NAS or share location impacted by network bandwidth. 

Go to Taskmaster Application Manager.  In 9.19 you can separate your rrs log from the batch.

What speed are you seeing in 9.16?  Same set of document? 

------------------------------
Blue Devil

Original Message:
Sent: Wed October 25, 2023 07:34 AM
From: Julian Fiegenbaum
Subject: Datacap 9.1.9 iFix 04 performance after migration from 9.1.6 and tuning

Hi community,

when we migrated from 9.1.6 our application took quite a performance hit of taking about twice as long as before, which is sadly not sufficient in the given use case.

There is one task in particular taking up most of the time. It consists of ~75 rulesets each of which fills a particular field with the method experimentally found to work best to get the data. Some rules to differentiate between subtypes of that document, but nothing complicated.

The rrs-logs show no single action taking even a whole second - so the hit is somewhat spread out :/

To be honest, i am a bit lost of how to proceed. I don't think there is much room to tune the rules themselves. Is there something like a delay for the Rulerunner between actions/functions/rules, that can be adjusted? What would be the trade-off?

What are your experiences with datacap 9.1.9 performance? Is this just normal? I saw Shaun had massive (more than x2) problems in DcDesktop, but those got alleviated by installing iFix  04, which is not the case here. Any ideas/input/help would be much appreciated :)

Regards and thanks,

Julian

------------------------------
Julian Fiegenbaum | ISR Information Products AG | Consultant | Germany
------------------------------