Hi Liam,
stomp_prefetch_limit is all about performance. The more messages which are cached early, than the faster they can be processed through your num_worker threads. Typically, I have not see a need to tune this value up as most application functions are IO bound and the time to respond is solely a factor on the datasource API calls.
------------------------------
Mark Scherfling
------------------------------
Original Message:
Sent: Wed June 15, 2022 12:50 PM
From: Liam Mahoney
Subject: Pause inside a playbook script
@Mark Scherfling am I understanding correctly that the `stomp_prefetch_limit` limits the number of messages received by the Resilient circuits server?
We have changed `num_workers` to be higher than the default on some of our apps (maybe around 50) but we haven't changed `stomp_prefetch_limit`. Does this mean the app is able to spin up `num_workers` number of threads, but circuits will only handle `stomp_prefetch_limit` number of messages at a time?
Also it looks like the default for `stomp_prefetch_limit` is set to 20
------------------------------
Liam Mahoney
Original Message:
Sent: Wed June 15, 2022 10:31 AM
From: Mark Scherfling
Subject: Pause inside a playbook script
We have another [resilient] app.config setting which may help: stomp_prefetch_limit. This setting decides how many messages are consumed and cached before dispatching an function. In your case, tuning down the num_workers and the stomp_prefetch_limit (default may be 100) will slow function processing. Unfortunately, it will effect all functions, not just VirusTotal.
Good luck,
Mark
------------------------------
Mark Scherfling
Original Message:
Sent: Wed June 15, 2022 08:28 AM
From: Alexey Fedorov
Subject: Pause inside a playbook script
Hello Mark,
Thank you for the answer. My case is a little bit complicated. I want to check some artifacts (like hashes) via VirusTotal function, but I have no commercial API key and can't use it inside Threat Sources. I have some restrictions - only 4 lookup per minute and I need to pause a process of creation artifact for prevent lookup quota exceed. Usually my cases came from QRadar and has one or two hashes, but sometime it can be 10 or 20. Maybe you can recommend to me some way to manage the issue? I created artifacts inside cycle in a script.
------------------------------
Alexey Fedorov
Original Message:
Sent: Wed June 15, 2022 07:38 AM
From: Mark Scherfling
Subject: Pause inside a playbook script
Hi Alexey,
There is no way to pause a playbook using a built-in capability. You should be able to use the timer function anywhere within a playbook, not just after another script or function. Be aware that each timer function consumes a thread of execution for the duration. So, it's easy to run out of the default 10 threads which can delay other functions executions. The solution is to increase the number of threads specified in the [resilient] section of your app.config file such as:
[resilient]
num_workers=50
Good luck
Mark
------------------------------
Mark Scherfling
Original Message:
Sent: Tue June 14, 2022 04:52 AM
From: Alexey Fedorov
Subject: Pause inside a playbook script
Hello,
Is it a way how I can make a pause inside a playbook script? I know about the Timer function, but I can call it only after script/another function.
When I try to use time.sleep command I got an error:
ResilientSecurityException: Unauthorized call of function "sleep"
------------------------------
Alexey Fedorov
------------------------------