IBM QRadar SOAR

IBM QRadar SOAR

Join this online user group to communicate across Security product users and IBM experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only

  • 1.  How to avoid Resilient REST API conflict while uploading attachments from two different workflows?

    Posted 03/04/20 02:16 PM
    Hi all,

    I am facing a very core problem with Resilient platform's Python SDK. I have a rule which gets triggered when a new artifact of type "DNS Name" is created. The rule invokes a workflow which has my function that connects to an external API, gets the screenshot of the latest URL under that "DNS Name" artifact and uploads it to the incident as an attachment. Everything works fine when a single artifact is created in the incident.

    The problem occurs when I create multiple artifacts of type "DNS Name" simultaneously from a script. The rule is triggered for each artifact and two separate workflows are launched. When both of these workflows try to upload the attachment, one of them fails and my function in that workflow raises the exception below:

    Traceback (most recent call last):
  File "/home/integration/fn_slashnext/fn_slashnext/components/funct_slashnext_host_report.py", line 76, in _slashnext_host_report_function
    attachment_utils.attach_screenshot(response_list[2], scan_id)
  File "/home/integration/fn_slashnext/fn_slashnext/util/attachment_utils.py", line 17, in attach_screenshot
    self._attach_file(json_response.get('scData').get('scBase64'), file_name, 'image/jpeg', self.incident_id)
  File "/home/integration/fn_slashnext/fn_slashnext/util/attachment_utils.py", line 39, in _attach_file
    incident_id, content_type=content_type)
  File "/home/integration/ibm/lib/python2.7/site-packages/resilient_lib/components/resilient_common.py", line 277, in write_file_attachment
    mimetype=content_type)
  File "/home/integration/ibm/lib/python2.7/site-packages/resilient/co3.py", line 505, in post_attachment
    _raise_if_error(ex.get_response())
  File "/home/integration/ibm/lib/python2.7/site-packages/resilient/co3.py", line 211, in _raise_if_error
    raise SimpleHTTPException(response)
SimpleHTTPException: Conflict:  {"error_code":"generic","hints":[],"message":"Conflict","success":false,"title":null}


    I am using the write_file_attachment function available in resilient_lib package. The class that I have implemented for this is given below (file: attachment_utils.py):

    from io import BytesIO
import base64
from resilient_lib import write_file_attachment


class ResilientAttachments:

    def __init__(self, incident_id, rest_client, logger):
        self.incident_id = incident_id
        self.rest_client = rest_client
        self.logger = logger

    def attach_screenshot(self, json_response, name):
        if json_response.get('errorNo') == 0:
            file_name = name + '.jpeg'
            self.logger.info('Uploading Screenshot Attachment')
            self._attach_file(json_response.get('scData').get('scBase64'), file_name, 'image/jpeg', self.incident_id)
            self.logger.info('Uploading Screenshot Attachment Complete')

    def _attach_file(self, base64_data, file_name, content_type, incident_id):
        datastream = BytesIO(base64.b64decode(base64_data))
        new_attachment = write_file_attachment(self.rest_client, file_name, datastream,
                                               incident_id, content_type=content_type)
        # self.logger.info(json.dumps(new_attachment))

    and then in my function, I use this class in my function code as (only relevant code shown):

    from attachment_utils import ResilientAttachments

attachment_utils = ResilientAttachments(incident_id, self.rest_client(), self.logger)
attachment_utils.attach_screenshot(response_list[2], scan_id)

    My guess is that the Resilient REST API, that the resilient_lib is using at back-end is not thread-safe and when two workflows simultaneously try to upload an attachment, one of them faces a conflict and has to abort. Can someone please point out any workaround for this issue as it is disturbing the core-functionality of Resilient and there is no way I can create my playbook without multiple workflows running simultaneously on different artifacts. Any help will be highly appreciated. 

    Thanks!



    ------------------------------
    Umair Ahmed
    ------------------------------


  • 2.  RE: How to avoid Resilient REST API conflict while uploading attachments from two different workflows?

    Posted 03/05/20 07:40 AM
    Edited by Mark Scherfling 03/05/20 07:40 AM
    Hi Umair,

    Your analysis may be correct. Can you open a support case with the Resilient log file so we can review? 

    Regards,
    Mark

    ------------------------------
    Mark Scherfling
    ------------------------------



  • 3.  RE: How to avoid Resilient REST API conflict while uploading attachments from two different workflows?
    Best Answer

    Posted 03/06/20 08:15 AM
    Edited by Umair Ahmed 03/06/20 01:12 PM
    Hi Umair,

    This is in response to the message you've sent to me on a different post (which I was unable to find, but it's not important).

    The problem you are facing is not related to thread-safety. You are getting this:
    SimpleHTTPException: Conflict:  {"error_code":"generic","hints":[],"message":"Conflict","success":false,"title":null}​


    This is basically a race condition between the two invocations to your workflow: by the time the second action wants to update the incident, the first one has performed an update already.

    I'm not using resilient_lib but I have implemented similar functions to add attachments to Resilient and what I've done is handle the "resilient.SimpleHTTPException" (which is just a subclass of a "requests" exception) and check the status code. If it's Conflict 429 (or any other you decide you want to retry the operation), we raise the exception to let a retry decorator catch this and retry the operation as many times as we want and with the delay we want and so on. It looks similar to this:

    @retry(exceptions=resilient.co3.SimpleHTTPException, tries=5, delay=10, backoff=2, logger=logging.getLogger(__name__))
def _add_attachment(self, attachment_filepath):
    try:
        self.rest_client.post_attachment(attachment_uri, attachment_filepath)
        return True
    except resilient.co3.SimpleHTTPException as e:
        if e.response.status_code in [409]:
            # Raise the exception so the retry decorator can see it
            raise e
        else:
            return False

    I hope this helps but feel free to reply to this message if you have troubles.
    ------------------------------

    Regards,
    Carlos Ortigoza
    ------------------------------



  • 4.  RE: How to avoid Resilient REST API conflict while uploading attachments from two different workflows?

    Posted 03/06/20 10:55 AM
    Hey Carlos,

    I tried your solution and it worked like a charm. It's really a very simple solution to this problem but I have observed that the "tries", "delay" and "backoff" parameters of the retry decorator need to be tuned according to the number of parallel workflows running. For some values of these parameters and a large number of workflows, I still saw some conflicts going on.

    Can you offer any insights on how should I tweak these parameters?

    Finally, I believe this approach should not have any impact on the performance of the platform since we are only increasing the number of requests to the REST server and nothing else. Please correct me if otherwise. 

    Thanks a lot for your valuable support!

    ------------------------------
    Umair Ahmed
    ------------------------------

    Original Message


  • 5.  RE: How to avoid Resilient REST API conflict while uploading attachments from two different workflows?

    Posted 03/06/20 11:10 AM
    Hi Umair,

    Indeed, you have to set those parameters according to your needs. There will always be the chance to keep having collisions and the more parallel actions running, the more likely the situation will be. The key in these cases is to also make use of a random jitter to improve the chances that these "workers" get away from each others:
    :param jitter: extra seconds added to delay between attempts. default: 0.
               fixed if a number, random if a range tuple (min, max)​

    By passing a tuple you will introduce randomness but you will still have to find the optimal values depending on the conditions of your environment.

    ------------------------------
    Regards,
    Carlos Ortigoza
    ------------------------------

    Original Message


  • 6.  RE: How to avoid Resilient REST API conflict while uploading attachments from two different workflows?

    Posted 03/06/20 01:12 PM
    Carlos,

    Can you share the retry decorator code that you are using? I found one online that does not have the jitter code and I am currently short on time to write my own. Thanks!

    ------------------------------
    Umair Ahmed
    ------------------------------

    Original Message


  • 7.  RE: How to avoid Resilient REST API conflict while uploading attachments from two different workflows?

    Posted 03/06/20 02:18 PM
    Hi Umair,

    It's this one: https://pypi.org/project/retry/

    ------------------------------
    Regards,
    Carlos Ortigoza
    ------------------------------

    Original Message


Related Content