IBM Security QRadar SOAR

 View Only
Expand all | Collapse all

Pre/post process script import limitations (parsing lists, ast workaround?)

  • 1.  Pre/post process script import limitations (parsing lists, ast workaround?)

    Posted Thu June 03, 2021 11:10 AM
    Is there a recommended workaround for not being able to import ast in any of the scripts?

    Since Resilient cannot store lists in its fields, I'm looking for a straight-forward way to parse list literals for custom data tables.

    I'm using the default "hosts" (incident.properties.hosts) field to store a list literal '["endpoint1","endpoint2"]' and need to parse this outside of function calls.

    Is 're' the only import allowed for parsing?

    ------------------------------
    Tom Prenderville
    ------------------------------


  • 2.  RE: Pre/post process script import limitations (parsing lists, ast workaround?)

    Posted Fri June 04, 2021 05:37 AM
    Hi Tom,

    What might help you is the json module which is one of the approved modules for use in Python 3 scripts
    Here is an example of how you could use it to parse a list from a string.
    # replace with incident.properties.hosts
endpoints = '["endpoint1","endpoint2"]'

import json
parsed_list = json.loads(endpoints)
log.info(parsed_list)​

    I found this answer https://stackoverflow.com/a/35461204 which notes you might have to replace single quotes with double quotes, but for your example that should not be needed.



    ------------------------------
    Sean Mc Cann
    ------------------------------

    Original Message


  • 3.  RE: Pre/post process script import limitations (parsing lists, ast workaround?)

    Posted Fri June 04, 2021 08:10 AM
    Just continuing from what Sean suggested....

    It probably will be more effective if the data stored in the fields is a JSON string instead of python string representation of objects. That will give more flexibility in the long term.

    Ben

    ------------------------------
    Ben Lurie
    ------------------------------

    Original Message


  • 4.  RE: Pre/post process script import limitations (parsing lists, ast workaround?)

    Posted Mon June 07, 2021 11:40 AM
    Thank you, both. JSON string is indeed necessary. 

    Is there an intuitive way to debug pre-process scripts? I've added "log.debug('data')" and "log.info(object)" but not sure where to view this output if the incident does not get created.

    Here's my workflow:

    python script fetches 3rd party incidents on app-host via cron job -> creates Resilient Incident -> Rule triggers workflow if 'Hosts' field is not 'None' -> Workflow pre-process script parses 'incident.properties.hosts' and 1) adds each host to a new row in a Data Table, 2) passes parsed data as input to function for retrieving more data about the host.

    Right now, the pre-process script fails with 'index out of range' and the incident does not get created (strange?) so there is no "Actions" menu available to see any output for workflows... Any recommendations?

    ------------------------------
    Tom Prenderville
    ------------------------------

    Original Message


  • 5.  RE: Pre/post process script import limitations (parsing lists, ast workaround?)

    Posted Tue June 08, 2021 05:43 AM
    Debugging workflows is a problem area that we will hopefully improve in a coming release by showing the log.info() statements in the scripting log file even if the script fails. At the moment the only way I know to get some kind of debug output is to use incident.addNote(str(object)) or something similar to add the contents you want to debug to an incident. This would only work if the script succeeds so you would have to edit any lines that are failing.

    Unfortunately I don't know of a more intuitive way to debug workflows at the moment.

    ------------------------------
    Sean Mc Cann
    ------------------------------

    Original Message


  • 6.  RE: Pre/post process script import limitations (parsing lists, ast workaround?)

    Posted Mon June 07, 2021 05:12 PM
    When trying to import json:
    <x-row>
    Bad Request:  {"success":false,"title":null,"message":"Pre-processing script for
 Function 'get_endpoint_info' from Workflow 'get_endpoint_wf' was unable to complete because: Script contains t
he following illegal imports: json","hints":[],"error_code":"generic"}​
    </x-row>
    it works in a post-process script for another workflow.. and I see "import json" in pre-process scrip code example: https://github.com/ibmresilient/resilient-scripts/blob/master/python3/workflow/pre-process-task-utils.py

    Any idea's why I'm getting an error when importing json lib?

    ------------------------------
    Tom Prenderville
    ------------------------------

    Original Message


  • 7.  RE: Pre/post process script import limitations (parsing lists, ast workaround?)

    Posted Tue June 08, 2021 05:06 AM
    Hi Tom,

    Make sure your script language is set to Python 3. Importing json is not available in Python 2.

    ------------------------------
    Sean Mc Cann
    ------------------------------

    Original Message


  • 8.  RE: Pre/post process script import limitations (parsing lists, ast workaround?)

    Posted Tue June 08, 2021 09:01 AM
    Thanks, Sean. Where is this set? I do not have the option to select a Python version from the pre-process script pane.

    is this a global setting somewhere?

    ------------------------------
    Tom Prenderville
    ------------------------------

    Original Message


  • 9.  RE: Pre/post process script import limitations (parsing lists, ast workaround?)

    Posted Tue June 08, 2021 10:07 AM
    Are you using SOAR version 38.2 or above? If so, Python 3 should be selectable for pre/post processing scripts, conditions or any standalone scripts in the script pane you have shown. You said it works for a post process script in a different workflow, does that one have a selectable dropdown?

    ------------------------------
    Sean Mc Cann
    ------------------------------

    Original Message


  • 10.  RE: Pre/post process script import limitations (parsing lists, ast workaround?)

    Posted Tue June 08, 2021 03:23 PM
    Upgrading from 38.1 to 38.2 fixed it. Thank you for all of your responses and help!

    ------------------------------
    Tom Prenderville
    ------------------------------

    Original Message