IBM Security QRadar SOAR

 View Only
  • 1.  Function - Utilities: Shell Command usage question

    Posted Mon February 10, 2020 09:41 AM

    <lightning-formatted-text data-aura-rendered-by="2:820;a">Hello,

    I'm wondering if it's possible to pass an attachment or file attached to an artifact, as a parameter into the "Shell Command" function?

    Per the Utilities documentation, it looks like you can pass TEXT as an input parameter (shell_param), but what about files attached to a case in Attachments or files attached to Artifacts (Email Attachment or Other File artifact types)?

    https://exchange.xforce.ibmcloud.com/api/hub/extensionsNew/5d24a4af4b149938ab8f7cb462eec46a/fn_utilities-v1.0.10.pdf

    Purpose: I want pass an .eml or .msg as a parameter to have a Shell command extract the full email header. (the current email parse functions only grab certain fields and add them to artifacts\doesnt do everything we want)

    I'm open to alternative suggestions on how to accomplish the 'full' header parsing\extraction as well, so we can add the entire email header to the Notes on an Incident.

    Thank you.</lightning-formatted-text>



    ------------------------------
    William Pope
    ------------------------------


  • 2.  RE: Function - Utilities: Shell Command usage question

    Posted Mon February 10, 2020 03:21 PM

    Reposting my comment because it did not get posted for some reason.  (Following up with support on that)

    ----------------------------
    Is it possible to pass a file as a parameter to the Shell Command function?

    I see in the documentation that you can at least pass TEXT, but I would like to pass an attachment or certain types of artifacts that allow you to attach files to the artifacts.

    I also saw in the doc that when using Shell Command and Volatility, the example showed calling memdump on disk, but in my case, I want to pass incident attachments\artifacts.

    Purpose:  I would like to pass .msg or .eml files to the function, that would then get parsed by a shell command (tool or python script) setup on the circuits box that can extract the full email header.  Currently, the Utilities: Email Parser function only parses\adds artifacts from the header and adds the email body to Notes.  My end goal is to get the full email header visible in Notes, so if there is a better idea than using a Shell command, please recommend.  I'm also looking into dev'ing a Function or Custom Action to do this.  I'm a bit new to the platform and I'm just exploring my options at this point basically.



    ------------------------------
    William Pope
    ------------------------------



  • 3.  RE: Function - Utilities: Shell Command usage question

    Posted Tue February 11, 2020 05:02 AM
    Hi William,

    https://pypi.org/project/mail-parser/ holds the key info, if you want to add this functionality to the existing code, simply edit the
    utilities_email_parse.py​
    script to parse out the header:
    parsed_email.headers
    
    ​
    examine the structure of this and sanitize before returning the result to the resultpayload object and posting to a note. No doubt some testing will be needed but probably the most straightforward solution.

    ------------------------------
    Sean OGorman
    ------------------------------



  • 4.  RE: Function - Utilities: Shell Command usage question

    Posted Thu February 13, 2020 01:58 PM
    Thank you for the suggestion.  Will let you know how the mail-parser idea goes.

    ------------------------------
    William Pope
    ------------------------------



  • 5.  RE: Function - Utilities: Shell Command usage question

    Posted Sun February 16, 2020 09:03 PM
    Hello.  I tried your suggestion, but I'm hitting a wall at this point.  Keep in mind, I just started getting into Python...and Resilient...double whammy!

    Here's where I'm at so far.

    After playing with mailparser and figuring out how that generally works, I added "parsed_email_dict["full_header"] = parsed_email.headers" to the "utilities_email_parse.py" script (see below).  I tried both with the "headers_json" property and without.  Using a Function in Resilient, I edited the post-process script to simply put the results in a Note for testing..."incident.addNote(str(results))" is what I'm using...but unfortunately the results of the ["full_header"] section in those results are always unordered.  I know this has to do with the attributes going into a dictionary, but I haven't figured out exactly why or how to change that so the full header displays in the correct order.

    Looking for some guidance\assistance.

    -----------------------------------------------------------
    if parsed_email is not None:
    if not parsed_email.mail:
    reason = u"Raw email in unsupported format. Failed to parse {0}".format(u"provided base64content" if fn_inputs.get("base64content") else attachment_metadata.get("name"))
    yield StatusMessage(reason)
    results = rp.done(success=False, content=None, reason=reason)

    else:
    # Load all parsed email attributes into a Python Dict
    parsed_email_dict = json.loads(parsed_email.mail_json, encoding="utf-8")
    parsed_email_dict["plain_body"] = parsed_email.text_plain_json
    parsed_email_dict["html_body"] = parsed_email.text_html_json
    parsed_email_dict["full_header"] = parsed_email.headers
    yield StatusMessage("Email parsed")


    ------------------------------
    William Pope
    ------------------------------



  • 6.  RE: Function - Utilities: Shell Command usage question

    Posted Mon February 17, 2020 04:17 AM
    As you said before, it's because of Python (2) dictionaries. They don't preserve order. What you can do is to manually order the output of your post-process by printing your values in the desired order. Of course, if there are too many keys it could a pain you know where.

    ------------------------------
    Bruce Wayne
    Senior Dark Knight
    ------------------------------



  • 7.  RE: Function - Utilities: Shell Command usage question

    Posted Mon February 17, 2020 11:40 AM

    Thanks for the feedback Bruce.

    Manually ordering a full header doesn't really seem like an option, as there are so many header fields that I may not be able to account for.  There are some that are constants of course like To, From, Received, etc, but there are others we would be missing.

    I know "parsed_email.headers" works (I tested the functionality outside of the script), its just not working how im using it in the existing script?

    Again, still looking for some guidance\assistance on how to bring in that full header in an ordered way\not an unordered dictionary result.



    ------------------------------
    William Pope
    ------------------------------