IBM Security Resilient

Expand all | Collapse all

Function - Utilities: Shell Command usage question

  • 1.  Function - Utilities: Shell Command usage question

    Posted 12 days ago



    ------------------------------
    William Pope
    ------------------------------


  • 2.  RE: Function - Utilities: Shell Command usage question

    Posted 11 days ago

    Reposting my comment because it did not get posted for some reason.  (Following up with support on that)

    ----------------------------
    Is it possible to pass a file as a parameter to the Shell Command function?

    I see in the documentation that you can at least pass TEXT, but I would like to pass an attachment or certain types of artifacts that allow you to attach files to the artifacts.

    I also saw in the doc that when using Shell Command and Volatility, the example showed calling memdump on disk, but in my case, I want to pass incident attachments\artifacts.

    Purpose:  I would like to pass .msg or .eml files to the function, that would then get parsed by a shell command (tool or python script) setup on the circuits box that can extract the full email header.  Currently, the Utilities: Email Parser function only parses\adds artifacts from the header and adds the email body to Notes.  My end goal is to get the full email header visible in Notes, so if there is a better idea than using a Shell command, please recommend.  I'm also looking into dev'ing a Function or Custom Action to do this.  I'm a bit new to the platform and I'm just exploring my options at this point basically.



    ------------------------------
    William Pope
    ------------------------------



  • 3.  RE: Function - Utilities: Shell Command usage question

    Posted 11 days ago
    Hi William,

    https://pypi.org/project/mail-parser/ holds the key info, if you want to add this functionality to the existing code, simply edit the
    utilities_email_parse.py​
    script to parse out the header:
    parsed_email.headers
    
    ​
    examine the structure of this and sanitize before returning the result to the resultpayload object and posting to a note. No doubt some testing will be needed but probably the most straightforward solution.

    ------------------------------
    Sean OGorman
    ------------------------------



  • 4.  RE: Function - Utilities: Shell Command usage question

    Posted 9 days ago
    Thank you for the suggestion.  Will let you know how the mail-parser idea goes.

    ------------------------------
    William Pope
    ------------------------------



  • 5.  RE: Function - Utilities: Shell Command usage question

    Posted 5 days ago
    Hello.  I tried your suggestion, but I'm hitting a wall at this point.  Keep in mind, I just started getting into Python...and Resilient...double whammy!

    Here's where I'm at so far.

    After playing with mailparser and figuring out how that generally works, I added "parsed_email_dict["full_header"] = parsed_email.headers" to the "utilities_email_parse.py" script (see below).  I tried both with the "headers_json" property and without.  Using a Function in Resilient, I edited the post-process script to simply put the results in a Note for testing..."incident.addNote(str(results))" is what I'm using...but unfortunately the results of the ["full_header"] section in those results are always unordered.  I know this has to do with the attributes going into a dictionary, but I haven't figured out exactly why or how to change that so the full header displays in the correct order.

    Looking for some guidance\assistance.

    -----------------------------------------------------------
    if parsed_email is not None:
    if not parsed_email.mail:
    reason = u"Raw email in unsupported format. Failed to parse {0}".format(u"provided base64content" if fn_inputs.get("base64content") else attachment_metadata.get("name"))
    yield StatusMessage(reason)
    results = rp.done(success=False, content=None, reason=reason)

    else:
    # Load all parsed email attributes into a Python Dict
    parsed_email_dict = json.loads(parsed_email.mail_json, encoding="utf-8")
    parsed_email_dict["plain_body"] = parsed_email.text_plain_json
    parsed_email_dict["html_body"] = parsed_email.text_html_json
    parsed_email_dict["full_header"] = parsed_email.headers
    yield StatusMessage("Email parsed")


    ------------------------------
    William Pope
    ------------------------------



  • 6.  RE: Function - Utilities: Shell Command usage question

    Posted 5 days ago
    As you said before, it's because of Python (2) dictionaries. They don't preserve order. What you can do is to manually order the output of your post-process by printing your values in the desired order. Of course, if there are too many keys it could a pain you know where.

    ------------------------------
    Bruce Wayne
    Senior Dark Knight
    ------------------------------



  • 7.  RE: Function - Utilities: Shell Command usage question

    Posted 5 days ago

    Thanks for the feedback Bruce.

    Manually ordering a full header doesn't really seem like an option, as there are so many header fields that I may not be able to account for.  There are some that are constants of course like To, From, Received, etc, but there are others we would be missing.

    I know "parsed_email.headers" works (I tested the functionality outside of the script), its just not working how im using it in the existing script?

    Again, still looking for some guidance\assistance on how to bring in that full header in an ordered way\not an unordered dictionary result.



    ------------------------------
    William Pope
    ------------------------------