Hi, 

Expected artifacts were not getting added when I tried to parse an attachment using Email Parser Attachment/Artifact. I ran the resilient circuits in debug mode to see why the artifacts were not getting updated as per the function definition. I understood that the function was working properly, and the logs showed that the data was extracted. However, when the function tries to add it to the incident, the following error is thrown. I'm also attching a screenshot for your perusal. It would be helpful someone could direct me as to what I might be missin.

<task_failure[functionworker] (<task[functionworker] (<function function.__call__.<locals>.decorated.<locals>._call_the_task at 0x7f6885811d90>, <utilities_email_parse[functions.utilities_email_parse] (id=9, workflow=example_email_parsing_artifact, user=k-aravind.menon@capgemini.com) 2020-04-03 11:59:26.472000> incident_id=2148, utilities_parse_email_attachments=True, artifact_id=50)>, (<class 'resilient_circuits.action_message.FunctionException_'>, FunctionException_(), <traceback object at 0x7f6884f32c88>) )>
<exception[*] (<class 'resilient_circuits.action_message.FunctionException_'>, FunctionException_(), [' File "/usr/local/lib/python3.6/site-packages/circuits/core/manager.py", line 856, in processTask\n raise value.extract()\n', ' File "/usr/local/lib/python3.6/site-packages/resilient_circuits/actions_component.py", line 70, in _on_task\n yield result.get()\n', ' File "/usr/lib64/python3.6/multiprocessing/pool.py", line 644, in get\n raise self._value\n', ' File "/usr/lib64/python3.6/multiprocessing/pool.py", line 119, in worker\n result = (True, func(*args, **kwds))\n', ' File "/usr/local/lib/python3.6/site-packages/resilient_circuits/decorators.py", line 97, in _call_the_task\n raise val\n'] handler=None, fevent=<task[functionworker] (<function function.__call__.<locals>.decorated.<locals>._call_the_task at 0x7f6885811d90>, <utilities_email_parse[functions.utilities_email_parse] (id=9, workflow=example_email_parsing_artifact, user=k-aravind.menon@capgemini.com) 2020-04-03 11:59:26.472000> incident_id=2148, utilities_parse_email_attachments=True, artifact_id=50)>)>

------------------------------
K Aravind Menon
------------------------------

We see the error at the bottom. The issue here is that an argument of the wrong data type is being passed as a function parameter. Specifically, the "data" argument given to write_to_tmp_file is a string when it was supposed to be a bytes-like object. On line 130 of utilities_email_parse.py we have:

# Write the attachment.payload to a temp file
path_tmp_file, path_tmp_dir = write_to_tmp_file(data=attachment.get("payload"), tmp_file_name=attachment.get("filename"), path_tmp_dir=path_tmp_dir)

This means that attachment.get("payload") is returning an unexpected data type of str (string), rather than bytes-like object -- which is what is expected since write_to_tmp_file() is opening the file in binary mode in the resilient_common.py (line 358).

I'm unsure where the bug is here-- it could be either in resilient_common.py, utilities_email_parse.py,  or in the pre-processor of the email workflow with how the attachment object is being passed to the script -- As a quick fix, you can modify the utilities_email_parse script like so:

Replace this code:

# Write the attachment.payload to a temp file
	path_tmp_file, path_tmp_dir = write_to_tmp_file(data=attachment.get("payload"),
	tmp_file_name=attachment.get("filename"),
	path_tmp_dir=path_tmp_dir)

With this code:

# Write the attachment.payload to a temp file
	data=attachment.get("payload")  # Check data type if isinstance(data, str): data = data = data.encode()  #Convert string to bytes-like object path_tmp_file, path_tmp_dir = write_to_tmp_file(data,
	tmp_file_name=attachment.get("filename"),
	path_tmp_dir=path_tmp_dir)

​Cheers!

------------------------------
Jared Fagel
Cyber Security Analyst I
Public Utility
------------------------------

What version of fn_utilities are you using? There is a new version, 1.0.12, on the app exchange now.

------------------------------
Mark Scherfling
------------------------------