IBM Security QRadar SOAR

 View Only

  • 1.  Text Area Field in json

    Posted Tue February 05, 2019 09:34 PM
    I have a problem using the "Call REST API" function

    1- I add a field of type "text area" inside a rule (see image 1a.png and 1b.png)
    2- Within the workflow I call a function called "Call REST API" and in the pre-script I refer to the field "description" created in point 1. (see image 2.png)

    3- Then the verifiable "Description1" should be entered as part of the "inputs.rest_body" (see image 3.png)

    4- Finally I will perform a test of the action "Abrir Peticion" that calls the workflow of point 3. Request that you complete the fields of point 1. As an example, enter text multi line (see image 4a.pgn and 4b.png )

    As a result I get an error because I can not interpret the json (see image log.png)

    The response of the rest:
    2019-02-05 22:59:00,882 DEBUG [actions_component] Result: {'cookies': {}, 'links': {}, 'text': u'{\n "Messages": ["JSONException: Unterminated string at character 304"],\n "ReturnCode": "-5"\n}', 'elapsed': 61, 'apparent_encoding': 'ascii', 'reason': 'Bad Request', 'ok': False, 'url': u'http://10.4.0.14:25001/SM/9/rest/ResilientQ/', 'headers': {'Content-Length': '95', 'X-Content-Type-Options': 'nosniff', 'Keep-Alive': 'timeout=1200000, max=1000', 'Server': 'Apache-Coyote/1.1', 'Connection': 'Keep-Alive, close', 'Date': 'Wed, 06 Feb 2019 02:14:06 GMT', 'Content-Type': 'application/json;charset=utf-8'}, 'json': {u'ReturnCode': u'-5', u'Messages': [u'JSONException: Unterminated string at character 304']}, 'status_code': 400}

    If the json be formed as follows this would work:
    2019-02-05 22:59:00,307 INFO [utilities_call_rest_api] rest_body:
    {
    "ResilientQ": {
    "Categoria": "Seguridad",
    "Subcategoria": "Seguridad -- Antivirus -- Consulta",
    "GrupoAsignado": "SEGURIDAD-ANTIVIRUS",
    "SolicitadoPara": "falcon",
    "FechaRequerida": "2019-02-28T03:00:00",
    "SolicitadoPor": "falcon",
    "Descripcion": ["Test of a text line 1\nTest of a text line 2\nTest of a text line 3\nTest of a text line 4"],
    "Titulo": "Prueba",
    "GrupoRegistradoPor": "SEGURIDAD-ANTIVIRUS",
    "Fase": "Registro"
    }
    }

    Another error occurs if the text has special characters such as \ or / or "

    for example (see image 5.png), in the app.log:
    2019-02-05 23:08:24,764 INFO [utilities_call_rest_api] rest_body:
    {
    "ResilientQ": {
    "Categoria": "Seguridad",
    "Subcategoria": "Seguridad -- Antivirus -- Consulta",
    "GrupoAsignado": "SEGURIDAD-ANTIVIRUS",
    "SolicitadoPara": "falcon",
    "FechaRequerida": "2019-02-28T03:00:00",
    "SolicitadoPor": "falcon",
    "Descripcion": ["Test of a text "line" 1
    Test of a text line 2
    Test of a text \line\ 3
    Test of a text /line"/ 4"],
    "Titulo": "Prueba",
    "GrupoRegistradoPor": "SEGURIDAD-ANTIVIRUS",
    "Fase": "Registro"
    }
    }

    2019-02-05 23:08:25,165 DEBUG [actions_component] Result: {'cookies': {}, 'links': {}, 'text': u'{\n "Messages": ["JSONException: Expected a \',\' or \']\' at character 299"],\n "ReturnCode": "-5"\n}', 'elapsed': 6, 'apparent_encoding': 'ascii', 'reason': 'Bad Request', 'ok': False, 'url': u'http://10.4.0.14:25001/SM/9/rest/ResilientQ/', 'headers': {'Content-Length': '97', 'X-Content-Type-Options': 'nosniff', 'Keep-Alive': 'timeout=1200000, max=1000', 'Server': 'Apache-Coyote/1.1', 'Connection': 'Keep-Alive, close', 'Date': 'Wed, 06 Feb 2019 02:23:34 GMT', 'Content-Type': 'application/json;charset=utf-8'}, 'json': {u'ReturnCode': u'-5', u'Messages': [u"JSONException: Expected a ',' or ']' at character 299"]}, 'status_code': 400}

    Can you help me solve this problem? Maybe with some regular expression or replace we can do something.

    Thank you!



    ------------------------------
    Juan Cruz Del Col
    ------------------------------


  • 2.  RE: Text Area Field in json

    Posted Wed February 06, 2019 09:39 AM
    The description is not valid json and will throw and error. The characters you reference \ or / or " will need to to be escaped and newlines transformed as well . It will require a bespoke solution and as you specified using regex in the pre-processing script may be  a possible option. The re module is available in the pre and post processing scripts so that could be used. Do you need help with a regex pattern?

    ------------------------------
    JOHN PRENDERGAST
    ------------------------------

    Original Message


  • 3.  RE: Text Area Field in json
    Best Answer

    Posted Wed February 06, 2019 01:16 PM
    I've asked Juan to try the following in the pre-process script:

    description = rule.properties.description["content"]

    inputs.description = """
    {
     "ResilientQ": {
       "description": [""" + repr(description) + """ ],
     }
    }
    """

    According to Juan, the above worked.

    ------------------------------
    Romina Jose
    ------------------------------

    Original Message


  • 4.  RE: Text Area Field in json

    Posted Wed February 13, 2019 09:05 AM
    Romina, Romina, I would like to send a similar text to the attachment and it fails me, I have tried it in many ways and it always fails.

    In the "Workflow status" error:

    An error occurred while processing the action acknowledgement. Additional information: Pre-processing script for Function 'Utilities: Call REST API' from Workflow 'AbrirPricion' was unable to complete because: UnicodeEncodeError('ascii', u'"Estimados,\n\n \n\nMicrosoft lanz\xf3 actualizaciones de seguridad para algunos de sus productos, ya que se han descubierto diversas vulnerabilidades, la m\xe1s grave podr\xeda permitir la ejecuci\xf3n de c\xf3digo arbitrario en la m\xe1quina cliente.\n\n \n\nHasta el momento ninguna de las vulnerabilidades solucionadas, se est\xe1 explotando activamente. Dentro de las m\xe1s importantes, se encuentra la CVE-2019-0547 , una debilidad en el componente de Windows responsable de asignar direcciones de Internet a las computadoras host (tambi\xe9n conocido como "cliente DHCP de Windows"). \n\n \n\n \n\n \n\nProductos afectados:\n\n \n\n\u2022 Adobe Flash Player\n\n\u2022 Internet Explorer\n\n\u2022 Microsoft Edge\n\n\u2022 Microsoft Windows\n\n\u2022 Microsoft Office, Microsoft Office Services y aplicaciones web\n\n\u2022 ChakraCore\n\n\u2022 .NET Framework\n\n\u2022 Microsoft Dynamics NAV\n\n\u2022 Microsoft Exchange Server\n\n\u2022 Microsoft Visual Studio\n\n\u2022 Windows Azure Pack (WAP)\n\n \n\n \n\nActualizaciones a aplicar\n\nVer listado aqu\xed: https://portal.msrc.microsoft.com/en-us/security-guidance/summary\n\n \n\n \n\nRecomendaciones\n\nAplicar las actualizaciones de seguridad correspondientes, tomando en cuenta las ventanas de tiempo establecidas para las pruebas."', 30, 31, 'ordinal not in range(128)')


    ------------------------------
    Juan Cruz Del Col
    ------------------------------

    Original Message


  • 5.  RE: Text Area Field in json

    Posted Thu February 14, 2019 03:30 AM
    Hi Juan,

    If the function returns results that are of Type Unicode, you need to pay particular attention when concatenating "strings" together in the post-process script.

    For example, we have a function that returns:
    results = {
  "event_name": જ ઝ ઞ ટ ઠ ડ ઢ ણ ત થ દ 
}​

    and in our Workflow's post-process script, we want to append the event_name to the name of the incident. So we do the following:

    custom_incident_name = u"{0} - {1}".format(unicode(results.event_name), unicode(incident.name))

incident.name = custom_incident_name


    Take note of how we "convert" everything to unicode when we want to concatenate.

    Hope this helps!



    ------------------------------
    Shane Curtin
    Integrations Engineer - IBM Resilient
    ------------------------------

    Original Message


  • 6.  RE: Text Area Field in json

    Posted Thu February 14, 2019 03:22 PM
    Shane, I have the problem with double quotes.
    Do the following:
    description1 = rule.properties.description ['content']
description2 = u "{0}". format (unicode (description1))
description3 = description2.replace ('\ "', '\\"')​

    The result is:
    {
  "ResilientQ": {
    "Categoria": "Seguridad",
    "Subcategoria": "Seguridad -- Antivirus -- Consulta",
    "GrupoAsignado": "SEGURIDAD-ANTIVIRUS",
    "SolicitadoPara": "falcon",
    "FechaRequerida": "2019-02-14T20:16:26",
    "SolicitadoPor": "falcon",
    "Descripcion": ["u'Estimados,\n\n \n\nMicrosoft lanz\xf3 actualizaciones de seguridad para algunos de sus productos, ya que se han descubierto diversas vulnerabilidades, la m\xe1s grave podr\xeda permitir la ejecuci\xf3n de c\xf3digo arbitrario en la m\xe1quina cliente.\n\n \n\nHasta el momento ninguna de las vulnerabilidades solucionadas, se est\xe1 explotando activamente. Dentro de las m\xe1s importantes, se encuentra la CVE-2019-0547 , una debilidad en el componente de Windows responsable de asignar direcciones de Internet a las computadoras host (tambi\xe9n conocido como \\"cliente DHCP de Windows\\"). \n\n \n\n \n\n \n\nProductos afectados:\n\n \n\n\u2022           Adobe Flash Player\n\n\u2022           Internet Explorer\n\n\u2022           Microsoft Edge\n\n\u2022           Microsoft Windows\n\n\u2022           Microsoft Office, Microsoft Office Services y aplicaciones web\n\n\u2022           ChakraCore\n\n\u2022           .NET Framework\n\n\u2022           Microsoft Dynamics NAV\n\n\u2022           Microsoft Exchange Server\n\n\u2022           Microsoft Visual Studio\n\n\u2022           Windows Azure Pack (WAP)\n\n \n\n \n\nActualizaciones a aplicar\n\nVer listado aqu\xed: https://portal.msrc.microsoft.com/en-us/security-guidance/summary\n\n \n\n \n\nRecomendaciones\n\nAplicar las actualizaciones de seguridad correspondientes, tomando en cuenta las ventanas de tiempo establecidas para las pruebas.'" ],
    "Titulo": "asd",
    "GrupoRegistradoPor": "SEGURIDAD-ANTIVIRUS",
    "Fase": "Registro"
  }
}​

    My problem is in
    \\ "Windows DHCP client \\"
    should be
    \"Windows DHCP client \"

    ------------------------------
    Juan Cruz Del Col
    ------------------------------

    Original Message


  • 7.  RE: Text Area Field in json

    Posted Fri February 15, 2019 07:03 AM
    Try the following:

    description1 = rule.properties.description['content']

description1 = description1.replace(u'"', u'\\"')
description1 = description1.replace(u"'", u"\\'")
description1 = description1.replace(u'\\', u'\\\\')
description1 = description1.replace(u'\n', u'\\n')
description2 = u"{0}". format(unicode(description1))

​


    ------------------------------
    Shane Curtin
    Integrations Engineer - IBM Resilient
    ------------------------------

    Original Message


  • 8.  RE: Text Area Field in json

    Posted Fri February 15, 2019 08:35 AM
    Shane, I had faith!
    but I do not know why I have this error:

    State: Anomalous
    User: System User
    Reason: An error occurred while processing the action acknowledgment. Additional information: Pre-processing script for Function 'Utilities: Call REST API' from Workflow 'AbrirPeticion' was unable to complete because: 'NoneType' object is unsubscriptable

    Attached pre and post script

    ------------------------------
    Juan Cruz Del Col
    ------------------------------

    Original Message


  • 9.  RE: Text Area Field in json

    Posted Thu February 21, 2019 01:30 PM
    Hi Juan,

    So I believe we are over the unicode and quotes handling issue

    There error you see relates to something in the pre-process script being None or undefined

    I was able to replicate your script in "true python" below.

    Two things to consider:
    • Your activity fields. Ensure they are defined
    • The workflow.peoperties.fech_salida.stout. Ensure this has a value too

    class inputs(object):
    def __init__(self):
        self.rest_body = None

class fecha_salida(object):
    def __init__(self):
        self.stdout = "test stdout"

class theproperties(object):
    def __init__(self):
        self.gruposm = "test gruposm"
        self.description = {
            "content": "test description"
        }
        self.titulo = "test titulo"
        self.fecha_salida = fecha_salida()

class arule(object):
   def __init__(self):
       self.properties = theproperties()

rule = arule()
workflow = arule()
inputs = inputs()

Grupete = str('"'+rule.properties.gruposm+'"')
#Descripcion1 = str(rule.properties.descripcion['content'])
#t1 = '\\'
#t2 = 'n'
#desc = str('"'+Descripcion1.replace("\r\n", t1+t2)+'"')

description1 = rule.properties.description['content']

description2 = description1.replace(u'"', u'\\"')
description3 = description2.replace(u"'", u"\\'")
description4 = description3.replace(u'\\', u'\\\\')
description5 = description4.replace(u'\n', u'\\n')
description6 = u"{0}". format(unicode(description5))

Titulo = str('"'+rule.properties.titulo+'"')
note_text = str('"'+workflow.properties.fecha_salida.stdout+'"')
text = workflow.properties.fecha_salida.stdout
text2 = text.replace("\n", "")
text3 = str('"'+text2+'"')

inputs.rest_body = """
{
  "ResilientQ": {
    "Categoria": "Seguridad",
    "Subcategoria": "Seguridad -- Antivirus -- Consulta",
    "GrupoAsignado": """+Grupete+""",
    "SolicitadoPara": "falcon",
    "FechaRequerida": """+text3+""",
    "SolicitadoPor": "falcon",
    "Descripcion": [""" +'"'+ repr(description6)+'"' + """ ],
    "Titulo": """+Titulo+""",
    "GrupoRegistradoPor": """+Grupete+""",
    "Fase": "Registro"
  }
}
"""

inputs.rest_headers = """
Content-Type: application/json
X-Frooble: Baz
Authorization: Basic ZmFsY29uOlBhc3N3MHJk
"""
print inputs
#"Descripcion": [""" + Descripcion1 + """]​


    ------------------------------
    Shane Curtin
    Integrations Engineer - IBM Resilient
    ------------------------------

    Original Message