IBM Security QRadar SOAR

 View Only
  • 1.  Artifact Value Validation

    IBM Champion
    Posted Tue March 16, 2021 10:50 AM
    All,

    Do you validate values before attempting to make an artifact with them? I'm wondering if anyone has some good regex patterns for validating artifact values (URLs, DNS, etc.) they'd mind sharing. In an ideal world I'm hoping that Resilient uses regex to validate artifact values somewhere on the backend, and that those regex patterns could be made available to us.

    We've run into the problem of some of our functions return false positives (for example strings that match our URL regex patterns, but aren't actually valid URLs) and trying to create artifacts with those values. When we try to create the artifact an error is thrown, and the execution of the workflow is stopped. This prevents everything else in our workflow from happening.

    I know an answer would be to clean up our regex pattern in the functions mentioned above, but this has proven to be quite the difficult task for me. I'd rather be returning false positives instead of missing false negatives too. I think a regex validation before creating the artifact would be the easiest way to prevent these errors.

    Does anyone have any thoughts?

    ------------------------------
    Liam Mahoney
    ------------------------------


  • 2.  RE: Artifact Value Validation

    Posted Wed March 17, 2021 07:41 AM
    Given that the SOAR platform validates artifact values, I'm not sure it makes sense to do it again elsewhere. Seems like there is always the possibility of an invalid artifact type value.

    When you say the workflow "stops". Is that because there is a script that uses 'incident.addArtifact()'? And that throws an exception inside the Resilient app when evaluated?

    Unfortunately the way to deal with that would be to write a Function that does this. Inside the function you can have a 'try/catch' to do what you want. The function results can return the failure information if so desired.

    In the future, you'll be able to write this as an "in product function" which will make this much less painful.

    Ben

    ------------------------------
    Ben Lurie
    ------------------------------



  • 3.  RE: Artifact Value Validation

    IBM Champion
    Posted Wed March 17, 2021 09:51 AM
    Hi Ben,

    Yeah you have it, `incident.addArtifact(...)` within an in product script will throw an exception and then the rest of the workflow doesn't execute. I was hoping to validate the value before trying `incidnet.addArtifact()` to prevent the exception, but I could see this getting complex to manage.

    I think I'll head down the path of adding artifacts in a function instead. I think it would be easier to catch exceptions rather than try to validate values before adding them in script. 

    Thanks Ben!

    ------------------------------
    Liam Mahoney
    ------------------------------



  • 4.  RE: Artifact Value Validation

    Posted Wed March 17, 2021 09:10 AM
    I have a Resilient script that parses a field that our SIEM populates with data from an alert.  Part of that script looks for potential artifacts within that incident field and adds them as artifacts to the incident.  I use the regular expressions to identify the artifact type and normalize artifacts coming from the SIEM - I also wrap the incident.addArtifact() in a try/except.

    This is for both built-in artifact types as well as custom artifact types I have defined.  Some of the regex patterns were pulled from various sources or created by myself based on our own use cases - so they may need tweaking in your environment (URL/MAC Address artifact types for example).

    REGMAP = {
      'DNS Name': re.compile(r'^(?=^.{4,253}$)(^((?!-)[a-zA-Z0-9-]{1,63}(?<!-)\.)+[a-zA-Z]{2,63}\.?$)$'),
      'COID': re.compile(r'^[0-9]+$'),
      'File Name': re.compile(r'.+'),
      'File Path': re.compile(r'.+'),
      'MAC Address': re.compile(r'^[a-fA-F0-9:-]+$'),
      'IP Address': re.compile(r'^(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$'),
      'Malware MD5 Hash': re.compile(r'^[a-fA-F0-9]{32}$'),
      'Malware SHA-1 Hash': re.compile(r'^[a-fA-F0-9]{40}$'),
      'Malware SHA-256 Hash': re.compile(r'^[a-fA-F0-9]{64}$'),
      'System Name': re.compile(r'^[a-zA-z0-9\.-]{5,}$'),
      'User Account': re.compile(r'^[a-zA-Z0-9\\@\.]+$'),
      'User Agent': re.compile(r'.+'),
      'URL': re.compile(r'^(http|hxxp|ftp)s{0,1}(:|\[:\])//.+'),
    }​


    ------------------------------
    David Vasil
    ------------------------------



  • 5.  RE: Artifact Value Validation

    IBM Champion
    Posted Wed March 17, 2021 09:58 AM
    Hey David, thanks for the reply.

    I appreciate all of the regex patterns, I'm definitely going to bookmark this response!

    Just a heads up - I believe the except block doesn't catch exceptions thrown by `incident.addArtifact()` (see this thread).

    It sounds like we have very similar use cases, although your regex might not be giving you bad data like mine are.

    Thanks again David, have a nice day!

    ------------------------------
    Liam Mahoney
    ------------------------------



  • 6.  RE: Artifact Value Validation

    Posted Wed January 18, 2023 09:10 PM
    Hi David Vasil,

    I have similar use-case where I want to filter artifact because of the function incident.addartifact(). May I know where do you put your REGMAP table for the validation.

    ------------------------------
    Luqman Nur
    Techlab
    ------------------------------