IBM Security QRadar

We are using rules which check certain properties with regex for the presence of a pattern. 

As we make frequent changes to these regular expressions we are looking for a way to consume the regex, used by these rules from an external location.

So far we have not found a useable API Endpoint for injection of content into rules. 

Do you have any hints how to push/pull the regex from an external source into QRadar Ruleengine ?  

Regards
Thomas

Hi Thomas,

At this point of time, I don't think QRadar has anything like this.  It doesn't seem feasible also as parsing CEP measure in milliseconds and the moment you think about external source, network comes into picture which can introduce latency and break the pipeline and rule engine.

We are using rules which check certain properties with regex for the presence of a pattern. 

As we make frequent changes to these regular expressions we are looking for a way to consume the regex, used by these rules from an external location.

So far we have not found a useable API Endpoint for injection of content into rules. 

Do you have any hints how to push/pull the regex from an external source into QRadar Ruleengine ?  

Regards
Thomas

Hi Prabir,

I was looking for a way to load the regex into the rule from remote,  not to process the regex remotely.

Regards
Thomas

Hi Thomas,

At this point of time, I don't think QRadar has anything like this.  It doesn't seem feasible also as parsing CEP measure in milliseconds and the moment you think about external source, network comes into picture which can introduce latency and break the pipeline and rule engine.

We are using rules which check certain properties with regex for the presence of a pattern. 

As we make frequent changes to these regular expressions we are looking for a way to consume the regex, used by these rules from an external location.

So far we have not found a useable API Endpoint for injection of content into rules. 

Do you have any hints how to push/pull the regex from an external source into QRadar Ruleengine ?  

Regards
Thomas

Hi Thomas,

Question for you:

1. why would you change a regex frequently?  I don't see a reason to change a regex (NOT frequently) until
   1. the payload changes
   2. or it's a greedy regex to which you want to do some optimization
2. At this point, if at all you need to change regex, you need to change that in the Custom Event Properties window of admin tab.
3. There is no such way to load the regex into a rule from remote.

But, I would like to understand your thought process behind this. Do you have any use case / example (with proper data) that you think you can share?

Hi Prabir,

I was looking for a way to load the regex into the rule from remote,  not to process the regex remotely.

Regards
Thomas

Hi Thomas,

At this point of time, I don't think QRadar has anything like this.  It doesn't seem feasible also as parsing CEP measure in milliseconds and the moment you think about external source, network comes into picture which can introduce latency and break the pipeline and rule engine.

We are using rules which check certain properties with regex for the presence of a pattern. 

As we make frequent changes to these regular expressions we are looking for a way to consume the regex, used by these rules from an external location.

So far we have not found a useable API Endpoint for injection of content into rules. 

Do you have any hints how to push/pull the regex from an external source into QRadar Ruleengine ?  

Regards
Thomas

We are using rules which check certain properties with regex for the presence of a pattern. 

As we make frequent changes to these regular expressions we are looking for a way to consume the regex, used by these rules from an external location.

So far we have not found a useable API Endpoint for injection of content into rules. 

Do you have any hints how to push/pull the regex from an external source into QRadar Ruleengine ?  

Regards
Thomas

You can achieve this using a Custom AQL function.

First, you need to create your function - which you can do by following (for example) Jose Bravo's example .. https://www.youtube.com/watch?v=6z8zjXw-xE4

When doing so, use this XML template as an example:

===

<content>
    <custom_function>
        <namespace>AQL_FILTER</namespace>
        <id>1</id>
        <name>Dynamic_URL_check</name>
        <return_type>boolean</return_type>
        <parameter_types>string</parameter_types>
        <description>URL checker</description>
        <execute_function_name>checkURL</execute_function_name>
        <script_engine>javascript</script_engine>
        <varargs>false</varargs>
        <script>
          function checkURL(in_url) {
            url_match = false;
            if (/.*dodgy.*/.test(in_url)) {
               url_match = true;
            }
            return url_match;
          }
        </script>
        <username>admin</username>
    </custom_function>
</content>

===

Once that has uploaded sucessfully, you can use the new AQL filter in a rule like this:

===

Rule Description
      Apply ZXZ_CUSTOM_AQL_TEST on events which are detected by the Local system
and when the event(s) were detected by one or more of testleef
and when the event matches AQL_FILTER::Dynamic_URL_check(test_url) AQL filter query
 
Rule Actions
      

    Set Severity to 10
    Set Credibility to 10
    Set Relevance to 10

This Rule will be: Enabled 

===

To modify the test, all you need to do is change the JavaScript in the Custom AQL filter, upload it and force the rules to reload.

Now, the way the video shows to do it is using the Content Management Tool (CMT) - but you can also manage Custom AQL Functions with the REST-API content extensions endpoints. (under "/config/extension_management"). 

You can then force a rule reload by toggling any rule using the REST-API endpoint "/analytics/ade_rules/{id}"

pfh

We are using rules which check certain properties with regex for the presence of a pattern. 

As we make frequent changes to these regular expressions we are looking for a way to consume the regex, used by these rules from an external location.

So far we have not found a useable API Endpoint for injection of content into rules. 

Do you have any hints how to push/pull the regex from an external source into QRadar Ruleengine ?  

Regards
Thomas

We are using rules which check certain properties with regex for the presence of a pattern. 

As we make frequent changes to these regular expressions we are looking for a way to consume the regex, used by these rules from an external location.

So far we have not found a useable API Endpoint for injection of content into rules. 

Do you have any hints how to push/pull the regex from an external source into QRadar Ruleengine ?  

Regards
Thomas

We received a hint which allowed us to implement this requirement in an even easy way and I want to share this option. 

The approach is, to store the regex in a refmap, and use an AQL to load the value from the refmap and use it for pattern matching.

The steps in detail are:
- Use Reference Data Management (or API) to create a new refmap of Type ALN . 
- Use Reference Data Management (or API) to add a new record with a unique key (here 'regexid') and the regex (here '.*testdomain.com.*' in the value field 
- write an AQL query which performs the match of the regex against the desired field.

For example 

We are using rules which check certain properties with regex for the presence of a pattern. 

As we make frequent changes to these regular expressions we are looking for a way to consume the regex, used by these rules from an external location.

So far we have not found a useable API Endpoint for injection of content into rules. 

Do you have any hints how to push/pull the regex from an external source into QRadar Ruleengine ?  

Regards
Thomas