IBM Security QRadar

Expand all | Collapse all

Rules not working after DSM Update

  • 1.  Rules not working after DSM Update

    Posted 9 days ago
    Hi Community,

    How to deal with rules not firing after a DSM Autoupdate?

    For example, here you see that the nee DSM FortinetFortigate was installed on 22nd October. Before that and I can confirm that by running a historical correlation rule against that period, I can see that my rules work.


    One example ist the event UTM Pass.


    Some properties were not being extracted for some events that I used in my rules and searches previously.

    I though that this was not supposed to happen, right?

    Greetings

    Bruno

    ------------------------------
    BrunoMarX
    ------------------------------


  • 2.  RE: Rules not working after DSM Update

    Posted 7 days ago

    Hi Bruno,

    You say some properties are not being extracted but your screenshot actually indicates that a particular QID is no longer showing up. This is a different kind of problem as the QID is not extracted from the raw event, rather it is looked up based on the Event ID and Event Category determined by the DSM as part of its processing logic. It could be that there was a change where the Event ID or Event Category are now being set differently, or they may now map to a different QID record. Neither should happen unless absolutely necessary, and even then we would usually have a migration option of some kind, so I think something did go wrong here, I will raise it with the team.

    Are there other QIDs you've noticed are no longer appearing, or are there any other properties that are not being populated as they were before?

    Can you provide a sample event payload for one of the events returned by the search in your screenshot? If so, please replace any usernames/IPs/hostnames/etc with placeholders values to protect your organization's privacy.

    Cheers
    Colin



    ------------------------------
    COLIN HAY
    IBM Security
    ------------------------------



  • 3.  RE: Rules not working after DSM Update

    Posted 6 days ago
    Hey Colin,

    Thank you for your response.

    I have a search and a rule looking for some property inside some Fortinet Events. What I noticed in the last months was:

    In the past we had Behavior 1 and then came a DSM update and we noticed Behavior 2. Then we changed our rules and searches.
    Then there was another DSM Update and we noticed Behavior 1 again. Now It changed to Behavior 2 again after the last DSM Update.
    For a long time I though it was my mistake or that Fortinet had changed something, but now I am sure that it was the DSM.

    The properties I am talking about are:
    - Hostname
    - URL


    Here one Example (XXXX -> Placeholder)

    Event before Update(my screenshot): mapped to QID 20257919

    <189>logver=602051142 timestamp=1603324098 tz="UTC+2:00" devname="XXXX" devid="XXXX" vd="XXXX" date=2020-10-22 time=01:48:18 logid="1501054802" type="utm" subtype="dns" eventtype="dns-response" level="notice" eventtime=XXXX tz="+0200" policyid=337 sessionid=1068646757 srcip=XXXX srcport=XXXX srcintf="XXXX" srcintfrole="undefined" dstip=XXXX dstport=53 dstintf="VIF1015" dstintfrole="undefined" proto=17 profile="XXXX" xid=51481 qname="www.google.com" qtype="AAAA" qtypeval=28 qclass="IN" ipaddr="XXXX" msg="Domain is monitored" action="pass" cat=41 catdesc="Search Engines and Portals"


    New Event mapped to QID 20286032

    <189>logver=602051142 timestamp=1606212160 tz="UTC+1:00" devname="FGGR" devid="XXXX" vd="XXXX" date=2020-11-24 time=11:02:40 logid="1501054802" type="utm" subtype="dns" eventtype="dns-response" level="notice" eventtime=1606212161975171866 tz="+0100" policyid=30 sessionid=210165476 srcip=XXXX srcport=51057 srcintf="XXXX" srcintfrole="undefined" dstip=XXXX dstport=53 dstintf="wan1" dstintfrole="undefined" proto=17 profile="XXXX" xid=62170 qname="0.de.pool.ntp.org" qtype="A" qtypeval=1 qclass="IN" ipaddr="XXXX" msg="Domain is monitored" action="pass" cat=52 catdesc="Information Technology"

    Thank you

    Regards,
    Bruno




    ------------------------------
    BrunoMarX
    ------------------------------



  • 4.  RE: Rules not working after DSM Update

    Posted 4 days ago
    Hi Bruno,

    I can confirm that some logic changes were made around how the Event ID and Event Category values are set for these events, resulting in the events now being mapped to a different QID (20286032). I believe the goal was to be more granular/precise (the old approach had "utm" as Event ID and the default value of "Fortinet" for Event Category; the new approach has "utm pass" as Event ID and "DNS" as Event Category, where the category is now being set dynamically based on the subtype field). This is intended as an improvement but the side effects of doing so may not have been fully considered; we are reviewing within the team tomorrow with the goal being to have a way of better communicating these changes to end users and hopefully being able to offer a method of updating related content like rules, searches and custom property expressions.

    For now updating your rules which keyed off the old QID specifically to use the new QID instead should get them working again.

    The properties not being populated is a separate but possibly related issue. Both URL and Hostname are technically custom properties (even though we do include them out-of-the-box), meaning the DSM is not responsible for populating them. Custom property expressions can be linked directly to a specific QID, such that they only execute for events with that QID. I would suggest checking the Custom Event Properties admin UI to see if your Fortigate expressions for these properties have a direct QID association. If they do, you'll need to update those expressions to link to the new QID.

    This is the only explanation I can think of for why a DSM change could impact those properties (URL and Hostname).

    Cheers
    Colin

    ------------------------------
    COLIN HAY
    IBM Security
    ------------------------------



  • 5.  RE: Rules not working after DSM Update

    Posted 4 days ago
    Hi Colin,

    Thank you for your reply.

    As I wrote above. I had a search(and a rule) looking for the event and the property. I don't remember when, but it stopped working. I then changed it to display to correct event. Then the search stopped working after a while (means displaing no events). I observed that the event had changed again and modified my search. Then I realized that it might have something to do with the DSM and came here to write this post. This behavior was observed in the last months.

    Greetings,

    Bruno

    ------------------------------
    BrunoMarX
    ------------------------------



  • 6.  RE: Rules not working after DSM Update

    Posted 4 days ago
    Ok so are your properties (URL and hostname) working again? Did you need to change which QID they were associated with as I suggested? The QID change I understand, but I just want to be sure the property issue is also understood and resolved.

    Cheers
    Colin

    ------------------------------
    COLIN HAY
    IBM Security
    ------------------------------



  • 7.  RE: Rules not working after DSM Update

    Posted 3 days ago
    Hi Colin,

    Thank you!

    The QID had no property extracting the DNS name.
    I had to change the DSM in the DSM Editor for the property URL. I added another selectivity based on High Level Category and Low level category: Firewall Permit. it then worked.




    ------------------------------
    BrunoMarX
    ------------------------------



  • 8.  RE: Rules not working after DSM Update

    Posted 7 days ago

    Bruno,

    In the last Fortigate DSM update some QIDs/Event Mappings changed.

    We had the same problem, so maybe you can find the new event searching by event name.



    ------------------------------
    Rodrigo Teixeira
    ------------------------------



  • 9.  RE: Rules not working after DSM Update

    Posted 6 days ago
    Obrigado Rodrigo!

    I think this have been changing over the last months.

    As I stated above, I noticed Behavior 1 and 2 over the last months after these DSM Updates. At first I thought that it was something regardin Fortinet Logs but in the end it turned out that it was the DSM.

    ------------------------------
    BrunoMarX
    ------------------------------