IBM Security QRadar

 View Only
  • 1.  Monitoring log source stopped sending logs for cluster log sources

    Posted Mon July 04, 2022 01:00 PM

    Monitoring log source stopped sending logs for cluster log sources

    A cluster group is a high-availability set of log sources that we monitor. In a high-availability set of log sources, one will be designated as the Primary, and others as Secondary.

    If the Primary log source fails, the system is set up to start receiving logs from the Secondary log source, to ensure there is no outage or time period where we are not logging events. This is often used in critical systems.

    Because HA systems are set up to fail over, we only need to receive and respond to alerts when all the members in the cluster (Primary + Secondary) failed to send logs.

     

    This document will show you how to set up log source stopped sending logs rules to monitor log sources that are members of a cluster.

     

    Steps:

    1. Design a reference map

    Design your reference map first.

    Create a table with 2 columns.

    Under the Cluster Member column you put the list of log sources that are part of any cluster pairs.

    Under the Cluster Group ID column you can come up with a "Cluster Group ID" for each cluster (any name you want, as long as the name represents the cluster members).

    The "Cluster Group ID" needs to be the same for all the log sources in the same cluster. It needs to be different between different clusters.

    Eg:

    Firewall1 and Firewall2 are in a HA pair, so they share the same Cluster Group ID = "Firewall_1_2".

    LoadBalancer1 and LoadBalancer2 are in a HA pair, so they share the same Cluster Group ID = "LoadBalancer_1_2".

     

    Cluster Member

    Cluster Group ID

    Firewall1

    Firewall_1_2

    Firewall2

    Firewall_1_2

    LoadBalancer1

    LoadBalancer_1_2

    LoadBalancer2

    LoadBalancer_1_2

    LoadBalancer3

    LoadBalancer_3_4

    LoadBalancer4

    LoadBalancer_3_4

     

    1. Create the reference map

    Now transform your design into the actual reference map.

    There are many ways to create a reference map on Qradar, in this document I will use the command line way.

    a. First go into the bin folder on the Qradar console:

    cd /opt/qradar/bin

    b. Then, use this command to create a Reference MAP with alphanumeric values, ignore the case.

    ./ReferenceDataUtil.sh create ClusterTable MAP ALNIC

    c. Create a csv file that matches your design.

    On a note pad, create the file like this (must include the "key1, data" text on the first row).

    The "Cluster Member" column from your design should go under "key1".

    The "Cluster Group ID" column from your design should go under "data".

    key1,data

    Firewall1,Firewall_1_2

    Firewall2,Firewall_1_2

    LoadBalancer1,LoadBalancer_1_2

    LoadBalancer2,LoadBalancer_1_2

     

    d. On the console,

    vi /store/clusterTable.csv

    Then, copy and paste the content of the file you created in step c.

    Save it. 

     

    e. Now you can load your reference map with the data.

    ./ReferenceDataUtil.sh load ClusterTable /store/clusterTable.csv

     

    d. Run this command to check the reference map has been loaded properly:

    ./ReferenceDataUtil.sh list ClusterTable displayContents

     

    Now you have a reference map. 😊

     

    1. Create the Custom Event Property with AQL

     

    Create a CEP (let's name it ClusterID) with AQL, using this exact AQL Expression:

    REFERENCEMAP('ClusterTable', LOGSOURCENAME(logsourceid))

    What this does is, it will look up the log source names in the reference map we created earlier, and populate the "Cluster Group ID" value in the CEP called "ClusterID".

     

    1. Create the log source stopped sending rule

     

    a. First create a Log source group called "Cluster Log Source Group".

     

    b. Add all the log sources in the cluster (in your design in step 1) to the "Cluster Log Source Group".

     

    c. Create a BB that references the log source group:

    Apply Cluster Log Sources on events which are detected by the Local system

    and when the event(s) were detected by one or more of Cluster Log Source Group

     

    d. Create the rule like this:

    Apply Cluster Members stopped sending logs for past x mins on events which are detected by the Local system

    and when none of Cluster Log Sources match in x minutes after Cluster Log Sources match with the same Cluster ID

                  

    e. Now you are done. 😊



    ------------------------------
    Qradar Kitty
    ------------------------------



  • 2.  RE: Monitoring log source stopped sending logs for cluster log sources

    IBM Champion
    Posted Sun July 17, 2022 05:44 AM
    Hello @Qradar Kitty,

    Interesting. Thanks for the share.

    Regards,
    @zoldax


    ​​

    ------------------------------
    @zoldax

    https://www.youracclaim.com/users/pascal-weber.029e134d/badges
    ------------------------------



  • 3.  RE: Monitoring log source stopped sending logs for cluster log sources

    Posted Mon July 18, 2022 11:10 AM
    Thank you. This is very helpful

    ------------------------------
    Ray Tam
    ------------------------------



  • 4.  RE: Monitoring log source stopped sending logs for cluster log sources

    Posted Wed February 22, 2023 07:41 AM

    Hi @Qradar Kitty What will be the value we should given in Cluster Member Column ?  Is it LogSource Name or Logsource ID ?




  • 5.  RE: Monitoring log source stopped sending logs for cluster log sources

    Posted Wed February 22, 2023 01:42 PM

    Log source name. :)



    ------------------------------
    Qradar Kitty
    ------------------------------



  • 6.  RE: Monitoring log source stopped sending logs for cluster log sources

    Posted Thu February 23, 2023 11:43 PM
    Edited by Cyber Post Mon February 27, 2023 11:21 PM

    @Qradar Kitty Is this work for you?

    I tried its not working



  • 7.  RE: Monitoring log source stopped sending logs for cluster log sources

    Posted Fri March 24, 2023 09:34 AM

    Hi,

    @Qradar Kitty, I have tested your solution. Sadly, it does not work for me. The problem I have is with last 4 step d action. This rule simply does not work. 

    I have created similar rule for testing still this rule does not work. I hope this information helps for others. All my tests was on version 7.4.3 p5.

    @Qradar Kitty could you share what version you were using.



    ------------------------------
    Andrius Lengvinas
    ------------------------------