IBM QRadar

IBM QRadar

Join this online user group to communicate across Security product users and IBM experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only

  • 1.  Qradar : Events/flows were dropped by the event pipeline.

    Posted Fri April 24, 2020 09:23 AM
    Hi everyone,

    How to tune the system to reduce the volume of events and flows that enter the event pipeline ?
    Below the system notification :


    Apr 20 09:01:31 127.0.0.1 [ba0e7e13-ac6c-4fc0-98e3-69e458bb92d8/SequentialEventDispatcher]
    com.q1labs.sem.monitors.SourceMonitor: [WARN] [NOT:0060005100][10.12.158.22/- -] [-/- -]A total
    of 60688248 dropped raw event(s) have been detected. 65596 raw event(s) have been dropped in
    the last 60 seconds. License restrictions have been applied 60 times in the last 60 seconds.
    The average event rate in the last 60 seconds was 2265.10 eps (with a peak of 3061.60 eps),
    and within that time has exceeded the threshold of 2510.00 eps 6 times.

    Thank you in advance.

    Regards.


    Hichem AZAIEZ

    ------------------------------
    hichem azaiez
    ------------------------------


  • 2.  RE: Qradar : Events/flows were dropped by the event pipeline.

    Posted Mon April 27, 2020 12:52 AM
    ​Hi Hichem,

    First of all check the top log source in the log activity tab (run a search), This is mostly caused if EPS license limit is crossed. Also chck if any recent changes happened to the QROC, If any new log source has been added? or any new types of diagnostic logs have been enabled.


    Regards
    Asif Siddiqui

    ------------------------------
    Asif Siddiqui Senior Security Analyst
    ------------------------------

    Original Message


  • 3.  RE: Qradar : Events/flows were dropped by the event pipeline.

    Posted Mon April 27, 2020 01:11 AM
    Hi Hichem,

    There are several options, depending on the nature of the events you're receiving.

    1. Some of our log source protocols (Windows Event Log over MSRPC, WinCollect, for example) allow you to configure what event types you want to retrieve from the remote system and in some cases to apply specific filters. You can use these sort of options to retrieve a subset of all events that you consider most important.
    2. Many of our pull-based protocols (Log File, JDBC, various REST API-based protocols) have an Event Rate throttle config parameter which allows you to control how fast they inject their retrieved events into the QRadar event pipeline. Lowering these values will slow down the rate of event ingestion to stay under your license threshold
    3. For log sources which use push-based protocols (where the event source is sending events to QRadar, via syslog, SNMP, etc) we can't control on the QRadar side what is being sent to us, so ideally any tuning or filtering would be done on the sending side to avoid sending QRadar any unneeded events. If this cannot be done, it is possible to use routing rules (configurable from the QRadar Admin tab) to drop unneeded events on the QRadar side
    4. If you have a multi-host deployment (event collectors and/or event processors in addition to your console), you may be able to redistribute your log sources across your available ECs/EPs such that the event rate is more evenly spread across your available systems.

    Cheers
    Colin

    ------------------------------
    COLIN HAY
    IBM Security
    ------------------------------

    Original Message


  • 4.  RE: Qradar : Events/flows were dropped by the event pipeline.

    Posted Mon April 27, 2020 03:21 AM

    Hi,

    Thank you all for your answers;     I'm going to analyze to see the origins of saturation.


    Thank you for all of your thoughts.

     

     

    Hichem AZAIEZ

    =========================================================

    Ce message et toutes les pieces jointes (ci-apres le "message")
    sont confidentiels et susceptibles de contenir des informations
    couvertes par le secret professionnel. Ce message est etabli
    a l'intention exclusive de ses destinataires. Toute utilisation
    ou diffusion non autorisee interdite.
    Tout message electronique est susceptible d'alteration. La SOCIETE GENERALE
    et ses filiales declinent toute responsabilite au titre de ce message
    s'il a ete altere, deforme falsifie.

    =========================================================

    This message and any attachments (the "message") are confidential,
    intended solely for the addresses, and may contain legally privileged
    information. Any unauthorized use or dissemination is prohibited.
    E-mails are susceptible to alteration. Neither SOCIETE GENERALE nor any
    of its subsidiaries or affiliates shall be liable for the message
    if altered, changed or falsified.

    =========================================================




    Original Message


Related Content