IBM QRadar

IBM QRadar

Join this online user group to communicate across Security product users and IBM experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only

  • 1.  QDI - 1+Billion Dropped Events

    Posted Tue October 15, 2019 10:35 AM

    Hello All,

    I'm new to QRadar SIEM Tool. In my QRadar Deployment Intelligence I'm getting a message like "A Total of 1+Billion dropped raw events are detected. 4000+raw events are dropped in last 60 seconds.License Restrictions have been applied 59 times in last 60 seconds.

    What does this messages means whether 1+ billion of events is a total number of dropped events since QRadar was installed, actual drops from QTS were 4000 events+ ?

    Need expertise here.

    I've gone through the QRadar Troubleshooting System Notifications guide and they insisting me to reduce the volume of events. But before do so I need to know what this error means.

    Much Appreciated!! Thanks
    Pranav



    ------------------------------
    Pranav Sankar
    ------------------------------


  • 2.  RE: QDI - 1+Billion Dropped Events

    Posted Wed October 16, 2019 03:35 AM
    I'd say the info/suggestion you found in the guide is correct. The occurrence suggests there are more events per second than the license allows it, and in your case it is basically all the time ("59 times in 60 second"). When you step over the license, the excess is moved to the buffer/queue and processed later on FIFO base when the load drops below the license limit - up to the license limit. However, bear in mind that the temporary queue size is limited (5GB).
    See: QRadar Event & Flow burst handling.

    ------------------------------
    Dusan VIDOVIC
    ------------------------------

    Original Message


  • 3.  RE: QDI - 1+Billion Dropped Events

    Posted Thu October 17, 2019 03:05 AM
    That's not actually quite the whole story. IBM talk about "the queue", but there are actually multiple different queues - one for syslog, one for Wincollect, etc. They are also not all the same size (the syslog queue is large because it gets a lot of traffic, the JDBC queue is less used and is consequently a lot smaller).

    ------------------------------
    James McLaren Security Consultant and SIEM Specialist
    Logicalis
    ------------------------------

    Original Message


  • 4.  RE: QDI - 1+Billion Dropped Events

    Posted Wed October 16, 2019 10:41 AM
    This has been a confusing topic for me (and I think many others as well).

    My understanding is that events get dropped when the ecs or maybe ecs-ingress process is overwhelmed and the queue of events fills up (shows as queue 100% full) in the logged event).

    My current understanding is that events that cannot be processed due to licensing restrictions are held in the queue and if the license is not maxed out in the next second, they are processed then.

    When the queue is showing 100% full I have been told two things, one that events then are truly dropped, and secondly that the events can still be processed if your licensing "give back"  is available. This can be seen in the QDI app. When I see the message you saw, I check in QDI on the graph for Event License give back where dropped events are tracked. I have never seen this graph show any dropped events. I do not know how to search for genuinely dropped events using Ariel.

    Maybe an IBM person could chime in with a really, really clear explanation of this for us, and include a search to check for genuinely dropped events and for a way to find out why they are dropped.  Exxplain how we can figure out if it is licensing restrictions causing the event, or a physical overflow of the hardware or just overwhelming the software.

    That would be great. 


    ------------------------------
    _____________________
    Daniel Sichel
    ------------------------------

    Original Message


  • 5.  RE: QDI - 1+Billion Dropped Events

    Posted Thu October 17, 2019 05:29 AM
    The give-back mentioned is there to cover internal/built-in events (System notification, Health, CRE ...). I would expect them generally to be up to 200 EPS. Though the licensing and license give-back are evaluated and applied every second, on few sites that had only 200-300 EPS licensed I did encounter notifications about exceeding the license somehow to occur before the give-back kicks-in (and in some cases anomalous over-generation of internal events). So, if the give-back is enough to get you under the license limit, I would not expect the events to be dropped. As I see it, the event/flow drop could occur when the surge (constant overload) is such and for such a long time that the queue and disk buffers are full (which could cause the pipeline to block the input until it processes the consumed data waiting and resume normal operation after this concludes).  Thus, if there's constant huge license overstepping it could be possible to have temporary event drops.
    You might consider a search in Log activity for particular event names such as: Event pipeline dropped events , Event pipeline dropped connections , Event(s) were routed directly to storage .

    ------------------------------
    Dusan VIDOVIC
    ------------------------------

    Original Message