IBM Security QRadar

 View Only

  • 1.  Upgrade to 7.5.0 increased reported FPS numbers by 75%

    Posted Mon November 14, 2022 02:38 AM


    After upgrading our deployment from 7.4.3 to 7.5.0 Update 3 with IF02 the reported FPS numbers have suddenly increased by 75%, compared to the numbers on the last 3 months running with 7.4.3

    The Flow Collector Configuration, as shown in Component Management, shows no changes. The number of incoming flow packets over time ist also unchanged.

    Has anybody noticed similar behaviour or even found a resolution for this.

    Thomas



    ------------------------------
    SIEM-2020
    ------------------------------


  • 2.  RE: Upgrade to 7.5.0 increased reported FPS numbers by 75%

    Posted Tue November 15, 2022 08:41 AM
    I was told there is a pre apar bug on this issue. We receive message that flow were dropped by Qlow processing once we upgrade to 7.5.0

    ------------------------------
    Bruce Hutchinson
    ------------------------------

    Original Message


  • 3.  RE: Upgrade to 7.5.0 increased reported FPS numbers by 75%

    Posted Mon November 21, 2022 10:02 AM
    Throughout the 7.5.0 release stream there have been a number of performance improvements made to both QNI and the core flows pipeline. These are listed under the what's new section within the documentation:
    - https://www.ibm.com/docs/en/qsip/7.5?topic=750-qradar-network-insights
    - https://www.ibm.com/docs/en/qsip/7.5?topic=qradar-flow-improvements

    It may have been possible that prior to 7.5.0, your deployment was dropping flows at some point in the flows pipeline and the performance improvements delivered in 7.5.0 has reduced that dropped flow rate.

    ------------------------------
    JARED HAYWARD
    ------------------------------

    Original Message


  • 4.  RE: Upgrade to 7.5.0 increased reported FPS numbers by 75%

    Posted Tue November 22, 2022 05:17 AM

    I believe we have been hunting down each and every indicator for dropped events or flows.  From Overflow Counters and hardware rate limiters, spillover files , missed flow counters to performance problems which resulted in events "routed to storage" without CRE processing.

    After several years we expected that we found and track all indicators for situations where your events or flows disappear.

    Can you recommend certain QRadar logs where to look for flows dropped in 7.4 , that might now be processed correctly in 7.5 ?

    Or would you expect that flows where dropped silently ?

    I am still not sure what is more scary -  missing flows undetected in 7.4, or the need to almost double the license with 7.5.

    Thomas



    ------------------------------
    SIEM-2020
    ------------------------------

    Original Message


  • 5.  RE: Upgrade to 7.5.0 increased reported FPS numbers by 75%

    Posted Tue November 29, 2022 05:54 PM
    Edited by JARED HAYWARD Tue November 29, 2022 07:40 PM
    From a QFlow perspective pre 7.5.0, for Netflow/IPFIX flows you could look in /var/log/qradar.log for instances of missing sequence number messages.
    A missing sequence number could mean a number of things:
    1. As the flows were sent via UDP, the flows may have been processed out of order.
    2. As the flows were sent via UDP, the flows may have been dropped on the UDP link.
    3. QFlow couldn't keep up with the incoming packet rate and the UDP buffer was overrun causing packets to be dropped.

    The logging was improved in 7.5.0 to give better visibility to instances of dropped flows within QFlow with additional notifications in the UI as well. Likewise the processing pipeline for QFlow was greatly enhanced in 7.5.0 in order to reduce the 3rd scenario above as much as possible.

    ------------------------------
    JARED HAYWARD
    ------------------------------

    Original Message