Published 8 April 2022
MITRE ATT&CK Wizard Spider and Sandworm Evaluation: IBM Security QRadar EDR demonstrates Best-in-Class capabilities for three years in a row.
MITRE Engenuity has just released the results of the latest round of ATT&CK Evaluations which this year focused on two well-known threat actors: Wizard Spider and Sandworm.
This marks the third time that IBM Security QRadar EDR, formerly known as ReaQta, has successfully completed the ATT&CK Evaluations with top-quality alerts, showing QRadar EDR's capabilities in delivering world-class protection against even the most complex attacks in real-time, without human intervention.
IBM Security QRadar EDR's achievements in this year’s evaluation included the following:
- 100% Detection coverage across the cyber kill chain.
- No configuration changes during the evaluation.
- 100% of detections done in real-time and without delays.
Before diving into the result details, we would like to give a short overview of the MITRE ATT&CK evaluations and the two threat groups Wizard Spider and Sandworm that form the subject of this year's evaluation as well as the testing environment.
About the Evaluations
MITRE introduced the ATT&CK framework in 2015 as a knowledge base of adversary tactics and techniques and it has since then become the de facto standard framework for cyber security professionals looking to make their organizations more cyber resilient. In 2019 MITRE started with the ATT&CK Evaluations to help vendors assess their capabilities against adversary behaviors. Besides the Triton Evaluation in 2021 focussed on ICS vendors, this is the third Evaluation for IT: APT29 (2019), Carbanak + FIn7 (2020), Wizard Spider & Sandworm (2021) and also the third time QRadar EDR has successfully participated.
This year’s MITRE Evaluation covered two infamous threat groups that have wreaked havoc for many organizations worldwide.
The first threat group in the evaluation, Wizard Spider, is a financially motivated Russian speaking criminal group and focuses primarily on extortion through Ransomware attacks.
This group has operated the Trickbot botnet (banking Trojan) since 2016 infecting over 1 million computing devices. They are also the group behind the Conti (ransomware) and started ransomware campaigns in 2018 targeting larger organizations like hospitals and big corporations. According to the FBI, Wizard Spider extorted USD$61 Million for ransomware attacks within just 1 a year.
Sandworm is the second threat group in this year’s evaluation. Sandworm Team is state-sponsored and focuses on destruction of data and system interoperability.
The Russian hacking group has been active since 2009 and operated the NotPetya malware in 2017 in a worldwide attack with the purpose of destroying data. This attack caused many casualties such as Maersk shipping, TNT Express and Merck pharmaceutical. The latter claimed USD$1.3 billion in losses due to interrupted operations. Other infamous attacks attributed to Sandworm include the attacks against Ukrainian electrical companies (2016) and the French presidential campaign (2017). In 2018, the Sandworm Team attacked the Winter Olympic Games in South Korea.
About the Testing Environment
Similar to previous years, the NanoOS, our live hypervisor used to detect high-level malicious behavior could not be used due to restrictions in the testing environment, and this resulted in several missed detections. Even without this core component, QRadar EDR was able to achieve 100% detection coverage across the cyber kill chain.
In previous editions, we participated with a Linux agent for detections, but this year we opted out for Linux evaluation as results would not represent our upcoming Next Gen Linux agent.
100% detection coverage across the cyber kill chain
In both Wizard Spider & Sandworm scenarios, QRadar EDR autonomously reconstructed the attack activity across the cyber kill chain into a few condensed high-fidelity alerts with meaningful and actionable steps to the analyst. QRadar EDR detected the most critical events needed for investigation and analysis as well as the key MITRE ATT&CK evaluation objective, Encryption for Data Impact, keeping customers secure.
Why is this Important?
Customers prefer less alerts that are highly consolidated as compared to multiple and less informative ones. Our approach reduces manual workload and provides a clear picture of unfolding events, with no need to chase attackers over thousands of different security events.
When malicious or suspicious activity is detected, QRadar EDR switches from smart-logging into deep monitoring mode, capturing all events pertaining to the incident presenting the information in a single consolidated alert. This provides a clear picture of unfolding events, with no need to piece together multiple triggers across thousands of different security events, saving the analyst precious time in triaging and incident response.
Experienced analysts understand that not all MITRE ATT&CK techniques have the same importance and we believe those detections missed (e.g. system discovery) to be less relevant, even if they are part of the framework describing an attacker step. Every attack leverages on a series of techniques and some techniques have sub-techniques, but not every technique/sub-technique has the same operational importance from the analyst perspective. Some of them have significant importance for investigation while others can be deduced logically.
By-design, IBM Security QRadar EDR does not observe techniques/sub-techniques of lesser importance, which are evaluated as misses. To increase the visibility of techniques/sub-techniques, most EDR solutions rely on API hooking but this approach is easy to circumvent by attackers and require frequent updates to maintain operational stability. QRadar EDR chooses to rely on other data sources at different OS layers for our detections. QRadar EDR only collects useful information that is essential for the analyst to make a difference in investigation and response outcomes.
Even with the NanoOS disabled, QRadar EDR still provides visibility into every stage of the attack life cycle, across the cyber kill chain.
No Configuration changes during the entire evaluation
The MITRE results evaluate the number of configuration changes. Configuration changes are essentially modifications to the product after the first evaluation. This means that the product was tweaked in order to improve the detection results. This year, configuration changes were placed in 4 main categories, Detection Logic, Data Source, UX and Miscellaneous.
Throughout the evaluation, QRadar EDR did all the detections without any configuration changes. Configuration changes help vendors adjust their detections as the attack progresses. Most vendors had to tweak their product ‘antennas’ multiple times before being able to detect meaningful techniques.
Why is this Important?
In real-life scenarios, configuration changes are usually unrealistic and implies high operational overheads which was not taken into consideration as part of the evaluation, but has a significant impact to organizations using the solution. The more configurations a solution requires, the more an organization has to invest in its operation and maintenance. Attackers do not give defenders a second chance to tweak their detections before moving to the next step.
100% of detections done in real-time without delays
The MITRE results also evaluate the number of delayed detections (shown in red, see chart: Configuration Changes + Delayed Detections). Delayed detections are detections generated with delay and that are not available to the analyst (e.g., require sandbox evaluation). This may be critical (or not) depending on the threat being detected with delay.
Using QRadar EDR's behavioral analysis engines, all detections were entirely in real-time.. Each technique of the attack was tracked as-it-happened, minimizing the risk of losing important events instead of waiting for external components to run their analyses.
Why is this Important?
As attackers innovate, automation allows attackers to move extremely quickly within networks. Operations that used to take minutes or hours now take seconds. The ability to have immediate identification & automated response could result in the difference between a threat stopped at its tracks or an organization compromised and having to perform cleanup and recovery operations.
Closing Remarks
With the completion of MITRE ATT&CK round 4, we are ever more sure of our mission and philosophy to only capture and present necessary information needed by the analyst to help them do their work in the most efficient way possible. This in turn translates to the reduction in operational costs, and lowering the overall Mean Time to Respond (MTTR), hence mitigating the actual cyber risks for organizations.
Acing the MITRE test (i.e., achieving the maximum score), requires monitoring a lot of events, which may often be unnecessary and result in more false positives, causing analysts to become prone to alert fatigue and miss out on the meaningful information in a timely fashion. This increased data collection also leads to increased storage costs and affects threat response efficiencies.
IBM Security QRadar EDR is committed to the MITRE Engenuity ATT&CK Evaluations that helps governments and organizations to combat cyber attacks through proven defense practices. We look forward to participating in the next round.
Delivering Security without complexity
IBM Security QRadar EDR's autonomous endpoint detection and response (EDR) platform aims to solve for the increasing number of businesses falling victim to malicious activities from cyber criminals and nation state actors. While traditional protection methods fight known threats but stand vulnerable to sophisticated attack techniques, QRadar EDR's revolutionary platform stops both known and unknown threats in real-time. Through the use of machine learning, the platform constantly improves on defining normal behavior tailored to each business per endpoint, allowing it to detect and block abnormal malicious behavior. QRadar EDR was named a 2020 Cool Vendor by Gartner in Network and Endpoint Security for this unique approach in tackling cyber threats of all forms.
To learn more about what makes QRadar EDR unique, learn more here and apply for a demonstration.