Security

Introduction to app development in QRadar 

QRadar can correlate data from various log sources that send event data, flow sources that transmit packets of flow data, threat intelligence from different vendors, asset information, network topology, etc., enabling us to build varied security use cases. QRadar supports an application framework which enables developers build containerised applications which can be published and installed on different QRadar systems. In this blog we will provide guidelines on developing QRadar apps which help with threat intelligence integration of different products with QRadar.

A few QRadar applications have already been developed by Recorded Future and Mandiant to help ingest respective data into QRadar. But this blog is for those who would like to build apps for their own vendors.

Contents of custom threat intel apps 

When we talk about Threat Intel, there are different types of threat intel that you would like to integrate. It could be:

1. Indicators of Compromise (IOC) - Information on IP addresses, domain names, file hashes, URLs etc.
2. Vulnerability information  -  Information on CVE, risk score, platforms affected
3. Geolocation information -  Information on location: could be physical and registered in case of IP address

Understanding how to use Reference Data 

QRadar has a feature called Reference Data. Reference Data can further be categorized into different types of data collection viz Reference Set, Reference Map, Reference Map of Sets, Reference Map of Maps and Reference Table. Details on these categories is clearly explained here -https://www.ibm.com/docs/en/qradar-on-cloud?topic=qradar-types-reference-data-collections

The threat intelligence data should ideally be imported into the Reference Data in QRadar at regular polling interval. This Reference Data can then be used by QRadar in correlation. Correlation helps implement security use cases defined using QRadar Rules. So the Reference Data can be directly used in QRadar Rules.

We have seen that QRadar app developers have a recurring concern on which Reference Data type to use their threat data. Here are a few pointers for the same.

The decision on which type of Reference Data to use completely depends on the kind of threat data you have.
 

1. If you have multiple lists of IOCs like malicious hashes, IP addresses, domain names - you can have multiple Reference Sets for each type of threat intel info. You will have one Reference set for IP address while other Reference Set for hashes, while a third could be for domain names.This helps keep data segregated and enables more refined security use cases.
    

1. If you have enrichment data, like more information on an asset or IP address. Such type of data could be stored in Reference Table. But beware that Reference Table should be designed in such a way that the two specific keys leads you to specific value of data. For example: If we consider an IP address and an application as primary and secondary key it should lead us to the Port used by the application.
   
    

The other recurring query that developers have is, how to import data into Reference Sets or Reference Tables. Importing data is not a one time activity because threat intel data is updated regularly and the threat intel data needs to be polled regularly. There are different ways to import data:

1. Manually adding data to Reference Data - You may create Reference Set and Reference Tables and enter the values manually. - https://www.ibm.com/docs/en/qradar-on-cloud?topic=overview-importing-iocs-reference-set
2. Manually adding data to Reference Data using Files - CSV files may be made available which may be added from the QRadar GUI. But still this method is not feasible.
3. Using QRadar REST APIs (RECOMMENDED way) - The current QRadar REST API available for use is defined here - https://ibmsecuritydocs.github.io/qradar_api_20.0/ . In this version we can see that there are 2 endpoints which are exposed that can be used are:

1. Reference_Data endpoints - You may notice that a few available options are in a DEPRECATED state. It is recommended not to use them. The DEPRECATED endpoints would be removed in the next update to QRadar API endpoints. You may see that METHOD /reference_data/sets is DEPRECATED. So instead of using this end point, please check option in reference_data_collections endpoint as discussed in next point. You may see that the methods like GET, POST, PATCH, DELETE are provided for these endpoints. You will need to use POST method with exposed endpoints to fetch the data programmatically.

          

2. Reference_Data_Collections endpoints - These are the new endpoints which are exposed. Check the screenshots for clarity.

QRadar SDK - Building Apps

You may write scripts that poll data from vendor sites into QRadar using the QRadar API endpoints discussed above. QRadar provides an App Software Development Kit (SDK) which can be downloaded from the link - " https://ibm.biz/QRadarSDK". The app also contains documentation regarding how to develop apps, sample apps and best practices. You may use this information to build your own Threat Intel app.

 
Additional features that can be incorporated

A custom threat intel app that you develop may also contain other QRadar features, such as custom rules, searches, reports, dashboards, and a right-click utility to enrich the available data. Let me cover these topics in brief:

 

Custom Rules - Rules are the way to implement Security use cases in QRadar. While designing the rules, you may use reference data as one of the conditions in the rule. This helps to fine tune rules. So along with the logic of importing threat intel into custom designed Reference Data, you can also include custom rules in the QRadar app that you would be publishing.

 

Custom Searches - You may be looking for some specific data, specially when you perform Threat Hunting in QRadar. Using Reference Data in Threat Hunting can lead to better results.

 

Right-Click utility - This is a way to enrich the data that we see in Log Activity. For the appropriate IOC or hash, you may set a right-click functionality which would display enrichment data for that particular element.

This is very helpful when it comes to visibility and context in data.

 

Custom Dashboards - Executives would like to see visual representation of data from different sources. This may need the augmentation of threat intel data too. Custom Dashboards if required, could be a part of your published app

 

Custom Reports -  Custom Reports are essentially scheduled searches. As searches can utilize the newly infused threat data, custom reports can also be built accordingly. Reports showing data location could be very helpful when you look for compliance reports like GDPR compliance etc.


Best Practices and Learning over time

We have also incorporated in this blog a few learning (on QRadar app development for threat intel) which we will keep updating :

1. Always enable the 'time_to_live' option available for Reference Sets and Reference Tables. Ensure that you have entered a practical value for 'time_to_live' which means that the threat intel data that you wish to enter is inline with the rate at which the threat intel data may be outdated.
   
   
2. Reference Set and Reference Tables should not grow out of proportion. This may affect performance of QRadar. It would affect the performance if the number of elements is more than 100K and the reference data is being used by multiple rules and searches. Performance completely depends on how the QRadar environment is designed and deployed. But above are best practice to follow.
   
   
3. Keep a watch on QRadar defects which may be affecting Reference Data. This may help building better apps for Threat intel.

#HowTo
#Training