IBM QRadar

IBM® QRadar® SIEM collect events from security products by using a plug-in file which is called a Device Support Module (DSM). IBM already has many DSMs available out of the Box.

IBM® QRadar® SIEM receive logs from systems and devices by using the standard protocol i.e. Syslog protocol. Supported DSMs can use any other protocols as well, as mentioned in the Supported DSM table. One can also try to configure third-party applications to send logs to IBM® QRadar® SIEM using the Syslog protocol. 

IBM® QRadar® SIEM is responsible for writing the regex, doing the parsing and Mapping of QIDs for supported log sources. Refer Supported DSM document to understand more about supported DSMs.

For unsupported DSMs, to make the sense out of events;  we need to parse, Normalised and map events to  Event Name. To find any anomaly during monitoring rule conditions can be applied on events to trigger offense.
In case of Supported DSMs, events will be parsed properly. However, if they are not parsed engage IBM Support.

The document describes detailed procedure about how to deal with unsupported events from the device Manufacturer and/or model. This integration is step by step guide to implement same. 

Configure and use a Custom DSM (UDSM):

Write a log source extension to parse events from unsupported device.:
For more information, see Log source extensions and the DSM Editor. Use content extensions for sending events to IBM® QRadar® SIEM that are provided by third-party vendors from the IBM Security App Exchange (https://exchange.xforce.ibmcloud.com/hub/). These third-party DSM integrations are supported by the resp. vendor, NOT by IBM.

NOTE: The third-party DSM integration or parsing issues observed are out of the IBM Support scope.
Please find below video on how to write DSM in DSM Editor. https://www.youtube.com/watch?v=KF40bba_kp0

Figure 1 – Log Source Integration with IBM® QRadar® SIEM

Choosing the Right Protocol:

First thing which important is to receive the Events on the IBM® QRadar® SIEM . We can use any of the below protocols to collect the events. Important ones are highlighted in the Table.

Next step is to parse the events. In case of trouble in configuring any of above protocol, reach out to IBM Software Support. SMB Tail or Log File protocol  can be used, in situations where we do not have direct protocol to receive the events; either by dumping application logs on any Linux or Windows server. IBM® QRadar® SIEM can pull these events from there.

Configure and use a Custom DSM (UDSM):

Let’s explore the option of configuring the Custom DSM manually.

Figure 2 - Flow Chart

Let’s consider an example to understand how to parse Unsupported Events:

The following events do not have any existing supported DSM from IBM® QRadar® SIEM.

Nov 25 12:20 unsupported eventid="New Unsupported Event" eventcategory="category1" username=user1 srcip=192.168.1.1 srcport=6666
Nov 25 12:22 unsupported eventid="New Unsupported Event" eventcategory="category2" username=user2 srcip=192.168.1.2 srcport=6667

The received events are categorized as SIM Generic which are unparsed.

Figure 3 - Unparsed and Unmapped Events in Log Activity Tab

Using DSM Editor:

Using DSM editor we can create create custom DSM. Open the events in DSM Editor, and follow below steps.

1. Choose the Log Source Type. As we do not have supported DSM, we need to Create one.

Figure 4 - Select Create New

2. Choose the Log Source Name as per your requirement. eg :  “Internal New Log Source” ( for demo purpose)

Figure 5 - Name the Log Source

3. It will show up in the available Log Source Type List.

Figure 6 - Available Log Source Types

4.

4. Once you select the Log Source Type “Internal New Log Source”, DSM Editor will show events information as : 

a.     Events here are showing Parsing Failed.

b.     All the properties are not getting parsed.

c.     Event Category is showing unknown.

d.     Event Name is showing unknown

e.     Event Name is showing unknown

Figure 7 - Parsing Failed Events in DSM Editor

5. Let us  create the regex to extract the Event Category

a.     Search Event Category Property

b.     Override system behaviour

c.     Choose Expression Type ( Available options are : Regex, JSON, CEF, LEEF, GENERIC LIST, NAME VALUE PAIR, XML. For more details check qradar-property-configuration-in-dsm-editor )

d.     Select Regex here and we can directly put the expression to extract the property we need or type in the property which you would like to extract and hit “Suggest Regex”

e.     The Regex and Dark Yellow highlighted text is the extracted value by Regex.

Figure 8 - Regex for Event Category

6. Regex extracts the correct value for Event Category.

Figure 9 - Event Category is populating now

7. Next is to create the regex to extract the Event ID. Follow the same steps used in the Step 6 to extract Event ID using Regex.

Figure 10 - Regex for Event ID

8.  Post this; Regex extracts the correct value and Event Category starts populating. Once you have Event ID and Event Category Extracted you can see the Parsing Status has been changed to “Parsed and NOT Mapped”

Figure 11 - Event ID is populating now and Parsing Status changed to “Parsed and NOT Mapped”

9.     Follow the same method used in steps 5 and 6 to generate Regex for Source IP. Or we can write  regex to extract the Source IP. You can see Source IP is populating now for all the events

Figure 12 - Source IP is populating now

10.  Same method is used in steps 5 and 6 to generate Regex for Source Port. Or you can write  regex to extract the Source Port. You can see Source Port is populating now for all the events.

Figure 13 - Source Port is populating now

11.  Same method used in steps 5 and 6 to generate Regex for Username. Or write regex to extract the Username. You can see Username is populating now for all the events

Figure 14 - Username is populating now

12.  At this we have Event Category and Event ID extracted. Also, all other properties are extracted. The  next step is Event Mappings. We can map the combination of Event Category and Event Name to the QID / Event Name. Then default  two mappings by default are unknown/ Stored and unknown / unknown.

13.  To add new mapping Click on + sign to add the new mapping.

Figure 15 - Available Default Event Mapping

14.  We can see available unknown event mappings, and select one by one to start mapping with available QIDs or by creating new QIDs. Here we select First Pair :
Event Category – New Unsupported Event
Event ID – category1

Figure 16 - Unknown Event Mapping available for mapping Select First Pair

15.  Choose any of the available QID or click on create new QID Record.

Figure 17 - QID Records Search or Create New QID Record

16.  Create new QID.

a.     Name :  Event - Category1

b.     Description : This is new Unsupported Event Of Category 1.

c.     Log Source Type : Internal New Log Source

d.     High Level Category : User Defined

e.     Low Level Category : Custom Policy 1

f.     Severity : 5

Figure 18 - Add New Custom QID Record for Event – Category1

17.  New QID is created with QID Number 1004000002. Click Ok and Create to get the QID mapped.

Figure 19 - New QID record created and available to use for mapping

18.  Now select the second Pair:

Event Category – New Unsupported Event
Event ID – category2

Figure 20 - Unknown Event Mapping available for mapping select second pair

19.   Create New QID Record.

a.     Name :  Event - Category1

b.     Description : This is new Unsupported Event Of Category 1.

c.     Log Source Type : Internal New Log Source

d.     High Level Category : User Defined

e.     Low Level Category : Custom Policy 1

f.      Severity : 5

Figure 21 - Add New Custom QID Record for Event – Category2

20.  We can see now new QID is created with QID Number 1004000003. Click Ok and Create to get the QID mapped.

Figure 22 - New QID record created and available to use for mapping

21.  Once All Events are parsed and mapped, Parsing Status has been changed to “Parsed and Mapped”. Event Name  is seen as per the QID mapping we have done in earlier steps.

Figure 23 - Parsing Status is changed to "Parsed and Mapped"

22.  At this point our DSM is capable of parsing the unsupported logs which we are going to received. We can either create Log Source manually from Log source management tab or we can use below method to enable the auto detection of the Custom Log Source As well.

 
Once we enable this if Auto Detection engine will detect the log source automatically based on the Advanced Options and create the log source as per the naming format
Log Source Name : $$DEVICE_TYPE$$ @ $$SOURCE_ADDRESS$$
Log Source Description : $$DEVICE_TYPE$$ device

Note below Advance option which you can modify as well:
Minimum Successful Events for Autodetection : Minimum number of events from an unknown source that must be successfully parsed for auto detection to occur.

Minimum Success Rate for Autodetection :  Minimum parsing success rate (percentage) for events from an unknown source for auto detection to occur.

Attempted Parse Limit : Maximum number of events from an unknown source to attempt before abandoning auto detection.

Consecutive Failed Parse Limit : Number of consecutive events from an unknown source to abandon auto detection.

Figure 24 - Auto Detection Enabled

Figure 25 - Autodetection Advanced options

23.  Save the DSM and Go to Log Activity tab to confirm DSM is parsing and mapping the events properly. We can confirm by  Event Name and Log source is auto detected. Source IP, Source Port, Username fields.

Figure 26 - Log Source Autodetected and Events are getting parsed and mapped

24.  Now when we go to Log source Management tab we can see that after 25 events Log Source is auto detected and started showing up on the Log Activity tab.

Figure 27 - Log Source is Autodetected on LSM App

Open and RFE:

• Open a request for enhancement on IBM Ideas portal to get officially supported DSM.
• Go to the IBM® QRadar® SIEM IBM Ideas page https://ibmsecurity.ideas.ibm.com/
• Log in to the support portal page.
• Click the Submit tab and type the necessary information.

Professional Services :

Contact your IBM Account Manager or IBM Sales Representative and request for Professional Services/ SEL. They will do an assessment on the integration and will review any customisation requirements for the integration. This is paid engagement, Client will be billed for the services provided.

Conclusion:

• You can now  map any unmapped and unparsed events reaching to IBM® QRadar® SIEM With help of DSM editor.
• We can now use use all the parsed properties in Searches, Reports, Rules and Dashboards.
• This custom DSM is unsupported by IBM. So if this starts showing up as expensive DSM you need to fine tune this DSM further. For tuning the regex go through how-to-avoid-while-writing-ceps-in-ibm-qradar
• First step would be to stop the Auto detection once we have required Log Sources auto detected. So that it will not add load on the Traffic Analysis Engine.

If at any point in time, you have any questions, have any comments or want to discuss this further, feel free to get in touch with IBM Support:

Vishal Tangadkar – vishal.tangadkar1@ibm.com
Special thanks for review  – Praphullachandra S Mujumdar  prmujumd@in.ibm.com

References:

https://www.ibm.com/docs/en/dsm?topic=configuration-qradar-supported-dsms
https://www.ibm.com/docs/en/dsm?topic=configuration-dsms-supported-by-third-party-vendors
https://www.ibm.com/docs/en/dsm?topic=configuration-undocumented-protocols
https://www.ibm.com/docs/en/dsm?topic=configuration-protocol-options
https://www.ibm.com/docs/en/qsip/7.5?topic=qradar-property-configuration-in-dsm-editor
https://www.ibm.com/docs/en/qradar-on-cloud?topic=qradar-dsm-editor-overview
https://community.ibm.com/community/user/security/blogs/saket-nimdeokar/2022/09/28/how-to-avoid-while-writing-ceps-in-ibm-qradar
https://community.ibm.com/community/user/security/blogs/saket-nimdeokar/2022/09/01/optimizing-cep-in-qradar
https://www.youtube.com/watch?v=KF40bba_kp0