IBM QRadar

Hi everyone,


I need to parse an event whose example shown as below. For example, How can I parse Failed or Succeeded text without affect the performance of QRadar?


17971908,someuser,Inner User,10.10.10.10,2020-06-01 19:57:12,Inventory Management,Query power from NE,someotheruser,Failed,something,2020-05-30 19:57:12

25970818,someuser,Inner User,10.10.10.10,2020-06-01 17:04:38,Inventory Management,Set project object status,Project Alarm,Succeeded,2020-05-30 17:04:38 


Thank you for your help in advance.

------------------------------
Halil BALIM
------------------------------

If only that part is the subject and you do not have many different options expected maybe the easiest way could be the explicit RegEx statement such as (Failed|Succeeded)

------------------------------
Dusan VIDOVIC
------------------------------

Original Message:
Sent: Sun May 31, 2020 05:50 PM
From: Halil BALIM
Subject: Parsing the Fields Separated by Comma (RegEx)

Hi everyone,


I need to parse an event whose example shown as below. For example, How can I parse Failed or Succeeded text without affect the performance of QRadar?


17971908,someuser,Inner User,10.10.10.10,2020-06-01 19:57:12,Inventory Management,Query power from NE,someotheruser,Failed,something,2020-05-30 19:57:12

25970818,someuser,Inner User,10.10.10.10,2020-06-01 17:04:38,Inventory Management,Set project object status,Project Alarm,Succeeded,2020-05-30 17:04:38 


Thank you for your help in advance.

------------------------------
Halil BALIM
------------------------------

Hi Dusan, 

I really appreciate your help. This works for this case. Thank you.

However, at the column which says Query power from NE part, there are 10 different values to be typed. 
Which regex would be the best for this case?

Such as:

Create a fiber/cable on Main Topology
Delete fibers/cables that carry trails
Create a fiber/cable

Best Regards.

------------------------------
Halil BALIM
------------------------------

Original Message:
Sent: Mon June 01, 2020 03:02 AM
From: Dusan VIDOVIC
Subject: Parsing the Fields Separated by Comma (RegEx)

If only that part is the subject and you do not have many different options expected maybe the easiest way could be the explicit RegEx statement such as (Failed|Succeeded)

------------------------------
Dusan VIDOVIC

Original Message:
Sent: Sun May 31, 2020 05:50 PM
From: Halil BALIM
Subject: Parsing the Fields Separated by Comma (RegEx)

Hi everyone,


I need to parse an event whose example shown as below. For example, How can I parse Failed or Succeeded text without affect the performance of QRadar?


17971908,someuser,Inner User,10.10.10.10,2020-06-01 19:57:12,Inventory Management,Query power from NE,someotheruser,Failed,something,2020-05-30 19:57:12

25970818,someuser,Inner User,10.10.10.10,2020-06-01 17:04:38,Inventory Management,Set project object status,Project Alarm,Succeeded,2020-05-30 17:04:38 


Thank you for your help in advance.

------------------------------
Halil BALIM
------------------------------

Since it varies and the part in between also is not very structured maybe :
\,([\w\s]+)\,[\w\s]+\,(Succeeded|Failed)\,
and use only the first capture group
... if the sentence you are looking for cannot have numbers than maybe use
\,([A-Za-z\s]+)\,[\w\s]+\,(Succeeded|Failed)\,
NOTE: sometimes it can happen that you get errors if escaping the character such as \,  so then avoiding the backslash \ should help.

------------------------------
Dusan VIDOVIC
------------------------------

Original Message:
Sent: Mon June 01, 2020 05:14 AM
From: Halil BALIM
Subject: Parsing the Fields Separated by Comma (RegEx)

Hi Dusan, 

I really appreciate your help. This works for this case. Thank you.

However, at the column which says Query power from NE part, there are 10 different values to be typed. 
Which regex would be the best for this case?

Such as:

Create a fiber/cable on Main Topology
Delete fibers/cables that carry trails
Create a fiber/cable

Best Regards.

------------------------------
Halil BALIM

Original Message:
Sent: Mon June 01, 2020 03:02 AM
From: Dusan VIDOVIC
Subject: Parsing the Fields Separated by Comma (RegEx)

If only that part is the subject and you do not have many different options expected maybe the easiest way could be the explicit RegEx statement such as (Failed|Succeeded)

------------------------------
Dusan VIDOVIC

Original Message:
Sent: Sun May 31, 2020 05:50 PM
From: Halil BALIM
Subject: Parsing the Fields Separated by Comma (RegEx)

Hi everyone,


I need to parse an event whose example shown as below. For example, How can I parse Failed or Succeeded text without affect the performance of QRadar?


17971908,someuser,Inner User,10.10.10.10,2020-06-01 19:57:12,Inventory Management,Query power from NE,someotheruser,Failed,something,2020-05-30 19:57:12

25970818,someuser,Inner User,10.10.10.10,2020-06-01 17:04:38,Inventory Management,Set project object status,Project Alarm,Succeeded,2020-05-30 17:04:38 


Thank you for your help in advance.

------------------------------
Halil BALIM
------------------------------

Just an FYI that if you're running QRadar 7.3.3 or later, it's possible to process list-type event formats in a more efficient way, without using regex. The "Generic List" expression type allows you to define a delimiter character sequence (in this case it would just be a comma), then refer to a particular item in the list by its index: $0 for the first item, $1 for the next, and so on. More information is here:

https://www.ibm.com/support/knowledgecenter/SSKMKU/com.ibm.qradar.doc/t_qradar_adm_dsm_ed_writing_generic_list_exp.html

Cheers
Colin

------------------------------
COLIN HAY
IBM Security
------------------------------

Original Message:
Sent: Mon June 01, 2020 05:47 AM
From: Dusan VIDOVIC
Subject: Parsing the Fields Separated by Comma (RegEx)

Since it varies and the part in between also is not very structured maybe :
\,([\w\s]+)\,[\w\s]+\,(Succeeded|Failed)\,
and use only the first capture group
... if the sentence you are looking for cannot have numbers than maybe use
\,([A-Za-z\s]+)\,[\w\s]+\,(Succeeded|Failed)\,
NOTE: sometimes it can happen that you get errors if escaping the character such as \,  so then avoiding the backslash \ should help.

------------------------------
Dusan VIDOVIC

Original Message:
Sent: Mon June 01, 2020 05:14 AM
From: Halil BALIM
Subject: Parsing the Fields Separated by Comma (RegEx)

Hi Dusan, 

I really appreciate your help. This works for this case. Thank you.

However, at the column which says Query power from NE part, there are 10 different values to be typed. 
Which regex would be the best for this case?

Such as:

Create a fiber/cable on Main Topology
Delete fibers/cables that carry trails
Create a fiber/cable

Best Regards.

------------------------------
Halil BALIM

Original Message:
Sent: Mon June 01, 2020 03:02 AM
From: Dusan VIDOVIC
Subject: Parsing the Fields Separated by Comma (RegEx)

If only that part is the subject and you do not have many different options expected maybe the easiest way could be the explicit RegEx statement such as (Failed|Succeeded)

------------------------------
Dusan VIDOVIC

Original Message:
Sent: Sun May 31, 2020 05:50 PM
From: Halil BALIM
Subject: Parsing the Fields Separated by Comma (RegEx)

Hi everyone,


I need to parse an event whose example shown as below. For example, How can I parse Failed or Succeeded text without affect the performance of QRadar?


17971908,someuser,Inner User,10.10.10.10,2020-06-01 19:57:12,Inventory Management,Query power from NE,someotheruser,Failed,something,2020-05-30 19:57:12

25970818,someuser,Inner User,10.10.10.10,2020-06-01 17:04:38,Inventory Management,Set project object status,Project Alarm,Succeeded,2020-05-30 17:04:38 


Thank you for your help in advance.

------------------------------
Halil BALIM
------------------------------

Hi Everyone,

I appreciate your guidance. I have at least solved the Succeeded|Failed column. Other columns still using more resource. Since I could not find a way to solve them at 7.3.2 patch 6 version of QRadar.

I have a little question by the way,

Which way is the best for parsing events:

Writing an LSX document and uploading it as Log Source Extension

or

Parse the events via DSM Editor.


I know they both are xml but which one is more effective?

Thank you for your responses in advance.

Regards. 



------------------------------
Halil BALIM
------------------------------

Original Message:
Sent: Mon June 01, 2020 11:11 AM
From: COLIN HAY
Subject: Parsing the Fields Separated by Comma (RegEx)

Just an FYI that if you're running QRadar 7.3.3 or later, it's possible to process list-type event formats in a more efficient way, without using regex. The "Generic List" expression type allows you to define a delimiter character sequence (in this case it would just be a comma), then refer to a particular item in the list by its index: $0 for the first item, $1 for the next, and so on. More information is here:

https://www.ibm.com/support/knowledgecenter/SSKMKU/com.ibm.qradar.doc/t_qradar_adm_dsm_ed_writing_generic_list_exp.html

Cheers
Colin

------------------------------
COLIN HAY
IBM Security

Original Message:
Sent: Mon June 01, 2020 05:47 AM
From: Dusan VIDOVIC
Subject: Parsing the Fields Separated by Comma (RegEx)

Since it varies and the part in between also is not very structured maybe :
\,([\w\s]+)\,[\w\s]+\,(Succeeded|Failed)\,
and use only the first capture group
... if the sentence you are looking for cannot have numbers than maybe use
\,([A-Za-z\s]+)\,[\w\s]+\,(Succeeded|Failed)\,
NOTE: sometimes it can happen that you get errors if escaping the character such as \,  so then avoiding the backslash \ should help.

------------------------------
Dusan VIDOVIC

Original Message:
Sent: Mon June 01, 2020 05:14 AM
From: Halil BALIM
Subject: Parsing the Fields Separated by Comma (RegEx)

Hi Dusan, 

I really appreciate your help. This works for this case. Thank you.

However, at the column which says Query power from NE part, there are 10 different values to be typed. 
Which regex would be the best for this case?

Such as:

Create a fiber/cable on Main Topology
Delete fibers/cables that carry trails
Create a fiber/cable

Best Regards.

------------------------------
Halil BALIM

Original Message:
Sent: Mon June 01, 2020 03:02 AM
From: Dusan VIDOVIC
Subject: Parsing the Fields Separated by Comma (RegEx)

If only that part is the subject and you do not have many different options expected maybe the easiest way could be the explicit RegEx statement such as (Failed|Succeeded)

------------------------------
Dusan VIDOVIC

Original Message:
Sent: Sun May 31, 2020 05:50 PM
From: Halil BALIM
Subject: Parsing the Fields Separated by Comma (RegEx)

Hi everyone,


I need to parse an event whose example shown as below. For example, How can I parse Failed or Succeeded text without affect the performance of QRadar?


17971908,someuser,Inner User,10.10.10.10,2020-06-01 19:57:12,Inventory Management,Query power from NE,someotheruser,Failed,something,2020-05-30 19:57:12

25970818,someuser,Inner User,10.10.10.10,2020-06-01 17:04:38,Inventory Management,Set project object status,Project Alarm,Succeeded,2020-05-30 17:04:38 


Thank you for your help in advance.

------------------------------
Halil BALIM
------------------------------

Hi Halil,

The DSM Editor actually generates a Log Source Extension on the user's behalf, so they're the same thing. It's possible to handle some very complex scenarios with a manually-written LSX that the DSM Editor may not be able to represent in its visual interface, but 95% of the time the DSM Editor should be able to do whatever you want and it's easier to use and provides more feedback, so I'd always recommend the DSM Editor unless you're dealing with a very specialized situation that you can't get working in the DSM Editor.

Cheers
Colin

------------------------------
COLIN HAY
IBM Security
------------------------------

Original Message:
Sent: Mon June 08, 2020 05:07 AM
From: Halil BALIM
Subject: Parsing the Fields Separated by Comma (RegEx)

Hi Everyone,

I appreciate your guidance. I have at least solved the Succeeded|Failed column. Other columns still using more resource. Since I could not find a way to solve them at 7.3.2 patch 6 version of QRadar.

I have a little question by the way,

Which way is the best for parsing events:

Writing an LSX document and uploading it as Log Source Extension

or

Parse the events via DSM Editor.


I know they both are xml but which one is more effective?

Thank you for your responses in advance.

Regards. 



------------------------------
Halil BALIM

Original Message:
Sent: Mon June 01, 2020 11:11 AM
From: COLIN HAY
Subject: Parsing the Fields Separated by Comma (RegEx)

Just an FYI that if you're running QRadar 7.3.3 or later, it's possible to process list-type event formats in a more efficient way, without using regex. The "Generic List" expression type allows you to define a delimiter character sequence (in this case it would just be a comma), then refer to a particular item in the list by its index: $0 for the first item, $1 for the next, and so on. More information is here:

https://www.ibm.com/support/knowledgecenter/SSKMKU/com.ibm.qradar.doc/t_qradar_adm_dsm_ed_writing_generic_list_exp.html

Cheers
Colin

------------------------------
COLIN HAY
IBM Security

Original Message:
Sent: Mon June 01, 2020 05:47 AM
From: Dusan VIDOVIC
Subject: Parsing the Fields Separated by Comma (RegEx)

Since it varies and the part in between also is not very structured maybe :
\,([\w\s]+)\,[\w\s]+\,(Succeeded|Failed)\,
and use only the first capture group
... if the sentence you are looking for cannot have numbers than maybe use
\,([A-Za-z\s]+)\,[\w\s]+\,(Succeeded|Failed)\,
NOTE: sometimes it can happen that you get errors if escaping the character such as \,  so then avoiding the backslash \ should help.

------------------------------
Dusan VIDOVIC

Original Message:
Sent: Mon June 01, 2020 05:14 AM
From: Halil BALIM
Subject: Parsing the Fields Separated by Comma (RegEx)

Hi Dusan, 

I really appreciate your help. This works for this case. Thank you.

However, at the column which says Query power from NE part, there are 10 different values to be typed. 
Which regex would be the best for this case?

Such as:

Create a fiber/cable on Main Topology
Delete fibers/cables that carry trails
Create a fiber/cable

Best Regards.

------------------------------
Halil BALIM

Original Message:
Sent: Mon June 01, 2020 03:02 AM
From: Dusan VIDOVIC
Subject: Parsing the Fields Separated by Comma (RegEx)

If only that part is the subject and you do not have many different options expected maybe the easiest way could be the explicit RegEx statement such as (Failed|Succeeded)

------------------------------
Dusan VIDOVIC

Original Message:
Sent: Sun May 31, 2020 05:50 PM
From: Halil BALIM
Subject: Parsing the Fields Separated by Comma (RegEx)

Hi everyone,


I need to parse an event whose example shown as below. For example, How can I parse Failed or Succeeded text without affect the performance of QRadar?


17971908,someuser,Inner User,10.10.10.10,2020-06-01 19:57:12,Inventory Management,Query power from NE,someotheruser,Failed,something,2020-05-30 19:57:12

25970818,someuser,Inner User,10.10.10.10,2020-06-01 17:04:38,Inventory Management,Set project object status,Project Alarm,Succeeded,2020-05-30 17:04:38 


Thank you for your help in advance.

------------------------------
Halil BALIM
------------------------------

Hi Colin,

I appreciate your help.

Regards.

------------------------------
Halil BALIM
------------------------------

Original Message:
Sent: Mon June 08, 2020 02:35 PM
From: COLIN HAY
Subject: Parsing the Fields Separated by Comma (RegEx)

Hi Halil,

The DSM Editor actually generates a Log Source Extension on the user's behalf, so they're the same thing. It's possible to handle some very complex scenarios with a manually-written LSX that the DSM Editor may not be able to represent in its visual interface, but 95% of the time the DSM Editor should be able to do whatever you want and it's easier to use and provides more feedback, so I'd always recommend the DSM Editor unless you're dealing with a very specialized situation that you can't get working in the DSM Editor.

Cheers
Colin

------------------------------
COLIN HAY
IBM Security

Original Message:
Sent: Mon June 08, 2020 05:07 AM
From: Halil BALIM
Subject: Parsing the Fields Separated by Comma (RegEx)

Hi Everyone,

I appreciate your guidance. I have at least solved the Succeeded|Failed column. Other columns still using more resource. Since I could not find a way to solve them at 7.3.2 patch 6 version of QRadar.

I have a little question by the way,

Which way is the best for parsing events:

Writing an LSX document and uploading it as Log Source Extension

or

Parse the events via DSM Editor.


I know they both are xml but which one is more effective?

Thank you for your responses in advance.

Regards. 



------------------------------
Halil BALIM

Original Message:
Sent: Mon June 01, 2020 11:11 AM
From: COLIN HAY
Subject: Parsing the Fields Separated by Comma (RegEx)

Just an FYI that if you're running QRadar 7.3.3 or later, it's possible to process list-type event formats in a more efficient way, without using regex. The "Generic List" expression type allows you to define a delimiter character sequence (in this case it would just be a comma), then refer to a particular item in the list by its index: $0 for the first item, $1 for the next, and so on. More information is here:

https://www.ibm.com/support/knowledgecenter/SSKMKU/com.ibm.qradar.doc/t_qradar_adm_dsm_ed_writing_generic_list_exp.html

Cheers
Colin

------------------------------
COLIN HAY
IBM Security

Original Message:
Sent: Mon June 01, 2020 05:47 AM
From: Dusan VIDOVIC
Subject: Parsing the Fields Separated by Comma (RegEx)

Since it varies and the part in between also is not very structured maybe :
\,([\w\s]+)\,[\w\s]+\,(Succeeded|Failed)\,
and use only the first capture group
... if the sentence you are looking for cannot have numbers than maybe use
\,([A-Za-z\s]+)\,[\w\s]+\,(Succeeded|Failed)\,
NOTE: sometimes it can happen that you get errors if escaping the character such as \,  so then avoiding the backslash \ should help.

------------------------------
Dusan VIDOVIC

Original Message:
Sent: Mon June 01, 2020 05:14 AM
From: Halil BALIM
Subject: Parsing the Fields Separated by Comma (RegEx)

Hi Dusan, 

I really appreciate your help. This works for this case. Thank you.

However, at the column which says Query power from NE part, there are 10 different values to be typed. 
Which regex would be the best for this case?

Such as:

Create a fiber/cable on Main Topology
Delete fibers/cables that carry trails
Create a fiber/cable

Best Regards.

------------------------------
Halil BALIM

Original Message:
Sent: Mon June 01, 2020 03:02 AM
From: Dusan VIDOVIC
Subject: Parsing the Fields Separated by Comma (RegEx)

If only that part is the subject and you do not have many different options expected maybe the easiest way could be the explicit RegEx statement such as (Failed|Succeeded)

------------------------------
Dusan VIDOVIC

Original Message:
Sent: Sun May 31, 2020 05:50 PM
From: Halil BALIM
Subject: Parsing the Fields Separated by Comma (RegEx)

Hi everyone,


I need to parse an event whose example shown as below. For example, How can I parse Failed or Succeeded text without affect the performance of QRadar?


17971908,someuser,Inner User,10.10.10.10,2020-06-01 19:57:12,Inventory Management,Query power from NE,someotheruser,Failed,something,2020-05-30 19:57:12

25970818,someuser,Inner User,10.10.10.10,2020-06-01 17:04:38,Inventory Management,Set project object status,Project Alarm,Succeeded,2020-05-30 17:04:38 


Thank you for your help in advance.

------------------------------
Halil BALIM
------------------------------