IBM QRadar

Hello everyone,

I'm new to QRadar and I need to check if there is a product feature or configuration that allow me do the following:

I need to keep up data logs with one year of retention, but I want to save the last month on a storage tier based on flash disks, the other eleven months on a storage tier based on traditional, rotative disks.

Do you know if it is something duable and, if affermative, can you give me any directions?

Thanks a lot

------------------------------
Fabio Guzzi
------------------------------

I'm not aware of an easy method to do this in QRadar. The events and flows are kept in /store/ariel/ paths and stored locally on the appliance that parses, indexes, and stores the events to be searched. The only option I've seen around this type of feature in our docs is for Fibre Channel multipath offboard storage: https://www.ibm.com/docs/en/qsip/7.5?topic=fcs-moving-store-file-system-multipath-fibre-channel-solution

However, this option reduces storage redundancy and not what you are looking for as a method to reduce costs by moving /store to different storage performance pricing/tiers. I'm not aware of any other supported option other than the Fibre Channel solution that is documented.

Typically, unless it is documented it is considered unsupported. 

------------------------------
Jonathan Pechta
IBM Security - Community of Practice Lead
jonathan.pechta1@ibm.com

Original Message:
Sent: Wed April 10, 2024 10:58 AM
From: Fabio Guzzi
Subject: Put data log on different storage tiers based on retention

Hello everyone,

I'm new to QRadar and I need to check if there is a product feature or configuration that allow me do the following:

I need to keep up data logs with one year of retention, but I want to save the last month on a storage tier based on flash disks, the other eleven months on a storage tier based on traditional, rotative disks.

Do you know if it is something duable and, if affermative, can you give me any directions?

Thanks a lot

------------------------------
Fabio Guzzi
------------------------------

Thank you Jonathan!

I was thinking to create two QR deployments (A and B) and configure a 30-days bucket on the first one and 365-bucket on the second one.

QR deployment A would be provisioned with SDD storage while QR deployment B with traditional HDD 

The same log source would forward data to two event collectors, the first one becoming to QR deployment A while the second one becoming to QR deployment B.

I'm aware this way I would introduce some inefficiency (the first month would be on both deployments) but the advantages in terms of storage costs would be still relevant.

I would use QR deployment A for short term query (80% in my case) and QR deployment B for long term query (less frequent and less urgent so I can accept less performance).

Does it make sense?

Thanks  

I'm not aware of an easy method to do this in QRadar. The events and flows are kept in /store/ariel/ paths and stored locally on the appliance that parses, indexes, and stores the events to be searched. The only option I've seen around this type of feature in our docs is for Fibre Channel multipath offboard storage: https://www.ibm.com/docs/en/qsip/7.5?topic=fcs-moving-store-file-system-multipath-fibre-channel-solution

However, this option reduces storage redundancy and not what you are looking for as a method to reduce costs by moving /store to different storage performance pricing/tiers. I'm not aware of any other supported option other than the Fibre Channel solution that is documented.

Typically, unless it is documented it is considered unsupported. 

------------------------------
Jonathan Pechta
IBM Security - Community of Practice Lead
jonathan.pechta1@ibm.com

Original Message:
Sent: Wed April 10, 2024 10:58 AM
From: Fabio Guzzi
Subject: Put data log on different storage tiers based on retention

Hello everyone,

I'm new to QRadar and I need to check if there is a product feature or configuration that allow me do the following:

I need to keep up data logs with one year of retention, but I want to save the last month on a storage tier based on flash disks, the other eleven months on a storage tier based on traditional, rotative disks.

Do you know if it is something duable and, if affermative, can you give me any directions?

Thanks a lot

------------------------------
Fabio Guzzi
------------------------------

Hi Fabio,

What you describe in you're reply above with 2 deployments would achieve what you want.   However you will require 2 sets of QRadar licenses.

Also if both systems are required to generate offenses and be used for monitoring then you would need to ensure that you have an effective Change Control process in place.

If A is only going to be used for monitoring and B for searching then there would not be a need to keep the rules in sync, you would however have to keep any CEP's in sync.

Also would there be any pull logsources in the deployment or would they all be push?

Thanks

------------------------------
John Dawson
Qradar Support Architect
IBM

Original Message:
Sent: Thu April 11, 2024 02:23 AM
From: Fabio Guzzi
Subject: Put data log on different storage tiers based on retention

Thank you Jonathan!

I was thinking to create two QR deployments (A and B) and configure a 30-days bucket on the first one and 365-bucket on the second one.

QR deployment A would be provisioned with SDD storage while QR deployment B with traditional HDD 

The same log source would forward data to two event collectors, the first one becoming to QR deployment A while the second one becoming to QR deployment B.

I'm aware this way I would introduce some inefficiency (the first month would be on both deployments) but the advantages in terms of storage costs would be still relevant.

I would use QR deployment A for short term query (80% in my case) and QR deployment B for long term query (less frequent and less urgent so I can accept less performance).

Does it make sense?

Thanks  

------------------------------
Fabio Guzzi

Original Message:
Sent: Wed April 10, 2024 05:16 PM
From: Jonathan Pechta
Subject: Put data log on different storage tiers based on retention

I'm not aware of an easy method to do this in QRadar. The events and flows are kept in /store/ariel/ paths and stored locally on the appliance that parses, indexes, and stores the events to be searched. The only option I've seen around this type of feature in our docs is for Fibre Channel multipath offboard storage: https://www.ibm.com/docs/en/qsip/7.5?topic=fcs-moving-store-file-system-multipath-fibre-channel-solution

However, this option reduces storage redundancy and not what you are looking for as a method to reduce costs by moving /store to different storage performance pricing/tiers. I'm not aware of any other supported option other than the Fibre Channel solution that is documented.

Typically, unless it is documented it is considered unsupported. 

------------------------------
Jonathan Pechta
IBM Security - Community of Practice Lead
jonathan.pechta1@ibm.com

Original Message:
Sent: Wed April 10, 2024 10:58 AM
From: Fabio Guzzi
Subject: Put data log on different storage tiers based on retention

Hello everyone,

I'm new to QRadar and I need to check if there is a product feature or configuration that allow me do the following:

I need to keep up data logs with one year of retention, but I want to save the last month on a storage tier based on flash disks, the other eleven months on a storage tier based on traditional, rotative disks.

Do you know if it is something duable and, if affermative, can you give me any directions?

Thanks a lot

------------------------------
Fabio Guzzi
------------------------------

Thank you Jonathan!

I was thinking to create two QR deployments (A and B) and configure a 30-days bucket on the first one and 365-bucket on the second one.

QR deployment A would be provisioned with SDD storage while QR deployment B with traditional HDD 

The same log source would forward data to two event collectors, the first one becoming to QR deployment A while the second one becoming to QR deployment B.

I'm aware this way I would introduce some inefficiency (the first month would be on both deployments) but the advantages in terms of storage costs would be still relevant.

I would use QR deployment A for short term query (80% in my case) and QR deployment B for long term query (less frequent and less urgent so I can accept less performance).

Does it make sense?

Thanks  

I'm not aware of an easy method to do this in QRadar. The events and flows are kept in /store/ariel/ paths and stored locally on the appliance that parses, indexes, and stores the events to be searched. The only option I've seen around this type of feature in our docs is for Fibre Channel multipath offboard storage: https://www.ibm.com/docs/en/qsip/7.5?topic=fcs-moving-store-file-system-multipath-fibre-channel-solution

However, this option reduces storage redundancy and not what you are looking for as a method to reduce costs by moving /store to different storage performance pricing/tiers. I'm not aware of any other supported option other than the Fibre Channel solution that is documented.

Typically, unless it is documented it is considered unsupported. 

------------------------------
Jonathan Pechta
IBM Security - Community of Practice Lead
jonathan.pechta1@ibm.com

Original Message:
Sent: Wed April 10, 2024 10:58 AM
From: Fabio Guzzi
Subject: Put data log on different storage tiers based on retention

Hello everyone,

I'm new to QRadar and I need to check if there is a product feature or configuration that allow me do the following:

I need to keep up data logs with one year of retention, but I want to save the last month on a storage tier based on flash disks, the other eleven months on a storage tier based on traditional, rotative disks.

Do you know if it is something duable and, if affermative, can you give me any directions?

Thanks a lot

------------------------------
Fabio Guzzi
------------------------------

A solution that may be less expensive than two QR systems would be to use two data nodes or event processors for storage, set the retention policies different on each.  Problem is it would balance storage between the datanodes.  If you did multiple EP's that may do it. 

Thank you Jonathan!

I was thinking to create two QR deployments (A and B) and configure a 30-days bucket on the first one and 365-bucket on the second one.

QR deployment A would be provisioned with SDD storage while QR deployment B with traditional HDD 

The same log source would forward data to two event collectors, the first one becoming to QR deployment A while the second one becoming to QR deployment B.

I'm aware this way I would introduce some inefficiency (the first month would be on both deployments) but the advantages in terms of storage costs would be still relevant.

I would use QR deployment A for short term query (80% in my case) and QR deployment B for long term query (less frequent and less urgent so I can accept less performance).

Does it make sense?

Thanks  

I'm not aware of an easy method to do this in QRadar. The events and flows are kept in /store/ariel/ paths and stored locally on the appliance that parses, indexes, and stores the events to be searched. The only option I've seen around this type of feature in our docs is for Fibre Channel multipath offboard storage: https://www.ibm.com/docs/en/qsip/7.5?topic=fcs-moving-store-file-system-multipath-fibre-channel-solution

However, this option reduces storage redundancy and not what you are looking for as a method to reduce costs by moving /store to different storage performance pricing/tiers. I'm not aware of any other supported option other than the Fibre Channel solution that is documented.

Typically, unless it is documented it is considered unsupported. 

------------------------------
Jonathan Pechta
IBM Security - Community of Practice Lead
jonathan.pechta1@ibm.com

Original Message:
Sent: Wed April 10, 2024 10:58 AM
From: Fabio Guzzi
Subject: Put data log on different storage tiers based on retention

Hello everyone,

I'm new to QRadar and I need to check if there is a product feature or configuration that allow me do the following:

I need to keep up data logs with one year of retention, but I want to save the last month on a storage tier based on flash disks, the other eleven months on a storage tier based on traditional, rotative disks.

Do you know if it is something duable and, if affermative, can you give me any directions?

Thanks a lot

------------------------------
Fabio Guzzi
------------------------------

Thank you Frank, I agree better with only one QR deployment!

Not sure I got what you mean when you say "Problem is it would balance storage between the datanodes", could you please further explain?

Another question is: is there a way to connect the same EC to multiple EPs and send them the same log record? That way I could act on collector side only, avoiding to change the log source configuration with multiple forward directives.

Thanks

Fabio

A solution that may be less expensive than two QR systems would be to use two data nodes or event processors for storage, set the retention policies different on each.  Problem is it would balance storage between the datanodes.  If you did multiple EP's that may do it. 

Thank you Jonathan!

I was thinking to create two QR deployments (A and B) and configure a 30-days bucket on the first one and 365-bucket on the second one.

QR deployment A would be provisioned with SDD storage while QR deployment B with traditional HDD 

The same log source would forward data to two event collectors, the first one becoming to QR deployment A while the second one becoming to QR deployment B.

I'm aware this way I would introduce some inefficiency (the first month would be on both deployments) but the advantages in terms of storage costs would be still relevant.

I would use QR deployment A for short term query (80% in my case) and QR deployment B for long term query (less frequent and less urgent so I can accept less performance).

Does it make sense?

Thanks  

I'm not aware of an easy method to do this in QRadar. The events and flows are kept in /store/ariel/ paths and stored locally on the appliance that parses, indexes, and stores the events to be searched. The only option I've seen around this type of feature in our docs is for Fibre Channel multipath offboard storage: https://www.ibm.com/docs/en/qsip/7.5?topic=fcs-moving-store-file-system-multipath-fibre-channel-solution

However, this option reduces storage redundancy and not what you are looking for as a method to reduce costs by moving /store to different storage performance pricing/tiers. I'm not aware of any other supported option other than the Fibre Channel solution that is documented.

Typically, unless it is documented it is considered unsupported. 

------------------------------
Jonathan Pechta
IBM Security - Community of Practice Lead
jonathan.pechta1@ibm.com

Original Message:
Sent: Wed April 10, 2024 10:58 AM
From: Fabio Guzzi
Subject: Put data log on different storage tiers based on retention

Hello everyone,

I'm new to QRadar and I need to check if there is a product feature or configuration that allow me do the following:

I need to keep up data logs with one year of retention, but I want to save the last month on a storage tier based on flash disks, the other eleven months on a storage tier based on traditional, rotative disks.

Do you know if it is something duable and, if affermative, can you give me any directions?

Thanks a lot

------------------------------
Fabio Guzzi
------------------------------

A solution that may be less expensive than two QR systems would be to use two data nodes or event processors for storage, set the retention policies different on each.  Problem is it would balance storage between the datanodes.  If you did multiple EP's that may do it. 

Thank you Jonathan!

I was thinking to create two QR deployments (A and B) and configure a 30-days bucket on the first one and 365-bucket on the second one.

QR deployment A would be provisioned with SDD storage while QR deployment B with traditional HDD 

The same log source would forward data to two event collectors, the first one becoming to QR deployment A while the second one becoming to QR deployment B.

I'm aware this way I would introduce some inefficiency (the first month would be on both deployments) but the advantages in terms of storage costs would be still relevant.

I would use QR deployment A for short term query (80% in my case) and QR deployment B for long term query (less frequent and less urgent so I can accept less performance).

Does it make sense?

Thanks  

I'm not aware of an easy method to do this in QRadar. The events and flows are kept in /store/ariel/ paths and stored locally on the appliance that parses, indexes, and stores the events to be searched. The only option I've seen around this type of feature in our docs is for Fibre Channel multipath offboard storage: https://www.ibm.com/docs/en/qsip/7.5?topic=fcs-moving-store-file-system-multipath-fibre-channel-solution

However, this option reduces storage redundancy and not what you are looking for as a method to reduce costs by moving /store to different storage performance pricing/tiers. I'm not aware of any other supported option other than the Fibre Channel solution that is documented.

Typically, unless it is documented it is considered unsupported. 

------------------------------
Jonathan Pechta
IBM Security - Community of Practice Lead
jonathan.pechta1@ibm.com

Original Message:
Sent: Wed April 10, 2024 10:58 AM
From: Fabio Guzzi
Subject: Put data log on different storage tiers based on retention

Hello everyone,

I'm new to QRadar and I need to check if there is a product feature or configuration that allow me do the following:

I need to keep up data logs with one year of retention, but I want to save the last month on a storage tier based on flash disks, the other eleven months on a storage tier based on traditional, rotative disks.

Do you know if it is something duable and, if affermative, can you give me any directions?

Thanks a lot

------------------------------
Fabio Guzzi
------------------------------