Hello Community!
This blog will have a bit of a different spin... especially with the sustained and continued growth of SaaS security solutions and their partnering with specialized on-prem solutions.
So, you've been using QRadar SIEM on-prem for years and leadership decided to move to a SaaS solution to continue the use of OpEx for your monitoring strategy implementation and QRadar SIEM is on-prem only... But what about audits and retention requirements? Is it worth exporting all of the QRadar data and re-importing it to the new SaaS solution? Will this cause a negative cost impact? So many questions and considerations run through any security professional's mind as these sorts of strategy shifts happen and are outside of our their control.
There are several questions that come up while migrating solutions, especially when there is a mandate from leadership to align ourselves towards a SaaS-forward IT strategy to reduce capital expenses and move to operating expenses. It's frustrating as we have to give up some control, and as security professionals, that's a struggle. That said, SaaS solutions align themselves to standards and can provide evidence of meeting requirements enough to be officially certified and can provide some help through alleviating some head aches of security tool administration... Fair enough. Then come the auditors... so, what strategy makes most sense?
One possible method for data retention when finding one's self in a SIEM transition phase with a requirement that would be otherwise prohibitive for SaaS upload includes the use of the IBM QRadar Community Edition! With a few simple steps, it may be possible to provide auditors evidence of data retention based on their requests. Keep legacy QRadar appliances available either in a power down mode, or fully up and running. For the former, I would definitely recommend the use of out of band management options (think IMM).
Spin up QRadar Community Edition and implement the temporary license, these are published and made available every quarter with an EPS of 100, but for this use-case, no processing is needed. Copy relevant data based on time frame. A few options may be available.rsync: Use this tool to copy the relevant event directories from /store/ariel/ This is a great method for precise and granular data migrations and can be done on a case by case basis for time windows. syncAriel.sh: Use this script to copy and validate copied data, and pick back up where having left off if progress is interrupted. This is good for complete migrations. Re-index the data for faster searching using the ariel_offline_indexer.sh script
Not shifting away from on-prem QRadar SIEM but looking to minimize hot data storage requirements? This same approach may still work for cold data! If dealing with auditors and struggling to provide evidence for log retention because only 90 days of hot data can be stored but have a 12 month requirement? Move older data to cold storage, then restore what's needed, when needed... it's even possible to automate the process of uploading backup files to cloud storage, here's an example with Azure!
From there, it's a matter of following the same steps as detailed just above after extracting the data directories from the relevant backups. This can help limit the impact on production instances while dealing with the resource intensive administrative tasks like evidence gathering from cold data.
NOTE: Be sure to have a retention period configured on the Community Edition that aligns to requirements and is not the out of the box 1 month period.
NOTE: For distributed environments, consider a process where data is moved/synced to and indexed in the Community Edition Console as needed, and removed after use. Establishing a process where data sources and their log destinations are well documented, where selective data migration from individual event processors for the purposes of providing auditors evidence throughout an audit is well defined, can allow for meeting retention requirements for data as it ages out and also the ability to provide audit evidence for retention requirements being met.