Introduction

This article details storage provisioning for IBM QRadar in its current state and assess solutions for storage expansion specific to the AWS cloud environment. Although there are other technical solutions and approaches to address storage expansion, such as using a Logical Volume Manager (LVM), this article will focus on solutions supported per IBM. The goal is to allow organizations to plan and implement IBM QRadar effectively from the start.

Initial Storage Provisioning

Prior to VM Creation:

Before deploying QRadar, it is recommended to plan in advance for current storage requirements as well for potential growth. This plan should be inclusive of all the current and known future log sources, log size, and retention requirements. The goal is to understand both the current storage demands and also to plan for growth. With this plan in place, it’s very easy to provision the necessary storage volume during the initial deployment. 

During VM Creation:

Now, with the projected storage requirements in mind, the customer can deploy IBM QRadar. Currently AWS images for QRadar 7.4.1 in the AWS Marketplace come with a single disk of 122 GB in size and a customizable Data Disk added by the customer. The first disk contains QRadar OS partitions that are configured according to standard QRadar usage. The Data Disk is added by the customer during the initial provisioning from the marketplace. Customers have the flexibility to select the size required based on storage needs within the given options (for the full list of volumes types please see the AWS Documentation).

Example:

In the example below, 2 TB DATA disk is added.

Figure 1: Provisioning a Data Disk in the AWS Marketplace

Figure 2: Viewing the Data Disk (prior to the setup script)

Post VM Creation:

At this moment, the VM is created and ready for QRadar software installation. Now, the setup script will detect the added disk, perform the necessary checks (2nd disk exists; Minimum disk size is met; Disk is empty and not partitioned). The setup does not continue if the check conditions not met. Then, the setup script creates and configures the required partitions following QRadar requirements. (80/20% split between store and transient on added disk). Next, the system will proceed with the standard QRadar setup. When the installation is complete, this disk will contain the “/store” and “/transient” partitions. This is automatically done by QRadar via the setup scripts. 

            Figure 3: Viewing the disk (post setup script)

Feature highlights

• Any size disk can be added, the only limitation is the size of the single largest disk provided by AWS. Currently in AWS single largest disk max size is 16 TB.
• The Data disk setup and partitioning is fully automated. Also, there are multiple checks in place to catch any potential misconfigurations and ensure the process is repeatable. However, the disk should be added only during the initial provisioning, when QRadar is not installed yet. This cannot be done after the installation.

Storage Expansion Options

Although it is desirable to estimate and plan storage needs and growth in advance, there are situations in real life when it is not possible.

Situation 1: The growth happened faster than planned. For example, the initially estimated storage of 5 TB is not enough now and needs to be expanded to 10TB.

Situation 2: The organization requires a greater storage volume than available in a single disk; such when the data disk is already at capacity and cannot be further expanded.

Below describes the two solutions to address the previously mentioned scenarios.

1. Redeployment and Migration: Suitable for situation 1
2. Adding a Data Node: Suitable for both situation 1 and 2

Redeployment and Migration

The high-level steps of this approach are the following. For more details, please see the IBM Documentation

1. Back up the data from the current VM
2. Create a new VM using marketplace image
3. Add a data disk of the new desired size
4. Install QRadar
5. Restore the backup from old VM
6. Test to make sure all the information has moved over (some guidance here would be useful)
7. Delete the old VM

Adding a Data Node

The steps to add a data node with AWS are available in IBM Documentation (Appliance Type ID = 1400) and then there are details specific to the data node. Beyond the technical details, there are a few other items to be aware of.

• Ensure proper QRadar entitlements
• Plan for and account for additional management effort

…………………

Future work

One of the limitations from the redeployment and migration approach above is that customers are limited by the size of the single largest disk size. In AWS that is 16 TB. Recently AWS announced new General Availability of the new type of Elastic Block Store (EBS) volumes called io2 Block Express Volumes.  https://press.aboutamazon.com/news-releases/news-release-details/aws-announces-general-availability-amazon-ebs-io2-block-express

EBS Block Express is a next-generation storage server architecture that provides the highest block storage performance. With io2 volumes running on Block Express, customers can achieve sub-millisecond latency and provision a single io2 volume with up to 256,000 IOPS, 4,000 MB/second throughput, and 64 TB of capacity—a 4x increase in performance, throughput, and capacity for existing io2 volumes. io2 Block Express volumes are ideal for the largest, most I/O-intensive, mission-critical deployments of QRadar.

io2 Block Express volumes are available with R5b instances, the Amazon EC2 instance with the highest EBS bandwidth, in all regions where AWS offers R5b instances today.

In the coming months, with the next release of QRadar images in AWS, IBM is planning to enable the support for R5b instances and block express volumes, thus allowing customers to create QRadar deployment with up to 64 TB DATA disk storage.