When using QRadar's historical correlation, it seems that catalogs are not cleaned up even when deleting the historical correlation profile.
Specifically, if I do the following:
Create a saved search and a historical correlation profile that is expected to return results
Run the HC profile that was created in step 1
Click View History on the HC profile that was created in step 1, and observe the catalog name for the Run that is shown
Close the View History dialog
Delete the HC profile that was created in step 1
Go to Log Activity and run the query select * from HC_XXX, where HC_XXX is the catalog name that was observed in step 3
select * from HC_XXX
Observe that results from the catalog are returned, even though the historical correlation has been deleted
Given the above:
Where are the HC catalogs actually stored? Do they take up space?
What is the proper way of cleaning up a HC profile along with its associated catalogs?
A Historical Correlation profile is in essence a non-aggregated saved search, a list of enabled rules, scheduling, and metadata information. The profile is stored in the database and contains the parameters for the data stored within QRadar. HC uses the existing data within QRadar and uses the existing events run the data through the rules to generate historical correlation offenses.
I would confirm if the profile is deleted from the API to confirm the profile is removed.
I would think that these would be cleaned up at the top of the hour, but this might also be referencing the saved search that is still available. You might confirm if you still have managed search results (Log Activity > New Search > Manage Search Results). The profile is stored in the database and if you've deleted your profile and the data still displays, it might need to age out or this might be a defect of some sort. You didn't mention your version, but if you are still seeing issues. I would start by confirming if the profile is available in the API after it is deleted from the user interface. If the API still lists the name of your HC profile after removal from the UI, then it is likely a defect that should be logged with support. You didn't mention your version, but if it continues, open a case.
------------------------------Jonathan PechtaQRadar Support Content LeadSupport forums: email@example.com------------------------------
------------------------------Jonathan PechtaQRadar Support Content LeadSupport forums: firstname.lastname@example.orgOriginal Message:Sent: Thu September 09, 2021 06:08 AMFrom: Yair ManorSubject: Cleaning up historical correlation catalogs