IBM QRadar

 View Only

 How to read raw logs in payloads without records?

Onur Tufan's profile image
Onur Tufan posted Thu April 24, 2025 03:34 AM

Hello all,

We have urgent situation.

We had saved payloads records of 2023 March as backups to another server from QRadar, due to storage limits on QRadar. 

(/store/ariel/events/payloads/2023/3 & /store/ariel/events/records/2023/3)

However, when today we wanted to send those files back to QRadar for an audit issue, it appeared that records folder is missing.

And when we send /store/ariel/events/payloads/2023/3 folder back to QRadar, we cannot see those logs....

Therefore, we are failing from audit, which is very important.

So, we need to read log files in payload folder without records, even reading from notepad would work. 

But it is encrypted and we cannot read it.

Is there any way to open the log files in payloads and read them in clear text?

Thanks

Frank Eargle's profile image
Frank Eargle IBM Champion posted Mon April 28, 2025 08:33 AM

At a minimum, I think you will need to re-index the data, there is a script to do so: ariel_offline_indexer.sh.  You may also need to restart the ariel proxy service.  You can always open a ticket and they will walk you through it. 

Onur Tufan's profile image
Onur Tufan posted Mon April 28, 2025 08:40 AM

Hello Frank,

Thank you for the answers. I had tried to re-index the data and restarting ariel proxy service, but it didn't work... 
And I had opened a ticket, and they told me that there is no solution as far as they (support team) know, but told me to ask expert labs etc. That's why I am here :) 

Rory Bray's profile image
Rory Bray posted Thu May 01, 2025 10:16 AM

They aren't actually encrypted, they are simply stored in a binary format use by ariel.  I don't believe we have any tools on the host to re-read the payloads without the records but you can see the payload text in the files.

Using a common command like utility like `strings` will dump out the text.  This is very crude but you'll at least be able to demonstrate that you still have the data.

Rory Bray's profile image
Rory Bray posted Thu May 01, 2025 10:18 AM

for example on a local console I can display the last few events from a payload file thusly:

[root@upgca 10]# strings "payload_events~9_0~aa1e0320b566419a~874c626f6a727357~0" | tail -5
May  1 10:09:59 127.0.0.1  [MPC/PersisterThread@0000081290] com.ibm.si.mpc.magi.contrib.ModelPersister: [INFO] [NOT:0000006000][192.168.125.130/- -] [-/- -]No persistence commands harvested to process
May  1 10:09:59 127.0.0.1  [MPC/CleanupAndPersistence[1]] com.ibm.si.mpc.magi.contrib.ModelPersister: [INFO] [NOT:0000006000][192.168.125.130/- -] [-/- -]Saving TX 0000081290 0.01MB
May  1 10:09:59 127.0.0.1  [MPC/PersisterThread@0000081290] com.ibm.si.mpc.magi.contrib.ModelPersister: [INFO] [NOT:0000006000][192.168.125.130/- -] [-/- -]Processing TX 0000081290 (1/1) 0.01MB
May  1 10:09:59 127.0.0.1  [MPC/CleanupAndPersistence[1]] com.ibm.si.mpc.magi.contrib.ModelPersister: [INFO] [NOT:0000006000][192.168.125.130/- -] [-/- -]Harvested 0 commands in 0:00:00.003
May  1 10:09:59 127.0.0.1  [[type=com.ibm.si.ec.filters.stat.StatFilter][parent=upgep.test.local:ecs-ec/EC/Processor2]] com.ibm.si.ec.filters.stat.StatFilter: [INFO] [NOT:0000006000][192.168.125.132/- -] [-/- -] Events per second:  1s:0,0 (peak 383,383) (compression: 0%)  5s:89,89 (peak 97,97) (compression: 0%)  10s:89,89 (peak 105,105) (compression: 0%)  30s:89,89 (peak 102,102) (compression: 0%)  60s:90,90 (peak 96,96) (compression: 0%)
[root@upgca 10]# 

Related Content