IBM QRadar

 View Only

 QRadar CE - Only partial logs visible when importing CSV/TXT source

Prakash Chandra's profile image
Prakash Chandra posted Tue August 05, 2025 01:29 AM
Hello Community,
 
I’m using QRadar Community Edition (CE) and trying to ingest logs from a .csv and .txt file using a custom log source.
 
The issue I’m facing is:
- When I upload the file, only a few lines (e.g., 5–10) show up in Log Activity.
- The rest of the log lines are missing, even though they exist in the file.
- No parsing errors or DSM errors are visible.
 
What I’ve tried:
- Ensured the file format is consistent (e.g., timestamps, delimiters).
- Tried different encoding (UTF-8, plain text).
- Increased the file size limits and EPS settings temporarily (no change).
- Verified that QRadar is ingesting the file and reaching the end of the file.
 
I’m uploading the file via [choose one: SFTP / WinSCP / manually placing in `/storetmp` / other method].
 
Any help or suggestions on how to ingest the full content of a CSV or TXT log file into QRadar CE would be appreciated.
 
Thanks in advance!
 
Prakash

Karl Jaeger's profile image
Karl Jaeger IBM Champion posted Thu August 07, 2025 11:33 AM

Prakash

thanks for your question. Earlier this year I ran into the same problem with 7.5.0 CE. Other than you I am using logrun.pl for uploeading test data. However the result is the same. This is a severe problem as I am using CE for education purposes and the test data for demonstrating all kind of issues. My current workaround is to use live logsources only. Demonstrating dsmedit does not work at all. To me it looks like that some kind of AI is working around my test cases being used for more than 10 years. Some IBM developers out there listening?

Jonathan Pechta's profile image
Jonathan Pechta posted Thu August 14, 2025 01:14 PM

Are the events less than 32k in length? Are the events in English or are they multi-language? 

Normally, files are not placed in /storetmp then ingested locally, but QRadar would make a copy of the file using the Log File Protocol to retrieve the file from a remote source and then ingesting the data line-by-line. Optionally, you could also use WinCollect if the data was on Windows and use the File Forwarder option and point the destination at QRadar. WinCollect would then generate Syslog payloads from the data and forward it over. 

Optionally, if you want further reach/discussion on this question you could also post it to the QRadar subreddit.  

If you are trying to just run a log file from local storage though, logrun.pl as @Karl Jaeger mentioned is likely the best method. If the data is still being truncated, then there is something wrong with the file format that QRadar does not like. 

Related Content