An Event Collector is a 1-1 connection from the EC component to an EP component on another host, such as an Event Processor or Console. ECs only send data to a single EP that they are connected to in Admin > System & License Mgmt > Actions > Edit Host Connection. The Host Connection user interface only allows you to connect the EC to a single EP component. The event data received by the EC will be forwarded to the EP component it is connected to by default so the events can be processed (parsed, rules, stored to local disk or Data Node, etc). Be aware that EP components do exist on multiple appliance types, such as Event Processors and Console appliances. This means that you can connect your EC to an Event Processor OR to a Console, but not multiple appliances as it is a 1-1 connection. If you go in to System and License Management, click Deployment Actions > View Deployment, you can see the where the EC is currently attached and how your current deployment is connected to difference appliances.
When an EC receives an event, the event is handed off to the EP it connected to, which is the next stage in the event pipeline. The EP parses, runs rules, and stores the event locally. If you had 3 ECs, they can all be connected to a single EP, but only 1 EP. As mentioned an EC is a 1-1 connection to the EP component.
The Target Event Collector field in the Log Source UI is about setting up connections and ensuring that the Event Collector has all of the requirements to listen or poll for new events, such as ports open, setting up file structures, paths, info required to remotely poll, setting up intervals for polling, etc.
Now, the important part about Target Event Collectors and your log sources.
Be wary of moving your Target Event Collector on polling log sources. For example, JDBC, MSRPC, Log File, or REST API log sources that use marker files to track events, such as S3 buckets without SQS queues. We've seen with AWS CloudTrail and BlueCoat log sources that markers get reset as when "polling" files, DBs, or buckets log source types typically read in data and store it on the EC locally to be forwarded. If there are connections still active in the background, the log source might still be reading data when it is switched to a new Target Event Collector. This can lead to events left-behind that haven't transitioned to the EP yet as connections were still open and marker files that haven't updated . As you do not want to leave behind events on the EC when you switch the Target Event Collector from EP-A -> EP-B.
-
- To avoid this, disable the log source and wait several minutes. As the current polling interval needs to complete and marker files need to update.
- Edit the log source and move the Target Event Collector to another EC.
- Re-enable the log source. This should prevent issues where the Target gets moved before a log source is done with work in progress when polling for events.
------------------------------
Jonathan Pechta
QRadar Support Content Lead
Support forums: ibm.biz/qradarforums
jonathan.pechta1@ibm.com------------------------------
Original Message:
Sent: Fri July 28, 2023 09:46 AM
From: Oscar Atienza
Subject: Target Event Collector parameter
Hello all.
Please, two specific questions regarding the "Target Event Collector" parameter of Log Source configuration in a distributed QRadar system:
- Is it possible to configure an Event Collector so that it sends the events to a specific Event Processor (in case there are several EP on the deployment)?.
- If an Event Processor is selected in the "Target Event Collector" option, it will be the EC of that EP that receives and processes the information received. Is that correct?.
Thank you in advanced.
Regards.
------------------------------
Oscar Atienza
------------------------------