IBM QRadar

IBM QRadar

Join this online user group to communicate across Security product users and IBM experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
Back to discussions
Expand all | Collapse all

[Universal Cloud Connector] No events in Log Activity despite test-workflow.sh showing events posted

  • 1.  [Universal Cloud Connector] No events in Log Activity despite test-workflow.sh showing events posted

    Posted Thu September 21, 2023 08:29 AM

    Hello,

    we are trying to create a log source using the Universal Cloud Connector. When clicking "Test" during the creation of the log source, we can see that events are getting pulled:

    Also, when testing the workflow using test-workflow.sh, we are seeing that events (should) get created:

    2023-09-21 13:11:49 [INFO ][UniversalCloudRESTAPITest] Received 100 events from [...]
    2023-09-21 13:11:49 [INFO ][UniversalCloudRESTAPITest] [Event payload 1]
    [...]
    2023-09-21 13:11:49 [INFO ][UniversalCloudRESTAPITest] [Event payload 100]
    2023-09-21 13:11:54 [INFO ][UniversalCloudRESTAPITest] Received 55 events from [...]
    2023-09-21 13:11:54 [INFO ][UniversalCloudRESTAPITest] [Event payload 101]
    2023-09-21 13:11:54 [INFO ][UniversalCloudRESTAPITest] [...]
    2023-09-21 13:11:54 [INFO ][UniversalCloudRESTAPITest] [Event payload 155]
    2023-09-21 13:11:57 [INFO ][UniversalCloudRESTAPITest] Received 0 events from [...]

    Unfortunately, when trying to show the events for this log source by clicking the "View Events"-button, there are no events visible. The log source has been active for more than half an hour now, and the reccurance is set to 5M (so it has had enough time to run at this point).

    Also, the content of /var/log/qradar.error clearly shows that the workflow has run from start to finish (to test this, we used <Log type="ERROR" message=<message> /> in the workflow-code).

    What could be possible causes of this issue?

    Kind regards,

    Julius



    ------------------------------
    Julius Cloos
    ------------------------------


  • 2.  RE: [Universal Cloud Connector] No events in Log Activity despite test-workflow.sh showing events posted

    Posted Tue October 03, 2023 09:57 AM

    Can you verify that there is a <PostEvent> XML element in your workflow.xml file?

    It will look something like this with either a hardcoded name or a variable to pull from the payload:

    <PostEvents path="/events" host="${/host}" /> 

    If no, then that is missing and is required to take the retrieved events and post them to the event pipeline. This field sets the Log Source Identifier for your log source and can be seen in debug where the fields retrieved are then posted to the event payload. 

    For example, 
    2023-09-06 15:59:27 [DEBUG][LogAction] We received a total of 100 Events.  
    2023-09-06 15:59:27 [DEBUG][PostEventsAction] PostEventsAction.execute() <-- This is the line that posts events from received data.
    2023-09-06 15:59:27 [DEBUG][WorkflowState] evaluate(/host) returns "hostname.example.com" <-- Sets LSI 
    2023-09-06 15:59:27 [DEBUG][PostEventsAction] Posting 100 events
    2023-09-06 15:59:27 [DEBUG][PostEventsAction] Posting Event:  ......

    The documentation has two sections about post events, one is in the section that talks about single events. The other is when you have multiple events with an array:

    1. Single event <PostEvent> 
    2. Array of events <PostEvents>

    Note how the action names are plural for the arrayed events. I believe your issue here is that you are requesting events, just not posting them to the pipeline in your test-workflow. The other item to check is ensure that your array names are correct. 



    ------------------------------
    Jonathan Pechta
    QRadar Support Content Lead
    Support forums: ibm.biz/qradarforums
    jonathan.pechta1@ibm.com
    ------------------------------

    Original Message


  • 3.  RE: [Universal Cloud Connector] No events in Log Activity despite test-workflow.sh showing events posted

    Posted Tue November 28, 2023 03:04 AM
    Edited by Martin Schmitt Wed December 20, 2023 04:20 AM

    Hello,

    i now got the case solved. I opened a support case. The Log message regarding binding SLF4J was not an issue, despite what the message tells. 

    If you want to understand the warning: https://www.baeldung.com/slf4j-classpath-multiple-bindings

    SLF4J: Found binding in [jar:file:/opt/ibm/si/services/ecs-ec-ingress/2021.6.7.20230822112654/bin/activemq-all-5.16.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/opt/ibm/si/services/ecs-ec-ingress/2021.6.7.20230822112654/bin/log4j-slf4j-impl-2.17.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
    In my case i looks like there has been not a lot of events to push to the event pipeline. So when i did the test in the GUI it showed me events that might have been inserted in the event log much earlier already.
    Nevertheless, there is a mistake in the Documentation that is confusing.
     
    In the section "PostEvents". The example there is wrong. Instead of the name host it should be source. 

    Greetings from Austria

    Martin



    ------------------------------
    Martin Schmitt
    ------------------------------

    Original Message


Related Content