IBM QRadar

IBM QRadar

Join this online user group to communicate across Security product users and IBM experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only

  • 1.  Qradar CE_ecs_ec and ecs_ep services are in failed state

    Posted Fri December 22, 2023 11:32 AM

    Hello Team,

    I've successfully installed Qradar community edition 7.3.3 on my system using VMware Workstation Player (version 17) with the following specifications:

    • RAM: 8 GB
    • CPU: 6 Cores
    • Storage: 250 GB

    After installation, I encountered issues with the ingress, ecs-ec, and ecs-ep services. I applied the workaround mentioned in the flash notice, which resolved the issue with the ingress service but left ecs-ec and ecs-ep in a failed state.

    I'm seeking your assistance in resolving this issue. Your help is greatly appreciated.

    Thanks in advance.

    Here are the current service statuses:

    ==========================================================================================================

    ecs-ec : 

    [root@localhost ~]# systemctl status ecs-ec.service
    ● ecs-ec.service - Event Correlation Services Event Collector
       Loaded: loaded (/usr/lib/systemd/system/ecs-ec.service; static; vendor preset: disabled)
       Active: failed (Result: exit-code) since Fri 2023-12-22 10:49:09 UTC; 39s ago
      Process: 9745 ExecStartPre=/opt/qradar/systemd/bin/generate_environment.sh ${SERVICENAME} (code=exited, status=255)
     
    Dec 22 10:49:09 localhost.localdomain systemd[1]: Starting Event Correlation Services Event Collector...
    Dec 22 10:49:09 localhost.localdomain ecs-ec[9745]: ecs-ec has been manually stopped.
    Dec 22 10:49:09 localhost.localdomain systemd[1]: ecs-ec.service: control process exited, code=exited status=255
    Dec 22 10:49:09 localhost.localdomain systemd[1]: Failed to start Event Correlation Services Event Collector.
    Dec 22 10:49:09 localhost.localdomain systemd[1]: Unit ecs-ec.service entered failed state.
    Dec 22 10:49:09 localhost.localdomain systemd[1]: ecs-ec.service failed.

    ecs-ep :

    [root@localhost ~]# systemctl status ecs-ep.service
    ● ecs-ep.service - Event Correlation Services Event Processor
       Loaded: loaded (/usr/lib/systemd/system/ecs-ep.service; static; vendor preset: disabled)
       Active: failed (Result: exit-code) since Fri 2023-12-22 10:49:38 UTC; 1s ago
      Process: 11693 ExecStartPre=/opt/qradar/systemd/bin/generate_environment.sh ${SERVICENAME} (code=exited, status=255)
     
    Dec 22 10:49:37 localhost.localdomain systemd[1]: Starting Event Correlation Services Event Processor...
    Dec 22 10:49:38 localhost.localdomain ecs-ep[11693]: ecs-ep has been manually stopped.
    Dec 22 10:49:38 localhost.localdomain systemd[1]: ecs-ep.service: control process exited, code=exited status=255
    Dec 22 10:49:38 localhost.localdomain systemd[1]: Failed to start Event Correlation Services Event Processor.
    Dec 22 10:49:38 localhost.localdomain systemd[1]: Unit ecs-ep.service entered failed state.
    Dec 22 10:49:38 localhost.localdomain systemd[1]: ecs-ep.service failed.

    ==========================================================================================================

    Attached the VM configuration and services screenshot for the reference.



    ------------------------------
    Regards,
    RK
    ------------------------------


  • 2.  RE: Qradar CE_ecs_ec and ecs_ep services are in failed state

    Posted Sat December 23, 2023 10:25 AM
    Edited by Qradarlearning SIEM Sat December 23, 2023 10:26 AM

    Followed the below steps but still the issue is not resolved.

    • Increased the RAM to 10 GB
    • stopped and started hostcontext, tomcat and hostservices services.

    Can someone please help me in resolving this issue.



    ------------------------------
    Regards,

    RK
    ------------------------------

    Original Message


  • 3.  RE: Qradar CE_ecs_ec and ecs_ep services are in failed state

    Posted Wed January 03, 2024 11:57 AM

    Have you tried reviewing the Trouble-shooting document:

    https://www.ibm.com/community/101/wp-content/uploads/sites/5/2023/07/QRadar_CE_Under_the_Radar_21Feb.pdf

    linked from the main page:
    https://www.ibm.com/community/101/qradar/ce/



    ------------------------------
    Comghall Morgan
    QRadar Support Architect
    IBM
    ------------------------------

    Original Message


  • 4.  RE: Qradar CE_ecs_ec and ecs_ep services are in failed state

    Posted Thu January 04, 2024 07:24 AM

    Hi Comghall Morgan,

    Thank you for the response.

    Yes, Reviewed the attached troubleshooting document. 

    I didn't face any issues while installation with respect the network settings and others, mentioned in the T-shooting document.

    Issue I can observe is ECS-EC and ECS-EP services are not getting started. 

    Can you please suggest the troubleshooting steps which I need to perform.

     



    ------------------------------
    Qradarlearning SIEM
    ------------------------------

    Original Message


  • 5.  RE: Qradar CE_ecs_ec and ecs_ep services are in failed state

    Posted Mon January 08, 2024 05:33 AM

    Hmm, I just checked my CE test system reporting in qradar.log

    Jan  8 10:31:12 ::ffff:127.0.0.1 [ecs-ec-ingress.ecs-ec-ingress] [abf115d4-9558-4aba-a0ef-b2cc4864ca73/SequentialEventDispatcher] com.q1labs.sem.monitors.SourceMonitor: [INFO] [NOT:0000006000][127.0.0.1/- -] [-/- -]Incoming raw event rate (5s: 0.40 eps), (10s: 0.70 eps), (15s: 23.33 eps), (30s: 12.13 eps), (60s: 6.78 eps), (300s: 6.67 eps), (900s: 6.67 eps). Peak in the last 60s: 68.60 eps. Max Seen 69.00 eps. EC Throttles/5s (60s: 0.25). Total EC Throttles in the last 60s: 3. Total EC Throttles: 55. License Threshold: 50.00

    so everythind looks good. Onle issue I see when comparing parameters is VM instance is running 

    4 vCPUs 
    16 GB 
    250 GB

    Only error I found in qradar.error was 

    Jan  8 10:12:52 ::ffff:127.0.0.1 [ecs-ec-ingress.ecs-ec-ingress] [4356c7f0-b517-4dbb-a07f-9d035dbb6e1f/SequentialEventDispatcher] com.ibm.si.ecingress.filters.QueuedEventThrottleFilter: [WARN] [NOT:0000004000][127.0.0.1/- -] [-/- -]The system is currently experiencing a spike in event rate exceeding the system's limits. If this spike is sustained, events will be dropped. On-disk queue stats: Event count=39 Utilization=0%

    which can be ignored. I would recommend to increase memory first before doing mor trouble shooting.



    ------------------------------
    [Karl] [Jaeger] [Business Partner]
    [QRadar Specialist]
    [pro4bizz]
    [Karlsruhe] [Germany]
    [4972190981722]
    ------------------------------

    Original Message


  • 6.  RE: Qradar CE_ecs_ec and ecs_ep services are in failed state

    Posted Sat January 13, 2024 04:23 AM
      |   view attached

    Dear Karl Jaeger,

    I appreciate your prompt response. Despite following your recommendation and increasing the virtual machine's memory, the issue persists. The ecs-ec and ecs-ep services are still failing to start.

    I also attempted to install Qradar CE using Oracle VM Virtual Box Manager, but unfortunately, the problem persists. I have attached the logs from my test machine for your review. I am new to Qradar, I am unsure how to analyze these logs to identify the root cause of the issue.

    Could you/someone please review the logs and provide guidance on resolving the problem? Your assistance is highly appreciated.

    Thank you.



    ------------------------------
    Qradarlearning SIEM
    ------------------------------

    Attachment(s)

    gz
    logs_localhost_20240113_34a8d1b1.tar.gz   3.11 MB 1 version
    Original Message