IBM Security Resilient

Expand all | Collapse all

Resilient - Skipping retry of any failed messages because STOMP connection is down

  • 1.  Resilient - Skipping retry of any failed messages because STOMP connection is down

    Posted Fri March 13, 2020 02:01 PM
    Hi,

    i'm using IBM resilient V36 with multiple organizations (MSSP), i'm using multiple instances of resilient circuits using systemd.

    When running resilient-circuits run command , i got the following errors :


    <Cannot handle command: ERROR [expected=CONNECTED, headers={u'message': u'User name [asabri@dataprotect.
    ma] or password is invalid.', u'content-type': u'text/plain'}]>

    The password i have in the app.config is correct,  it's the same password i'm using in the UI.

    When i start the service everything is OK, but after 4 or 5 minutes i got this error. ( and this not the case for all the instances, i have 2 instances that works perfectly)

    I runned this command to check the listening ports :

    -bash-4.2$ sudo netstat -plont | grep -E "65000|443|65001"
    tcp6 0 0 :::443 :::* LISTEN 2306/jsvc.exec off (0.00/0/0)
    tcp6 0 0 :::65000 :::* LISTEN 2243/jsvc.exec off (0.00/0/0)
    tcp6 0 0 :::65001 :::* LISTEN 2243/jsvc.exec off (0.00/0/0)

    Please find attached my app.conf and app.log for more details.

    can you help solving this issue:



    ------------------------------
    Ayman Sabri Cyber Security Analyst II
    ------------------------------

    Attachment(s)

    txt
    appconf.txt   3K 1 version
    txt
    applog.txt   22K 1 version


  • 2.  RE: Resilient - Skipping retry of any failed messages because STOMP connection is down

    Posted Sun March 15, 2020 01:00 PM
    Hi,

    I tried to install resilient in my lab and do the same config , i didn't have the same problem ; i guess it's related to the network , but all the scripts are running localy, how can i troubleshot the problem ?

    Thank you

    ------------------------------
    Ayman Sabri Cyber Security Analyst II
    ------------------------------



  • 3.  RE: Resilient - Skipping retry of any failed messages because STOMP connection is down

    Posted Mon March 16, 2020 12:44 PM
    If this error is coming from the Resilient server I suggest looking in the client.log file on the Resilient server (typically /usr/share/co3/logs/client.log).

    ------------------------------
    Ben Lurie
    ------------------------------



  • 4.  RE: Resilient - Skipping retry of any failed messages because STOMP connection is down

    Posted Tue March 17, 2020 07:07 AM
    Hi,

    I think the problem is when running multiple instances(30 instance of resilient circuits) at the same time using the same port 65001 the stomp goes down.

    If i used a different port for each instance ( ex: 65001, 65002, 65003, ) in each app.conf, is this possible to avoid the stomp issue ( Skipping retry of any failed messages because STOMP connection is down)?

    Thank you

    ------------------------------
    Ayman Sabri Cyber Security Analyst II
    ------------------------------



  • 5.  RE: Resilient - Skipping retry of any failed messages because STOMP connection is down

    Posted Tue March 17, 2020 07:31 AM
    Do you mean changing this?

    # Actions Module connection
    #stomp_port=65001

    If so, that would not make sense to me because the Resilient server is just listening on port 65001 for all organizations. My understanding is that tells resilient circuits to connect to the resilient server on port 65001. So if that is changed to port 65002 then it would not be able to connect to Resilient because there is nothing on the Resilient side listening on port 65001.

    If you have checked on the Resilient client.log and there are no errors you may need to open a support ticket.

    Ben

    ------------------------------
    Ben Lurie
    ------------------------------



  • 6.  RE: Resilient - Skipping retry of any failed messages because STOMP connection is down

    Posted Tue March 24, 2020 11:05 AM

    Hi,

    Are you using the same USER/Password in the app config for ALL 30 Orgs ? 

    I was wondering if this could not be the case of conflicting session opening in Resilient, if some integrations are connecting in RestAPI to Resilient, using the app.config user information.
    If so you will see session open / close in the /usr/share/co3/logs/client.log



    ------------------------------
    BENOIT ROSTAGNI
    ------------------------------



  • 7.  RE: Resilient - Skipping retry of any failed messages because STOMP connection is down

    Posted Wed March 25, 2020 05:41 AM
    Using the same account for each instance is not practical. Please use unique user accounts (API accounts are not supported with MSSP yet) for each of the instances. Resilient does not allow multiple sessions for the same user. I have requested clarification in our documentation to make this clear in instances such as this, that is, unique accounts should be used.

    ------------------------------
    BEN WILLIAMS
    ------------------------------