IBM Security QRadar

 View Only
Expand all | Collapse all

Apps stopped working after update 7.5.0UP8IF01 - docker replaced with podman..

  • 1.  Apps stopped working after update 7.5.0UP8IF01 - docker replaced with podman..

    IBM Champion
    Posted Fri April 19, 2024 05:46 AM

    Hi community,

    I'm struggling with the new "App-Framework" in our qradar-lab which comes with UP8. Instead of docker now podman is used to manage apps. qappmanager shows running and completed, but the apps are not running and no tab shows up in the console ui.

    The command podman ps on the apphost shows nothing. podman images shows all containers which worked with UP8 and before as expected. If I try to restart an app a message occurs saying that this container is missing in the registry.

    @IBM Support With docker there was an option with the deliver.sh push command to update the registry... How does this work with podman? How can I fix this issue, are you aware about this issue with apps which worked before and stopped working after applying UP8IF01?

    Thanks in advance for any advice or helpful hints to get the installed apps up and running again :)

    Ralph



    ------------------------------
    Ralph Belfiore
    Managing Consultant | Senior SIEM Expert
    connecT SYSTEMHAUS AG
    Siegen
    +491726365525
    ------------------------------


  • 2.  RE: Apps stopped working after update 7.5.0UP8IF01 - docker replaced with podman..

    Posted Sun April 21, 2024 04:02 AM

    Hello Ralph,

    Have you solved the issue?

    Thanks

    Ali



    ------------------------------
    Ali Mohamed
    ------------------------------



  • 3.  RE: Apps stopped working after update 7.5.0UP8IF01 - docker replaced with podman..

    IBM Champion
    Posted Tue April 23, 2024 03:39 PM

    Hi all,

    thanks for your useful and supporting comments. Currently i am working also with IBM Support to solve this issue. In our case it seems to be a little bit different to make progress, but finally the apps are up and running again :)

    Regards,

    Ralph



    ------------------------------
    Ralph Belfiore
    Managing Consultant | Senior SIEM Expert
    connecT SYSTEMHAUS AG
    Siegen
    +491726365525
    ------------------------------



  • 4.  RE: Apps stopped working after update 7.5.0UP8IF01 - docker replaced with podman..

    IBM Champion
    Posted Mon April 22, 2024 04:05 AM
    Edited by Pascal Weber Mon April 22, 2024 07:56 AM

    Hello my friend @Ralph Belfiore :)

    I hope you're doing well. 

    We encountered the same issue in UP8 after applying IF1 (and a few others as well)...

    Indeed, switching to podman makes a lot of the support tech notes obsolete, but the recon and qappmanager commands are still there.

    To resolve this registry issue, we proceeded as follows:

    • systemctl stop hostcontext
    • rm -rf /opt/qradar/ca/certs/* ; /opt/qradar/ca/bin/reset-qradar-ca.sh all --reset (This command take a long time, go to /var/log to see the update process)
    • systemctl start hostcontext
    • Unfortunately, you will have to re-import your certificates afterward.

    Of course, it's essential to test this on a test environment before making changes in production where you should be assisted by support via a ticket.

    Hope this help ;) 

    Regards,

    Pascal

    Alias zoldax

    https://github.com/zoldax



    ------------------------------
    zoldax

    https://www.credly.com/users/pascal-weber.029e134d/badges
    ------------------------------



  • 5.  RE: Apps stopped working after update 7.5.0UP8IF01 - docker replaced with podman..

    Posted 30 days ago

    Hello,

    How do you re-import certificates?



    ------------------------------
    Konstantin Kim
    ------------------------------



  • 6.  RE: Apps stopped working after update 7.5.0UP8IF01 - docker replaced with podman..

    Posted Mon April 22, 2024 07:23 AM

    Hi Ralph,

    Are you seeing anything in the logs like the below?

    Unable to read keystore podman-client-registry.p12

    What is the output of the following command on the console?

    keytool -list -v -storetype pkcs12 -keystore /etc/podman/tls/registry/podman-client-registry.p12 -storepass $(echo $(psql -U qradar -tAc "select token from application_credentials where name = 'podman-client-registry';") | java -jar /opt/qradar/jars/ibm-si-mks.jar decrypt_command_line 2>&1) 

    Thanks



    ------------------------------
    John Dawson
    Qradar Support Architect
    IBM
    ------------------------------



  • 7.  RE: Apps stopped working after update 7.5.0UP8IF01 - docker replaced with podman..

    Posted Tue April 23, 2024 08:19 AM

    Hi! in my case working with support we can resolve this issue with the following commands but we still working on some apps:

    On consola run the following command to copy the file:
    scp /etc/podman/tls/registry/podman-client-registry.p12 user@<apphostip>:/etc/podman/tls/registry/podman-client-registry.p12
     
    On the apphost run the following commands:
    systemctl stop hostcontext
    systemctl stop hostservices
    systemctl stop conman
    systemctl stop podman
    rm -rf /store/docker-data/engine
    systemctl start podman
    systemctl start conman
    systemctl start hostcontext
    systemctl start hostservices
    After that you can run the apps from qappmanger.
    Regards!


    ------------------------------
    Felipe Ortiz
    ------------------------------



  • 8.  RE: Apps stopped working after update 7.5.0UP8IF01 - docker replaced with podman..

    Posted 23 days ago

    Hi John.

      I have the same issue, my log conatines the 

    Unable to read keystore podman-client-registry.p12

    The keytool ... command says: 

    keytool error (likely untranslated): java.io.IOException: keystore password was incorrect

    Did you manage to find any solution?

    Thank you



    ------------------------------
    Gergely Csaba
    ------------------------------



  • 9.  RE: Apps stopped working after update 7.5.0UP8IF01 - docker replaced with podman..

    Posted 23 days ago

    Test if the podman-client-registry password works with this command:

    keytool -list -v -storetype pkcs12 -keystore /etc/podman/tls/registry/podman-client-registry.p12 -storepass $(echo $(psql -U qradar -tAc "select token from application_credentials where name = 'podman-client-registry';") | java -jar /opt/qradar/jars/ibm-si-mks.jar decrypt_command_line 2>&1) 

    Failed password will look like:

    keytool error (likely untranslated): java.io.IOException: keystore password was incorrect 

    #regenerate password for podman-client-registry
    /opt/qradar/bin/runjava.sh com.ibm.si.application.commandline.KeyStoreGenerator -c /etc/podman/tls/registry/podman-client-registry.cert -k /etc/podman/tls/registry/podman-client-registry.key -s /etc/podman/tls/registry/podman-client-registry.p12

    Please do not attempt this workaround without verifying the password does not work first through the keytool command.

    #regenerate password for podman-client-registry
    /opt/qradar/bin/runjava.sh com.ibm.si.application.commandline.KeyStoreGenerator -c /etc/podman/tls/registry/podman-client-registry.cert -k /etc/podman/tls/registry/podman-client-registry.key -s /etc/podman/tls/registry/podman-client-registry.p12

    Please do not attempt this workaround without verifying the password does not work first through the keytool command.

    Thanks



    ------------------------------
    John Dawson
    Qradar Support Architect
    IBM
    ------------------------------



  • 10.  RE: Apps stopped working after update 7.5.0UP8IF01 - docker replaced with podman..

    Posted 21 days ago

    Thank You John!

    It worked! To be honest I tried some of what the previous poster wrote and mixed in some of the old cert regeneration support page told me.

    systemctl stop hostcontext
    rm -rf /opt/qradar/ca/certs/*
    /opt/qradar/ca/bin/reset-qradar-ca.sh all --reset
    systemctl stop tomcat hostservices;
    rm -rf /opt/tomcat/work/Catalina
    systemctl restart conman traefik podman
    systemctl daemon-reload
    systemctl start hostservices tomcat hostcontext
    $(locate wait_for_start.sh | tail -1)

    I did the above and it worked. Obviously your solution is shorter, better and more elegant.

    Thanks again.



    ------------------------------
    Gergely Csaba
    ------------------------------