IBM QRadar SOAR

 View Only

  • 1.  [Errno 13] Permission denied when deploying custom application

    Posted 19 days ago
    Hello SOAR communities.
     
    I'm stuck with the deploying my custom application on the following environments:
       - SOAR : 51.0.3.2.8
       - AppHost: 1.15.3.1
     
    Due to Permission error of /opt/app-root/lib/python3.11/site-packages/<my-app>-<ver>.dist-info, the pod goes to Error state.
    By viewing the pod instantly, the file permission of <my-app>-<ver>.dist-info directory was drwxr-x---. while the other directories on the same path have drwxr-xr-x.
     
    I have no idea why the directory lacks the permission of others.
    The details are as follows: Could someone give me advices?

    --------------------------

    <Problem>
     
    - TEST with app.config
     
    The last test failed
    Error: [Errno 13] Permission denied: '/opt/app-root/lib/python3.11/site-packages/fn_shape-1.0.0.dist-info' Details
    Traceback (most recent call last):
      File "/opt/app-root/bin/resilient-circuits", line 5, in <module>
        from resilient_circuits.bin.resilient_circuits_cmd import main
      File "/opt/app-root/lib64/python3.11/site-packages/resilient_circuits/__init__.py", line 12, in <module>
        from .actions_component import ResilientComponent
      File "/opt/app-root/lib64/python3.11/site-packages/resilient_circuits/actions_component.py", line 19, in <module>
        import resilient
          :
          : 
      File "/opt/app-root/lib64/python3.11/site-packages/pkg_resources/__init__.py", line 641, in __init__
        self.add_entry(entry)
      File "/opt/app-root/lib64/python3.11/site-packages/pkg_resources/__init__.py", line 697, in add_entry
        for dist in find_distributions(entry, True):
      File "/opt/app-root/lib64/python3.11/site-packages/pkg_resources/__init__.py", line 2309, in find_on_path
        yield from factory(fullpath)
      File "/opt/app-root/lib64/python3.11/site-packages/pkg_resources/__init__.py", line 2366, in distributions_from_metadata
        if len(os.listdir(path)) == 0:
               ^^^^^^^^^^^^^^^^
    PermissionError: [Errno 13] Permission denied: '/opt/app-root/lib/python3.11/site-packages/fn_shape-1.0.0.dist-info'
     
    - Deploying fn_shape app
     
    [root@apphost util]# kubectl -n c72e0903-e721-4205-bccf-c1371c212879 logs eff7d0eb-d979-4f4c-9384-85f0a231c211-56fc6b6b9b-ggk2r
    Defaulted container "eff7d0eb-d979-4f4c-9384-85f0a231c211" out of: eff7d0eb-d979-4f4c-9384-85f0a231c211, protected-secrets-provision (init)
    Traceback (most recent call last):
      File "/opt/app-root/bin/resilient-circuits", line 5, in <module>
        from resilient_circuits.bin.resilient_circuits_cmd import main
      File "/opt/app-root/lib64/python3.11/site-packages/resilient_circuits/__init__.py", line 12, in <module>
        from .actions_component import ResilientComponent
      File "/opt/app-root/lib64/python3.11/site-packages/resilient_circuits/actions_component.py", line 19, in <module>
        import resilient
      File "/opt/app-root/lib64/python3.11/site-packages/resilient/__init__.py", line 13, in <module>
        from .co3 import SimpleClient, \
      File "/opt/app-root/lib64/python3.11/site-packages/resilient/co3.py", line 17, in <module>
        from . import co3base
      File "/opt/app-root/lib64/python3.11/site-packages/resilient/co3base.py", line 24, in <module>
        from resilient import constants, helpers
      File "/opt/app-root/lib64/python3.11/site-packages/resilient/constants.py", line 6, in <module>
        import pkg_resources
      File "/opt/app-root/lib64/python3.11/site-packages/pkg_resources/__init__.py", line 3624, in <module>
        @_call_aside
         ^^^^^^^^^^^
      File "/opt/app-root/lib64/python3.11/site-packages/pkg_resources/__init__.py", line 3608, in _call_aside
        f(*args, **kwargs)
      File "/opt/app-root/lib64/python3.11/site-packages/pkg_resources/__init__.py", line 3637, in _initialize_master_working_set
        working_set = _declare_state('object', 'working_set', WorkingSet._build_master())
                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/opt/app-root/lib64/python3.11/site-packages/pkg_resources/__init__.py", line 648, in _build_master
        ws = cls()
             ^^^^^
      File "/opt/app-root/lib64/python3.11/site-packages/pkg_resources/__init__.py", line 641, in __init__
        self.add_entry(entry)
      File "/opt/app-root/lib64/python3.11/site-packages/pkg_resources/__init__.py", line 697, in add_entry
        for dist in find_distributions(entry, True):
      File "/opt/app-root/lib64/python3.11/site-packages/pkg_resources/__init__.py", line 2309, in find_on_path
        yield from factory(fullpath)
      File "/opt/app-root/lib64/python3.11/site-packages/pkg_resources/__init__.py", line 2366, in distributions_from_metadata
        if len(os.listdir(path)) == 0:
               ^^^^^^^^^^^^^^^^
    PermissionError: [Errno 13] Permission denied: '/opt/app-root/lib/python3.11/site-packages/fn_shape-1.0.0.dist-info'
     
    ===
    <Investigation>
    By viewing the pod by 'kubectl -n c72e0903-e721-4205-bccf-c1371c212879 exec -it eff7d0eb-d979-4f4c-9384-85f0a231c211-56fc6b6b9b-ggk2r -- bash'
     
    The '/opt/app-root/lib/python3.11/site-packages/' directory has the following entries:
     
    :
    drwxr-xr-x.  2 root root     126 Nov  5 05:14 keyring-23.13.1.dist-info
    drwxr-xr-x.  6 root root    4096 Nov  5 05:14 keyring
    drwxr-xr-x.  2 root root     102 Nov  5 05:14 jwcrypto-1.5.6.dist-info
    drwxr-xr-x.  3 root root     172 Nov  5 05:14 jwcrypto
    drwxr-xr-x.  3 root root      87 Nov  5 05:14 cryptography-43.0.3.dist-info
    drwxr-xr-x.  5 root root     160 Nov  5 05:14 cryptography
    drwxr-xr-x.  3 root root      44 Nov  5 05:14 co3
    drwxr-xr-x.  5 root root     160 Nov  5 05:14 bs4
    drwxr-xr-x.  3 root root      82 Nov  5 05:14 beautifulsoup4-4.12.3.dist-info
    drwxr-x---.  2 root root     151 Nov  5 06:47 fn_shape-1.0.0.dist-info
    drwxr-x---.  5 root root      89 Nov  5 06:47 fn_shape
    (app-root) bash-5.1$


    ------------------------------
    Yohji Amano
    ------------------------------


  • 2.  RE: [Errno 13] Permission denied when deploying custom application

    Posted 17 days ago

    Hi Yohji

    I am looking in the container of the fn_sumo_logic app I worked on and I see the permissions like this:

    (app-root) bash-5.1$ ls -la /opt/app-root/lib/python3.11/site-packages/
    total 1376
    drwxrwxr-x  1 default root      64 Oct  3 21:11 .
    drwxrwxr-x  1 default root      27 Sep 18 13:06 ..
    drwxr-xr-x  3 root    root      82 Oct  1 05:14 beautifulsoup4-4.12.3.dist-info
    drwxr-xr-x  5 root    root     160 Oct  1 05:14 bs4
    drwxr-xr-x  3 root    root      74 Oct  1 05:14 cachetools
    drwxr-xr-x  2 root    root     102 Oct  1 05:14 cachetools-5.5.0.dist-info
    drwxr-xr-x  3 root    root     112 Oct  1 05:14 certifi
    drwxr-xr-x  2 root    root     102 Oct  1 05:14 certifi-2024.8.30.dist-info
    drwxr-xr-x  3 root    root    4096 Oct  1 05:14 cffi
    drwxr-xr-x  2 root    root     126 Oct  1 05:14 cffi-1.17.1.dist-info
    -rwxr-xr-x  1 root    root 1068624 Oct  1 05:14 _cffi_backend.cpython-311-x86_64-linux-gnu.so
    drwxr-xr-x  4 root    root    4096 Oct  1 05:14 charset_normalizer
    drwxr-xr-x  2 root    root     126 Oct  1 05:14 charset_normalizer-3.3.2.dist-info
    drwxr-xr-x 11 root    root     192 Oct  1 05:14 circuits
    drwxr-xr-x  2 root    root     142 Oct  1 05:14 circuits-3.2.3.dist-info
    drwxr-xr-x  3 root    root      44 Oct  1 05:14 co3
    drwxr-xr-x  5 root    root     160 Oct  1 05:14 cryptography
    drwxr-xr-x  3 root    root      87 Oct  1 05:14 cryptography-43.0.1.dist-info
    drwxr-xr-x  2 root    root     122 Oct  1 05:14 decorator-5.1.1.dist-info
    -rw-r--r--  1 root    root   16752 Oct  1 05:14 decorator.py
    drwxr-xr-x  3 root    root      79 Oct  1 05:14 deprecated
    drwxr-xr-x  2 root    root     106 Oct  1 05:14 Deprecated-1.2.14.dist-info
    drwxr-xr-x  3 root    root      63 Oct  1 05:14 _distutils_hack
    -rw-r--r--  1 root    root     151 Oct  1 05:14 distutils-precedence.pth
    drwxr-xr-x  2 root    root     106 Oct  1 05:14 docopt-0.6.2.dist-info
    -rw-r--r--  1 root    root   19946 Oct  1 05:14 docopt.py
    drwxr-xr-x  3 root    root    4096 Oct  1 05:14 filelock
    drwxr-xr-x  3 root    root      82 Oct  1 05:14 filelock-3.16.1.dist-info
    drwxr-xr-x  7 root    root     114 Oct  3 21:11 fn_sumo_logic
    drwxr-xr-x  2 root    root     151 Oct  3 21:11 fn_sumo_logic-1.0.0.dist-info

    How are you building your container?

    The Sumo Logic app Dockerfile looks like this https://github.com/ibmresilient/resilient-community-apps/blob/main/fn_sumo_logic/Dockerfile



    ------------------------------
    AnnMarie Norcross
    ------------------------------

    Original Message


  • 3.  RE: [Errno 13] Permission denied when deploying custom application

    Posted 17 days ago

    Hi AnnMarie.

    Thanks for your reply. There was some progress.

    Through some trials. I found that I had to modify the Dockerfile generated from resilient-sdk codegen.

    :

    < original Dockerfile>
       :
    RUN pip install /tmp/packages/${APPLICATION}-*.tar.gz
     
    USER 1001
      
    <Dockerfile after retouch>
     :
    RUN pip install /tmp/packages/${APPLICATION}-*.tar.gz
     
    RUN chmod -R 755 /opt/app-root/lib/python3.11/site-packages/
    #USER 1001
    I could run the application pod on the AppHost with the above modification though still failing app.config test.

    I wonder someone can run the application pod on with the default Dockerfile when resilient-sdk is 51.0.2.2.1096(latest).

    resilient                    51.0.2.2.1096
    resilient-app-config-plugins 1.0.0
    resilient_circuits           51.0.2.2.1096
    resilient_lib                51.0.2.2.1096
    resilient_sdk                51.0.2.2.1096



    ------------------------------
    Yohji Amano
    ------------------------------

    Original Message


  • 4.  RE: [Errno 13] Permission denied when deploying custom application

    Posted 17 days ago

    Further update.

    With regard to failure of app.config test,  it was resolved by modifying selftest.py.

    In summary Dockerfile generated from resilient-sdk codegen(51.0.2.2.1096) is needed to replace the last line(USER 1001) with the following:

    RUN chmod -R 755 /opt/app-root/lib/python*/site-packages/
    #USER 1001

    I still wonder whether or not the above will affect other application in general.



    ------------------------------
    Yohji Amano
    ------------------------------

    Original Message


  • 5.  RE: [Errno 13] Permission denied when deploying custom application

    Posted 16 days ago

    It is a security risk to run the container as root (USER 0), which is probably what happens when commenting out the line 

    #USER 1001

    I do not know the effect of changing all files in the site-packages directory.

    Need to figure out why the permission on the files is not correct.  How are you creating the app .tar.gz file?  If you unpack it what do the file permissions look like?



    ------------------------------
    AnnMarie Norcross
    ------------------------------

    Original Message


  • 6.  RE: [Errno 13] Permission denied when deploying custom application

    Posted 16 days ago

    Hi AnnMarie

    Thank you for your advice. 

    I confirmed that I could run the app POD on App host with USER 1001 though I still need the directive of "RUN chmod -R 755 /opt/app-root/lib/python*/site-packages/". 

    The reason why I commented "USER 1001" was the container test failure under "podman run", which was the restriction under the local container test.

    With regard to the app .tar.gz, other group permission was NOT set.

    $ tar -tvf fn_shape2-1.0.0.tar.gz | awk '{print $1}' | sort -u
    -rw-r-----
    -rw-r--r--
    drwxr-x---
    $

    I tried with umask 0022 but in vein.

      



    ------------------------------
    Yohji Amano
    ------------------------------

    Original Message


  • 7.  RE: [Errno 13] Permission denied when deploying custom application

    Posted 15 days ago

    Hi Yohji

    When I run that command on my sumo logic .tar.gz, I have different permissions.  Can you change the permissions on the file before creating the tar.gz file?

     $ tar -tvf fn_sumo_logic-1.0.0.tar.gz| awk '{print $1}' | sort -u
-rw-r--r--
-rw-rw-r--
drwxrwxr-x


    ------------------------------
    AnnMarie Norcross
    ------------------------------

    Original Message


  • 8.  RE: [Errno 13] Permission denied when deploying custom application

    Posted 13 days ago

    Hi AnnMarie

    Thank you for your supports. I think I reach the cause and have some ideas how to do with it.

    SOAR custom application container image implicitly requires read/execute permissions for directories to others and also read permission for files to others related to the developing application.
    The root cause was my SOAR application development environment did not satisfy them.
     
    It's a bit humiliating, but I did not notice what security constraints are applied to it. 
     
    causes) In My system permissions to others were disabled in two ways:
     a) umask (0027)
     b) default acl(default:user::rwx, default:group::r-x, default:other::---)
     
    actions)
     
    To resolve it, there are two ways:
     1) change umask with 0022 and remove configured acls (action to causes)
     2) keep the existing security constraints and modify Dockerfile as follows:
     
    --- BOF ---
    ARG BASE_IMAGE_REPO=quay.io
    ARG BASE_IMAGE_TAG=latest
     
    FROM ${BASE_IMAGE_REPO}/ibmresilient/soarapps-base-docker-image:${BASE_IMAGE_TAG}
     
    ARG APPLICATION=fn_shape
    ARG VERSION=1.0.1
     
    COPY ./dist /tmp/packages
    USER 0
    RUN yum -y update && yum clean all
    RUN pip install /tmp/packages/${APPLICATION}-*.tar.gz
     
    RUN chmod 755 /opt/app-root/lib/python*/site-packages/${APPLICATION}-${VERSION}.dist-info
    RUN chmod 644 /opt/app-root/lib/python*/site-packages/${APPLICATION}-${VERSION}.dist-info/*
    RUN find /opt/app-root/lib/python*/site-packages/${APPLICATION} -type d | xargs chmod 755
    RUN find /opt/app-root/lib/python*/site-packages/${APPLICATION} -type f | xargs chmod 644
     
    USER 1001
    --- EOF ---
     
    I confirmed that both 1) and 2) work OK.    



    ------------------------------
    Yohji Amano
    ------------------------------

    Original Message