Scott, I just wanted to report some interesting findings. DSC and runtime ran fine with the OOTB SCC. However, the WRP container did not. Again this is OpenShift 3.11. No big deal right now since I still have to have the SCC for the config container. I just wanted to pass this along.
Manifest:
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: isam-wrp-test
labels:
app: isam-wrp-test
spec:
selector:
matchLabels:
app: isam-wrp-test
replicas: 1
podManagementPolicy: Parallel
template:
metadata:
labels:
app: isam-wrp-test
spec:
# serviceAccountName: isam
# securityContext:
# runAsNonRoot: true
# runAsUser: 6000
containers:
- name: isam-wrp-test
image: verify-access-wrp:build
ports:
- containerPort: 9443
env:
- name: SERVICE
value: webseal
- name: INSTANCE
valueFrom:
configMapKeyRef:
name: isam-wrp-instances
key: low
- name: CONFIG_SERVICE_URL
valueFrom:
configMapKeyRef:
name: isam-config
key: config-service-url
- name: CONFIG_SERVICE_USER_NAME
valueFrom:
secretKeyRef:
name: isam-creds-cfgsvc
key: username
- name: CONFIG_SERVICE_USER_PWD
valueFrom:
secretKeyRef:
name: isam-creds-cfgsvc
key: password
- name: ADMIN_PWD
valueFrom:
secretKeyRef:
name: isam-creds-admin
key: password
- name: CONTAINER_LOG_ID
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: CONTAINER_TIMEZONE
valueFrom:
configMapKeyRef:
name: isam-config
key: timezone
- name: SNAPSHOT_ID
valueFrom:
configMapKeyRef:
name: isam-config
key: snapshot_id
- name: FIXPACKS
valueFrom:
configMapKeyRef:
name: isam-environment
key: fixpacks
livenessProbe:
exec:
command:
- /sbin/health_check.sh
- livenessProbe
initialDelaySeconds: 10
periodSeconds: 10
readinessProbe:
exec:
command:
- /sbin/health_check.sh
initialDelaySeconds: 10
periodSeconds: 10
...
Console logs:
{"instant":{"epochSecond":1658173311},"threadId":"1","level":"INFO","loggerName":"system","component":"bootstrap","source":{"file":"/sbin/bootstrap.sh"}, "content":"WGAWA0960I Downloading data from the configuration service."}
{"instant":{"epochSecond":1658173312},"threadId":"1","level":"INFO","loggerName":"system","component":"bootstrap","source":{"file":"/sbin/bootstrap.sh"}, "content":"WGAWA0962I Data has been downloaded from the configuration service."}
{"instant":{"epochSecond":1658173312},"threadId":"1","level":"INFO","loggerName":"system","component":"bootstrap","source":{"file":"/sbin/bootstrap.sh"}, "content":"WGAWA0971I Applying the configuration snapshot: 7446c4780e34e060be0f038316c609eeb114daf7998d53f9d92204d92f3c8f59"}
warning: cannot set UID 6000 and/or GID 0 for /usr/lib64/iss-pam/categoryxref.csv
Operation not permitted
warning: cannot set UID 6000 and/or GID 0 for /usr/lib64/iss-pam/checkdates.csv
Operation not permitted
warning: cannot set UID 6000 and/or GID 0 for /usr/lib64/iss-pam/cvexref.csv
Operation not permitted
warning: cannot set UID 6000 and/or GID 0 for /usr/lib64/iss-pam/featureCategory.xml
Operation not permitted
warning: cannot set UID 6000 and/or GID 0 for /usr/lib64/iss-pam/iss-pam1.so
Operation not permitted
warning: cannot set UID 6000 and/or GID 0 for /usr/lib64/iss-pam/issue_blocks.csv
Operation not permitted
warning: cannot set UID 6000 and/or GID 0 for /usr/lib64/iss-pam/issue_blocks2.csv
Operation not permitted
warning: cannot set UID 6000 and/or GID 0 for /usr/lib64/iss-pam/issue_icmp_blocks.csv
Operation not permitted
warning: cannot set UID 6000 and/or GID 0 for /usr/lib64/iss-pam/issue_icmp_blocks2.csv
Operation not permitted
warning: cannot set UID 6000 and/or GID 0 for /usr/lib64/iss-pam/issue_responses2.csv
Operation not permitted
warning: cannot set UID 6000 and/or GID 0 for /usr/lib64/iss-pam/issues.csv
Operation not permitted
warning: cannot set UID 6000 and/or GID 0 for /usr/lib64/iss-pam/pamfscheck.so
Operation not permitted
warning: cannot set UID 6000 and/or GID 0 for /usr/lib64/iss-pam/pamopensigchk.so
Operation not permitted
warning: cannot set UID 6000 and/or GID 0 for /usr/lib64/iss-pam/protocols.csv
Operation not permitted
warning: cannot set UID 6000 and/or GID 0 for /usr/lib64/iss-pam/safe.so
Operation not permitted
warning: cannot set UID 6000 and/or GID 0 for /usr/lib64/iss-pam/stats.csv
Operation not permitted
warning: cannot set UID 6000 and/or GID 0 for /usr/lib64/iss-pam/stats_tags.csv
Operation not permitted
warning: cannot set UID 6000 and/or GID 0 for /usr/lib64/iss-pam/user-defined.csv
Operation not permitted
{"instant":{"epochSecond":1658173315},"threadId":"1","level":"INFO","loggerName":"system","component":"bootstrap","source":{"file":"/sbin/bootstrap.sh"}, "content":"WGAWA0966E The fixpack file, IJ36093_10031.fixpack, is not currently available from the configuration service!"}
{"instant":{"epochSecond":1658173315},"threadId":"1","level":"INFO","loggerName":"system","component":"bootstrap","source":{"file":"/sbin/bootstrap.sh"}, "content":"WGAWA0966E The fixpack file, IJ38236_10031.fixpack, is not currently available from the configuration service!"}
{"instant":{"epochSecond":1658173316},"threadId":"1","level":"INFO","loggerName":"system","component":"bootstrap","source":{"file":"/sbin/bootstrap.sh"}, "content":"WGAWA0966E The fixpack file, remove_log4j_v1.fixpack, is not currently available from the configuration service!"}
{"instant":{"epochSecond":1658173316},"threadId":"1","level":"INFO","loggerName":"system","component":"bootstrap","source":{"file":"/sbin/bootstrap.sh"}, "content":"WGAWA0967I Applying the fixpack: IJ36093_10031.fixpack."}
{"instant":{"epochSecond":1658173316},"threadId":"1","level":"INFO","loggerName":"system","component":"bootstrap","source":{"file":"/sbin/bootstrap.sh"}, "content":"WGAWA0968E Failed to apply the fixpack: IJ36093_10031.fixpack."}
{"instant":{"epochSecond":1658173316},"threadId":"1","level":"INFO","loggerName":"system","component":"bootstrap","source":{"file":"/sbin/bootstrap.sh"}, "content":"WGAWA0967I Applying the fixpack: IJ38236_10031.fixpack."}
{"instant":{"epochSecond":1658173316},"threadId":"1","level":"INFO","loggerName":"system","component":"bootstrap","source":{"file":"/sbin/bootstrap.sh"}, "content":"WGAWA0968E Failed to apply the fixpack: IJ38236_10031.fixpack."}
{"instant":{"epochSecond":1658173316},"threadId":"1","level":"INFO","loggerName":"system","component":"bootstrap","source":{"file":"/sbin/bootstrap.sh"}, "content":"WGAWA0967I Applying the fixpack: remove_log4j_v1.fixpack."}
{"instant":{"epochSecond":1658173316},"threadId":"1","level":"INFO","loggerName":"system","component":"bootstrap","source":{"file":"/sbin/bootstrap.sh"}, "content":"WGAWA0968E Failed to apply the fixpack: remove_log4j_v1.fixpack."}
{"instant":{"epochSecond":1658173316},"threadId":"1","level":"INFO","loggerName":"system","component":"bootstrap","source":{"file":"/sbin/bootstrap.sh"}, "content":"WGAWA0964I Starting the OpenLDAP Server."}
{"instant":{"epochSecond":1658173317},"threadId":"1","level":"INFO","loggerName":"system","component":"bootstrap","source":{"file":"/sbin/bootstrap.sh"}, "content":"WGAWA0965I Starting the WRP Server."}
whoami: cannot find name for user ID 1005240000
USAGE: /opt/PolicyDirector/sbin/pdconf setentry [-obfuscate] <stanza> <entry> <value>
{"instant":{"epochSecond":1658173318},"threadId":"0x369298c0","level":"INFO","loggerName":"message","content":{"product":"IBM Security Verify Access: Web Reverse Proxy", "version":"10.0.4.0 (Build 20220608_0037)","copyright":"Copyright (C) IBM Corporation 1994-2022. All Rights Reserved."}}
{"instant":{"epochSecond":1658173319},"threadId":"0x7fa9369298c0","level":"ERROR","loggerName":"webseald","component":"wwa.isv","message_id":"0x38CF07D8","source":{"file":"HTTPAuxHeaders.cpp","line":166}, "content":"DPWWA2008E Error reading stanza '[header-names]': ienv"}
{"instant":{"epochSecond":1658173319},"threadId":"0x7fa9369298c0","level":"WARNING","loggerName":"webseald","component":"wwa.server","message_id":"0x38CF0156","source":{"file":"config.cpp","line":6205}, "content":"DPWWA0342W The configuration data for this WebSEAL instance has been logged in '\/var\/pdweb\/test.acme.org-dca\/log\/config_data__test.acme.org-dca-webseald-testing_blue.log'"}
{"instant":{"epochSecond":1658173319},"threadId":"0x7fa9369298c0","level":"ERROR","loggerName":"webseald","component":"ivc.general","message_id":"0x1354A0A9","source":{"file":"IVServer.cpp","line":1384}, "content":"HPDCO0169E Could not change process GID ( Failed to set supplementary groups)"}
------------------------------
Matt Jenkins
------------------------------
Original Message:
Sent: Sun July 17, 2022 04:06 PM
From: Scott Exton
Subject: ISVA container on Kubernetes / OpenShift - Is security context and running as specific user needed for new lightweight containers?
Matt,
It is only the lightweight containers (i.e. wrp, runtime, dsc) which are able to run with the OOTB security constraint. Unfortunately the configuration container still needs the 'special' security constraint. The good news is that the configuration container does not need to be executed in the same environment as the runtime. You can even use a local Docker environment to run the configuration container and then just transfer your configuration snapshot to the runtime environment.
I hope that this helps.
Scott A. Exton
Senior Software Engineer
Chief Programmer - IBM Security Verify Access
IBM Master Inventor
Original Message:
Sent: 7/15/2022 6:26:00 PM
From: Matt Jenkins
Subject: RE: ISVA container on Kubernetes / OpenShift - Is security context and running as specific user needed for new lightweight containers?
@Scott Exton I am coming back around to testing this. I removed the serviceAccountName and securityContext from the verify-access:10.0.4.0 image. However, it will not start, it throws all sorts of errors. This is on a brand new "out of the project" new project on OpenShift 3.11. I did not apply the security context to this project.
Are we able to run without the security context being applied at all? Or do I have something wrong here? I was hoping I could run without the SCC as it would greatly help me and my colleagues build labs without the aid of the OpenShift cluster administrator team to apply the SCC. I'll paste the manifest I used and then the logs from the console below. Thanks!
---apiVersion: apps/v1kind: StatefulSetmetadata: name: isam-config labels: app: isam-configspec: selector: matchLabels: app: isam-config replicas: 1 podManagementPolicy: Parallel template: metadata: labels: app: isam-config spec:# serviceAccountName: isam# securityContext:# runAsNonRoot: true# runAsUser: 6000 volumes: - name: isam-shared persistentVolumeClaim: claimName: isam-shared-pvc - name: isam-logs persistentVolumeClaim: claimName: isam-logs-pvc containers: - name: isam-config image: verify-access:build ports: - containerPort: 9443 env: - name: SERVICE value: config - name: ADMIN_PWD valueFrom: secretKeyRef: name: isam-creds-admin key: password - name: CONTAINER_LOG_ID valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.name - name: CONTAINER_TIMEZONE valueFrom: configMapKeyRef: name: isam-config key: timezone - name: SNAPSHOT_ID valueFrom: configMapKeyRef: name: isam-config key: snapshot_id - name: FIXPACKS valueFrom: configMapKeyRef: name: isam-environment key: fixpacks readinessProbe: exec: command: - /sbin/health_check.sh initialDelaySeconds: 15 periodSeconds: 20 timeoutSeconds: 3 successThreshold: 3 failureThreshold: 20 livenessProbe: exec: command: - /sbin/health_check.sh - livenessProbe timeoutSeconds: 1 initialDelaySeconds: 120 periodSeconds: 20 successThreshold: 1 failureThreshold: 9 volumeMounts: - mountPath: /var/shared name: isam-shared - mountPath: /var/application.logs name: isam-logs...
The console shows:2022-07-15T22:14:34+00:00: WGAWA0989I Bootstrapping.
Error: WGAWA0004E The data which was passed into the program is not valid: clean_files.sh
Error: WGAWA0004E The data which was passed into the program is not valid: set_admin_password.sh
Error: WGAWA0004E The data which was passed into the program is not valid: init_logging.sh
Error: WGAWA0004E The data which was passed into the program is not valid: init_paths.sh
Error: WGAWA0004E The data which was passed into the program is not valid: start_daemons.sh
/sbin/bootstrap.sh: line 348: 18 Aborted /usr/sbin/mesa_syslogd
ls: cannot access '/var/shared/extensions': No such file or directory
sh: line 0: cd: /var/shared/snapshots: No such file or directory
sh: line 0: cd: /var/shared/snapshots: No such file or directory
Error: WGAWA0004E The data which was passed into the program is not valid: init_wrp_log_paths.sh
Error: WGAWA0004E The data which was passed into the program is not valid: update_hostname_policy.sh
Error: WGAWA0004E The data which was passed into the program is not valid: policy_server_logging.sh
ls: cannot access '/var/shared/fixpacks': No such file or directory
sed: couldn't open temporary file /etc/pam.d/sedkycGVQ: Permission denied
sed: couldn't open temporary file /etc/pam.d/sedV4ZZvX: Permission denied
DPWAP0130E The username for the logged in UID could not be determined
Error: WGAWA0010E An internal error has occurred (/workspace/src/appliance/app-core/modules/config_isam_ec2.cpp:579).
Module syslog failed to commit. Ignoring the error and continuing to boot the appliance. Check the system log for further details.
/bin/sh: /opt/IBM/wlp/usr/servers/default/certs/lmi-sso.key: Permission denied
CTGSK3026W The key file "/opt/IBM/wlp/usr/servers/default/certs/lmi.p12" does not exist or cannot be read.
CTGSK2101W The key database does not exist.
CTGSK3026W The key file "/opt/IBM/wlp/usr/servers/default/certs/lmi.p12" does not exist or cannot be read.
CTGSK2101W The key database does not exist.
Error: WGAWA0009E The file, lmi_trust_store.p12, does not exist.
chown: changing ownership of '/opt/IBM/wlp/usr/shared/config/host.properties': Operation not permitted
chown: cannot access '/opt/IBM/wlp/usr/servers/default/jvm.options': Permission denied
chmod: cannot access '/opt/IBM/wlp/usr/servers/default/jvm.options': Permission denied
WGAWA0009E The file, /opt/IBM/wlp/usr/servers/default/server.xml.bak, does not exist.
chown: changing ownership of '/opt/ibm/wlp/etc/liberty.cache': Operation not permitted
chmod: changing permissions of '/opt/ibm/wlp/etc/liberty.cache': Operation not permitted
CTGSK3036W The output file "/opt/IBM/wlp/usr/servers/default/certs/lmi.p12" already exists.
CTGSK3026W The key file "/opt/IBM/wlp/usr/servers/default/certs/lmi.p12" does not exist or cannot be read.
CTGSK2101W The key database does not exist.
CTGSK3026W The key file "/opt/IBM/wlp/usr/servers/default/certs/lmi.p12" does not exist or cannot be read.
CTGSK2101W The key database does not exist.
Error: WGAWA0009E The file, lmi_trust_store.p12, does not exist.
chown: changing ownership of '/opt/ibm/wlp/etc/liberty.cache': Operation not permitted
chmod: changing permissions of '/opt/ibm/wlp/etc/liberty.cache': Operation not permitted
Password: su: Authentication failure
rm: cannot remove '/var/pdweb/*/log/*.pid': Permission denied
WGAWA0002E Error accessing the database file: embedded_ldap_keys (:0x1)
DPWAP0130E The username for the logged in UID could not be determined
DPWAP0130E The username for the logged in UID could not be determined
DPWAP0130E The username for the logged in UID could not be determined
DPWAP0130E The username for the logged in UID could not be determined
DPWAP0130E The username for the logged in UID could not be determined
DPWAP0130E The username for the logged in UID could not be determined
DPWAP0130E The username for the logged in UID could not be determined
DPWAP0130E The username for the logged in UID could not be determined
DPWAP0130E The username for the logged in UID could not be determined
DPWAP0130E The username for the logged in UID could not be determined
DPWAP0130E The username for the logged in UID could not be determined
DPWAP0130E The username for the logged in UID could not be determined
Error: DPWAP0003I An error occurred while executing the command: /bin/sh -c /bin/rm -f -- /var/pdweb/*/log/*.pid (0x1)
DPWAP0130E The username for the logged in UID could not be determined
DPWAP0130E The username for the logged in UID could not be determined
DPWAP0130E The username for the logged in UID could not be determined
DPWAP0130E The username for the logged in UID could not be determined
DPWAP0130E The username for the logged in UID could not be determined
DPWAP0130E The username for the logged in UID could not be determined
DPWAP0130E The username for the logged in UID could not be determined
DPWAP0130E The username for the logged in UID could not be determined
DPWAP0130E The username for the logged in UID could not be determined
find: '/var/pdweb/*/server-root/lib/junction-root': Permission denied
find: '/var/PolicyDirector/pdmgrd': Permission denied
/bin/sh: /tmp/tmp.dWCUJj/Warnings: Permission denied
/bin/sh: /tmp/tmp.dWCUJj/Warnings: Permission denied
/bin/sh: /tmp/tmp.dWCUJj/Warnings: Permission denied
/bin/sh: /tmp/tmp.dWCUJj/Warnings: Permission denied
/bin/sh: /tmp/tmp.dWCUJj/Warnings: Permission denied
/bin/sh: /tmp/tmp.dWCUJj/Warnings: Permission denied
/bin/sh: /tmp/tmp.dWCUJj/Warnings: Permission denied
/bin/sh: /tmp/tmp.dWCUJj/Warnings: Permission denied
/bin/sh: /tmp/tmp.dWCUJj/Warnings: Permission denied
/bin/sh: /tmp/tmp.dWCUJj/Warnings: Permission denied
/bin/sh: /tmp/tmp.dWCUJj/Warnings: Permission denied
/bin/sh: /tmp/tmp.dWCUJj/Warnings: Permission denied
/bin/sh: /tmp/tmp.dWCUJj/Warnings: Permission denied
/bin/sh: /tmp/tmp.dWCUJj/Warnings: Permission denied
/bin/sh: /tmp/tmp.dWCUJj/Warnings: Permission denied
/bin/sh: /tmp/tmp.dWCUJj/Warnings: Permission denied
Bootstrap failed
Rebooting with factory default policies
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down
Failed to talk to init daemon.
2022-07-15T22:15:09+00:00: WGAWA0990I Running.
2022-07-15T22:15:28+00:00: WGAWA0987I Log file: /var/log/messages
2022-07-15T22:19:19+00:00: WGAWA0983I Shutting down.
Stopping all processes
rm: cannot remove '/var/pdweb/*/log/*.pid': Permission denied
------------------------------
Matt Jenkins
Original Message:
Sent: Thu May 05, 2022 06:52 PM
From: Scott Exton
Subject: ISVA container on Kubernetes / OpenShift - Is security context and running as specific user needed for new lightweight containers?
Matt,
The good news is that with the lightweight containers you do not need to set any additional security contexts, and you can use the default service account if you want. This means that the following YAML entries, from the documentation, are not required:
serviceAccountName: isva
securityContext:
runAsNonRoot: true
runAsUser: 6000
In fairness to Jon, the removal of the requirement to run as the '6000' user is a fairly recent change.
The YAML in the documentation still includes these entries because in non-OpenShift environments it is still a good idea to set these values. The documentation does include a subtle reference to not requiring the '6000' user in the OpenShift section:
Note: The default security context is adequate for the verify-access-runtime, verify-access-wrp, and verify-access-dsc containers. When using the default security context the 'isam' user (uid: 6000), defined by the runAsUser entry, should not be specified in the deployment descriptor.
I hope that this helps.
Scott A. Exton
Senior Software Engineer
Chief Programmer - IBM Security Verify Access
IBM Master Inventor
Original Message:
Sent: 5/5/2022 10:02:00 AM
From: Matt Jenkins
Subject: ISVA container on Kubernetes / OpenShift - Is security context and running as specific user needed for new lightweight containers?
I thought I had heard the new lightweight containers (at least everything but the config, maybe the runtime) could run without the special permissions. I am wondering, in this case:
- Does the security context constraint still need added when using OpenShift?
- Does each app need the serviceAccountName (isva/isam)?
- Is the securityContext on each app set to runAsNonRoot true and runAsUser 6000 needed?
Of course all this comes from the knowledge center here, and it still shows all these settings on the app manifests:
https://www.ibm.com/docs/en/sva/10.0.3?topic=orchestration-kubernetes-support
I am just curious, especially on the WRPs we have in their own clusters in the DMZ, if maybe I can remove the SCC completely, and remove these settings.
Adding the SCC requires cluster admin privs on OpenShift which I have to get our OpenShift admins involved to do. I could deploy a lot faster, especially in lab situations, if that SCC wasn't required.
Thanks!
------------------------------
Matt Jenkins
------------------------------