How to Install ADS 23.0.2 on IKS/ IBM Cloud
Created with guidance from Xavier Fabre
Step 1:
Git clone / down this repository locally
git clone https://github.com/icp4a/automation-decision-services-kubernetes.git
cd automation-decision-services-kubernetes
git switch 23.0.2
cd to that directory where the script is located
./scripts/ads-install-prereqs.sh -a [1]
./scripts/ads-install.sh -a -n ads
Step 2:
Installed storage file using [https://cloud.ibm.com/docs/containers?topic=containers-storage-file-vpc-install]
-
Update the container-service plug-in to the most recent version. You can update the plug-in by running the ibmcloud plugin update container-service command.
-
Get a list of the add-on versions and decide which version to install for your cluster version.
ibmcloud ks cluster addon versions --addon vpc-file-csi-driver
Example output
-
OK
Name Version Supported Kubernetes Range Supported Openshift Range
vpc-file-csi-driver 1.0 (default) >=1.21.0 >=4.7.0
-
Enable the add-on. The add-on might take a few minutes to become ready.
-
ibmcloud ks cluster addon enable vpc-file-csi-driver --version VERSION --cluster CLUSTERID
-
Verify that the add-on is enabled.
ibmcloud ks cluster addon ls --cluster <cluster_name_or_ID>
Example output
Name Version Health State Health Status
vpc-file-csi-driver 1.0.0 normal Addon Ready
Step 3.
Change username and password to your entitlement https://myibm.ibm.com/products-services/containerlibrary
kubectl create secret docker-registry ibm-entitlement-key --namespace ads --docker-server=cp.icr.io --docker-username=cp --docker-password=$REGISTRY_PASSWORD"
Step 4.
Create the external MongoDB database following https://www.mongodb.com/docs/manual/tutorial/install-mongodb-on-red-hat/
Ensure the version is 4.4 or 5.0
Note, it is only supported in Redhat Linux 8, not 9 https://www.mongodb.com/docs/v5.0/administration/production-notes/#std-label-prod-notes-supported-platforms-x86_64
Make note of the internal and external ip and port of your node or server that you have your mongodb hosted on. This will be needed in the next section. Also, please confirm you can login with admin role. You can also use this directly to create the user and password that is used.
echo 'use admin
db.createUser({user:"mongo", pwd:"passw0rd", roles:[{role:"userAdminAnyDatabase", db:"admin"}]})
'>createMongoUser.js
mongo < createMongoUser.js
Step 5.
This step is to create and apply secret
echo 'apiVersion: v1
kind: Secret
metadata:
name: "ibm-dba-ads-mongo-secret"
type: Opaque
stringData:
gitMongoUri: "mongodb://mongo:passw0rd@hostname/ads-git?retryWrites=true&w=majority&authSource=admin"
mongoUri: "mongodb://mongo:passw0rd@hostname/ads?retryWrites=true&w=majority&authSource=admin"
mongoHistoryUri: "mongodb://mongo:passw0rd@hostname/ads-history?retryWrites=true&w=majority&authSource=admin"
runtimeMongoUri: "mongodb://mongo:passw0rd@hostname/ads-runtime-archive-metadata?retryWrites=true&w=majority&authSource=admin"
'>mongoSecret.yaml
kubectl apply -f mongoSecret.yaml
You need to replace the mongo = username , passw0rd = password, and hostname = your ip:port, it may be good to do a curl to see if you can connect to the mongo db as well as try your username and password to see if you can login into your mongoDB
You can run this to see if it connects, you may need to try with internal ip, external ip, or configure any firewall to get it working
kubectl exec -it <rest-api-pod> -- curl -kv https://hostname:portIf you get the message pod <rest-api-pod> not found, you need to delete your pod and it will regenerate, and rerun the command
Step 6.
It is optional depending on if TLS was activated or when installing the mongo server. If you are using TLS enabled, then it is required to do
Generate the certificate: https://www.ibm.com/docs/en/ads/23.0.2?topic=preparing-configuring-mongodb-storage
openssl s_client -connect <hostname>:<port> -servername <hostname> -showcerts </dev/null 2>/dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /tmp/cert.pem
kubectl create secret generic external-mongo-cert --from-file=tls.crt=/tmp/cert.pem
Step 7.
Create and Apply the CR:
Create an file named ads_iks.yaml with the content:
The trusted_certificate_list is optional and should be taken out if Step 6 is not applicable.
apiVersion: ads.ibm.com/v1
kind: AutomationDecisionService
metadata:
name: ads
namespace: ads
spec:
debug: true
license:
accept: true
version: "23.0.2"
zen:
file_storage_classname: ibmc-vpc-file-dp2 # Has to be RWX
block_storage_classname: ibmc-vpc-block-10iops-tier
decision_designer:
enabled: true
decision_runtime:
enabled: true
# Uncomment to use another storage class then default one. RWX class is needed
decision_runtime_service:
persistence:
storage_class_name: ibmc-vpc-file-dp2
mongo:
use_embedded: false
admin_secret_name: ibm-dba-ads-mongo-secret
trusted_certificate_list:
- external-mongo-cert
Note: storage_class_name and run_as_user is only used if use_embedded is true
mongo:
use_embedded: true
run_as_user: 50001
persistence:
storage_class_name: ibmc-vpc-file-dp2
external-mongo-cert may be needed if there is TLS handshake issues but strongly recommended
kubectl apply -f ads_iks.yaml
Troubleshooting Common Issues:
Warning Failed 58m (x2 over 74m) ADS-controller Some deployments have no recent pod ready: {'ads-standalone-ads-git-service', 'ads-standalone-ads-embedded-build-service', 'ads-standalone-ads-rest-api', 'ads-standalone-ads-parsing-service', 'ads-standalone-ads-credentials-service', 'ads-standalone-ads-runtime-service', 'ads-standalone-ads-run-service'}
You need to check one of these logs for describe/ logs to diagnosis the issue
kubectl describe $POD > podDescribe.txt
kubectl logs $POD --previous > podPreviousLog.txt
If you see this below it is something dealing with MongoDb Connection:
exception={com.mongodb.MongoSocketException: <my-mongo-server>}, caused by {java.net.UnknownHostException: <my-mongo-server>}}]}. Waiting for 30000 ms before timing out"
Get the yaml status:
$ kubectl get ads ads-standalone -n automation-decision-services -o yaml -o go-template='{{range .status.conditions}}CONDITION: {{.type}}{{"\n"}} STATUS: {{.status}}{{"\n"}} MESSAGE: {{.message}}{{"\n"}}{{end}}'
Warning Failed 9m28s (x4 over 10m) ADS-controller cannot find mongo secret ibm-dba-ads-mongo-secret
The secret was not created
echo 'apiVersion: v1
kind: Secret
metadata:
name: "ibm-dba-ads-mongo-secret"
type: Opaque
stringData:
gitMongoUri: "mongodb://mongo:passw0rd@hostname/ads-git?retryWrites=true&w=majority&authSource=admin"
mongoUri: "mongodb://mongo:passw0rd@hostname/ads?retryWrites=true&w=majority&authSource=admin"
mongoHistoryUri: "mongodb://mongo:passw0rd@hostname/ads-history?retryWrites=true&w=majority&authSource=admin"
runtimeMongoUri: "mongodb://mongo:passw0rd@hostname/ads-runtime-archive-metadata?retryWrites=true&w=majority&authSource=admin"
'>mongoSecret.yamlkubectl apply -f mongoSecret.yaml