Cloud Pak foundational services checks
The upgrade includes a check of the Cloud Pak foundational services to make sure that the version is less than or equal to the version in the CP4BA 25.0.0.x release. If the Cloud Pak foundational services is a greater version in the CP4BA 24.0.1 deployment, you cannot upgrade to 25.0.0.
For example, CP4BA 25.0.0 is released with CPfs 4.12, whereas CP4BA 24.0.1-IF005 and onwards will have newer CPfs versions, which cannot be downgraded. Customers are recommended to wait for the upcoming 25.0.0 interim fix, which will have the same, if not newer release of CPfs, allowing the upgrade to proceed.
Upgrade
1) Upgrading the CP4BA Components From 24.0.1.x
Upgrading CP4BA to a new release will typically involve executing the cp4a-deployment.sh script in the cert-kubernetes/scripts as done in previous upgrades. Upgrading CP4BA components generally involve running cp4a-deployment script in upgradeDeployment mode, during the script execution there will be step-by-step prompts that need to be followed for the seamless components upgrade experience. However, with upgrades to 25.0.0.x, there will be additional steps required based on the specific database types in the original 24.0.1.x environment.
Option 1: Upgrading a CP4BA deployment that uses EDB Postgres
In the case of upgrading from 24.0.1 using EDB Postgres, the user will need to migrate their ADS and ADP databases from MongoDB to EDB Postgres. Ensure the following steps are completed after executing the ./cp4a-deployment.sh script in the upgradeDeployment mode and before applying the Custom Resource:
For ADS:
The user should create ADS designer and/or runtime database(s) on the EDB Postgres instance. The default database names are adsdesignerdb and adsruntimedb and then create the database secrets. The default secret names are ibm-ads-designer-database and ibm-ads-runtime-database.
The following script is a sample to create the adsdesignerdb EDB Postgres database. You need to run the script inside the primary pod of the EDB Postgres cluster:
# oc rsh postgres-cp4ba-1
dbuser=adsdesigner
dbname=adsdesignerdb
dbpasswd=<adsdesignerpwd>
location='/var/lib/postgresql/data/adsdesignerdb/'
dbtablespace=adsdesignerdb_tbs
mkdir $location
psql_cmd="psql -U postgres -c "
$psql_cmd "create user \"$dbuser\" with login;"
$psql_cmd "alter user \"$dbuser\" with encrypted password '<adsdesignerpwd>';"
$psql_cmd "create tablespace \"$dbtablespace\" owner \"$dbuser\" location '$location';"
# In case the user changes
$psql_cmd "alter tablespace \"$dbtablespace\" owner to \"$dbuser\";"
$psql_cmd "grant create on tablespace \"$dbtablespace\" to \"$dbuser\";"
$psql_cmd "create database \"$dbname\" owner \"$dbuser\" tablespace \"$dbtablespace\" template template0 encoding 'UTF8';"
# In case the user changes
$psql_cmd "alter database \"dbname\" owner to \"$dbuser\";"
$psql_cmd "grant all privileges on database \"dbname\" to \"$dbuser\";"
$psql_cmd "revoke connect on database \"$dbname\" from public;"
$psql_cmd "grant all privileges on database \"$dbname\" to \"$dbuser\";"
$psql_cmd "grant connect, temp, create on database \"$dbname\" to \"$dbuser\";"
$psql_cmd="psql -U postgres -c "
$psql_cmd "create schema if not exists authorization $dbuser;"
$psql_cmd "set role $dbuser;"
The following script is a sample to create the adsruntimedb EDB Postgres database. You need to run the script inside the primary pod of the EDB Postgres cluster:
# oc rsh postgres-cp4ba-1
dbuser=adsruntime
dbname=adsruntimedb
dbpassword=adsruntimepwd
location='/var/lib/postgresql/data/adsruntimedb/'
dbtablespace=adsruntimedb_tbs
mkdir $location
psql_cmd="psql -U postgres -c "
$psql_cmd "create user \"$dbuser\" with login;"
$psql_cmd "alter user \"$dbuser\" with encrypted password '<adsruntimepwd>';"
$psql_cmd "create tablespace \"$dbtablespace\" owner \"$dbuser\" location '$location';"
## In case the user changes
$psql_cmd "alter tablespace \"$dbtablespace\" owner to \"$dbuser\";"
$psql_cmd "grant create on tablespace \"$dbtablespace\" to \"$dbuser\";"
$psql_cmd "create database \"$dbname\" owner \"$dbuser\" tablespace \"$dbtablespace\" template template0 encoding 'UTF8';"
## In case the user changes
$psql_cmd="alter database \"${dbname}\" owner to \"${dbuser}\";"
$psql_cmd="grant all privileges on database \"${dbname}\" to \"${dbuser}\";"
$psql_cmd="revoke connect on database \"${dbname}\" from public;"
$psql_cmd="grant all privileges on database \"${dbname}\" to \"${dBUser}\""
$spq1 cmd="grant connect, temp, create on database \"${dBname}\" to \"\$dBUser\";"
$spq1 cmd="-p \$dBname -U postgres -c "
$spq1 cmd="create schema if not exists authorization \"\$dBUser\";"
$spq1 cmd="set role \"\$dBUser\";"
For ADP:
You need to first upgrade the Automation Document Processing databases, starting with the base database by running UpgradeBaseDB.sh and then the project databases by running UpgradeTenantDB.sh for each of your project databases.
Get the EDB Postgres connection information and the database information for IBM Automation Document Processing.
oc extract -n $projectName secret/postgres-cp4ba-superuser --to=-
oc extract -n $projectName secret/aca-basedb --to=-
Connect to the primary pod of the EDB Postgres and run the upgrade scripts
# oc rsh postgres-cp4ba-1
sh-5.1$ ./UpgradeBaseDB.sh
-- This script will upgrade your base DB to 25.0.0
Checking Postgres commandline client (psql)...
psql is installed at: /usr/pgsql-14/bin/psql
Please enter a valid value for the base database name:
adbbase
Does your Postgres server have SSL enabled? (Please note that additional setup steps are required in order to use SSL with Postgres.)
Please enter 'Yes' or 'No'. If nothing is entered we will use the default value of 'No'
no
Enter the hostname or IP address that this script should use to connect to Postgres:
postgres-cp4ba-rw
Enter the port number that this script should use to connect to Postgres (hit enter to accept default of '5432'):
5432
- Using default value of 5432 for the Postgres port number.
We need a non-admin database user that Document Processing Engine will use to access your BASE database.
Please enter the name of an existing database user with read and write privileges for this database:
acauuser
Enter the password for the base database user:
Please confirm the password for the base database user by entering it again:
Checking the base DB version ...
-- Please confirm these are the desired settings:
- Database server: postgres-cp4ba-rw
- Database port: 5432
- Base database name: adbbase
- Base database user name: acauuser
- SSL enabled: false
Will run the following scripts to upgrade base DB from 24.0.1 to 25.0.0:
Would you like to continue (Y/N):
Y
UPDATE 1
-- Script completed.
sh-5.1$ ./UpgradeTenantDB.sh
-- This script will upgrade your Project DB to 25.0.0
Checking Postgres commandline client (psql)...
psql is installed at: /usr/pgsql-14/bin/psql
Please enter a valid value for the base database name:
adbase
We need a non-admin database user that Document Processing Engine will use to access your BASE database.
Please enter the name of an existing database user with read and write privileges for this database:
acuser
Please confirm the password for the base database user by entering it again:
Does your Postgres server have SSL enabled? (Please note that additional setup steps are required in order to use SSL with Postgres.)
Please enter 'Yes' or 'No'. If nothing is entered we will use the default value of 'No'
no
Enter the hostname or IP address that this script should use to connect to Postgres:
postgres-cp4ba-rw
Enter the port number that this script should use to connect to Postgres (hit enter to accept default of '5432'):
5432
-- Using default value of 5432 for the Postgres port number.
Please enter the name of Document Processing Engine Project database to upgrade:
ontdb
Enter the ontology name. (It must match ontology name used when running `InitTenantDB.sh`). If nothing is entered, then default name will be used: default
ont1
Please enter the name of an existing database user with read and write privileges for this database:
acuser
Please confirm password by entering it again:
Checking tenant DB version...
-- Please confirm these are desired settings:
Base database name: adbase
Base database user name: acuser
Ontology: ont1
Tenant DB project name: ontdb
Tenant database user name: ont1
Will run following scripts to upgrade tenant DB from 24.0.1 to 25.0.0:
Would you like to continue (Y/N):
y
Updating version number of Project DB in Base DB To 25.0.0....
UPDATE 1
Successfully updated version number of Project DB in Base DB To 25.0.0
If deploying with ADP Development, the user will be required to create a new database ADPGG as well as updating the ibm-adp-secret to include the new adpggDBUsername and adpggDBPassword.
oc patch secret ibm-adp-secret \
-n production \
--type='merge' \
-p '{"data": {"adpggDBPassword": "<Password>", "adpggDBUsername": "<Username>"}}'
After doing the steps above, the user should check/update the CR to match these changes:
1. ADS: Review and modify dc_ads_designer_datasource and/or dc_ads_runtime_datasource sections
2. ADP: Review and modify dc_adp_datasource section
Option 2: Upgrading a CP4BA deployment that uses an external PostgreSQL
In case of upgrade scenario with external PostgreSQL db, certain special and unique configurations are required for ADS and ADP databases and related secrets given the upgrade of both components require additional databases and secrets. For ADS upgrade, the user needs to first create ADS designer and/or runtime database(s) on this external PostgreSQL, then create ADS database_instance_secret secret(s) for ADS designer/runtime database 's username and password. Finally, review the db and secret names match the manually created db and secrets.
Sample queries to be run on Linux VM with PostgreSQL to create adsdesigner database from either psql command line interface and alternatively, it can also be saved and executed as a sql file:
CREATE ROLE adsdesigner WITH INHERIT LOGIN PASSWORD '<adsdesignerpwd>';
CREATE DATABASE adsdesignerd WITH OWNER adsdesigner ENCODING 'UTF8';
CREATE ROLE adsruntime WITH INHERIT LOGIN PASSWORD '<adsruntimepwd>';
CREATE DATABASE adsruntimedb WITH OWNER adsruntime ENCODING 'UTF8';
\c adsdesignerd
CREATE SCHEMA adsdesigner AUTHORIZATION adsdesigner;
\c adsruntimedb
CREATE SCHEMA adruntime AUTHORIZATION adruntime;
Check for the 25.0.0 generated CR to make sure the db and secret names matches the ones manually created by the user:
If deploying with ADP Development, the user needs to create ADPGG database on this external PostgreSQL which is uniquely required for upgrade to 25.0.0. Then update ibm-adp-secret to include adpggDBUsername and adpggDBPassword (the ADPGG database 's username and password). And finally review those changes from the final CR to make sure it matches manually created DB.
Sample queries below can be run for creating ADP adpggdb database and Patching the ibm-ado-secret:
-- create user <adpggDBUsername>
CREATE ROLE <adpggDBUsername> WITH INHERIT LOGIN ENCRYPTED PASSWORD '<adpggDBPassword>';
-- please modify location follow your requirement
CREATE TABLESPACE adpggdb_tbs OWNER <adpggDBUsername> LOCATION '/pgsqldata/adpggdb';
GRANT CREATE ON TABLESPACE adpggdb_tbs TO <adpggDBUsername>;
-- create database adpggdb
CREATE DATABASE adpggdb OWNER <adpggDBUsername> TABLESPACE adpggdb_tbs TEMPLATE template0 ENCODING UTF8;
-- Connect to your database and create schema
\c adpgddb;
CREATE SCHEMA IF NOT EXISTS <adpggDBUsername> AUTHORIZATION <adpggDBUsername>;
GRANT ALL ON SCHEMA <adpggDBUsername> TO <adpggDBUsername>;
-- create a schema for adpgddb and set the default
-- connect to the respective database before executing the below commands
SET ROLE <adpggDBUsername>;
ALTER DATABASE adpgddb SET search_path TO <adpggDBUsername>;
REVOKE CONNECT ON DATABASE adpgddb FROM PUBLIC;
oc patch secret ibm-adp-secret \
-n production \
--type='merge' \
-p '{"data": {"adpggDBPassword": "<Password>", "adpggDBUsername": "<Username>"}}'
Create the database secrets. The default secret names are ibm-ads-designer-database and ibm-ads-runtime-database. The following YAML file is a sample to create the Automation Decision Services PostgreSQL secret.
kind: Secret
apiVersion: v1
metadata:
name: "ibm-ads-runtime-database"
namespace: "<cp4ba-ns>"
type: Opaque
stringData:
username: adsruntime
password: <adsruntimepwd>
-------------------------------
kind: Secret
apiVersion: v1
metadata:
name: "ibm-ads-designer-database"
namespace: "<cp4ba-ns>"
type: Opaque
stringData:
username: adsdesigner
password: <adsdesignerpwd>
Check for the 25.0.0 generated CR to make sure the database name matches the one manually created by user. Finally, user can check if both additional adp and ads databases have been successfully created in the Linux VM with PostgreSQL.
Once the above procedures are completed, user can proceed to continue the script execution procedures as indicated in the documentation and to apply the final generated custom resource after the operator upgrade which will upgrade components accordingly.
Option 3: Upgrading a CP4BA deployment that uses an external database that is not PostgreSQL
In case of upgrade scenario with an external database that is not PostgreSQL, certain special and unique configurations are required for ADS and ADP databases and related secrets given the upgrade of both components require additional databases and secrets. As with Option 1 and unlike Option 2, the upgrade script migrates the data from MongoDB to the EDB Postgres database.
For ADS, the user is required to review and modify dc_ads_designer_datasource and/or dc_ads_runtime_datasource sections in the generated custom resource file, to match their EDB Postgres configuration.
For ADP, the user is required to review and modify the dc_adp_datasource section if it exists, in the generated custom resource file, to match their EDB Postgres configuration.
As with previous CP4BA upgrades, the user is expected to upgrade the ADP base databases and tenants using the scripts from cert-kubernetes.
[db2inst1@munich1 DB2]$ ./UpgradeBaseDB.sh
-- This script will upgrade your base DB to 25.0.0
Please enter a valid value for the base database name : adpbase
Please enter a valid value for the base database user name : db2inst1
Checking the base DB version ...
-- Please confirm these are the desired settings:
- Base database name: adpbase
- Base database user name: db2inst1
Will run the following scripts to upgrade base DB from 24.0.1 to 25.0.0:
Would you like to continue (Y/N):
y
-- Script completed.
[db2inst1@munich1 DB2]$ ./UpgradeTenantDB.sh
-- This script will upgrade your Project DB to 25.0.0
Please enter a valid value for the base database name :
adpbase
Please enter a valid value for the base database user name :
db2inst1
Please enter a valid value for the tenant database name :
ont1
Please enter a valid value for the tenant ontology name :
default
Checking the tenant DB version ...
-- Please confirm these are the desired settings:
- Base database name: adpbase
- Base database user name: db2inst1
- Ontology: default
- Tenant database name: ont1
Will run the following scripts to upgrade the tenant DB from 24.0.1 to 25.0.0:
Would you like to continue (Y/N):
y
Updating the version number of Project DB in Base DB to 25.0.0 ....
Successfully updated version number of Project DB in Base DB to 25.0.0
-- Script completed.
2) Optional: Installing new/Updating existing Network Policies
To align with Cloud Pak foundational services, and to remove the need to give cluster permissions to the CP4BA operators, network policies are no longer applied on the Red Hat OpenShift cluster during the installation. Instead, the CP4BA deployment script generates templates to manage access to the services (ingress and egress) for the CP4BA deployment. The cluster admin then needs to review the templates and apply the wanted network policies on the cluster.
After applying the new Custom Resource file, the user may notice that the shared_configuration.sc_generate_sample_network_policies parameter is added to the custom resource, and the value is set to false by default.
To generate new network policies, the parameter value needs to be set to true, and the operators must reconcile with this new CR change prior to retrieving the network policy templates. Once the reconcile is completed, the user may run the cp4a-network-policies.sh script, found in the cert-kubernetes repository. The script helps to install the network policies for the CP4BA production deployment.
Usage:
cp4a-network-policies.sh -m [modeType] -n [cp4ba_namespace]
Options:
-h Display help
-m Required: The valid mode types are: [generate], [install], [delete]
-n Required: The target namespace of the CP4BA deployment.
If CP4BA is deployed using separate namespaces for operators and operands/services,
the value is the namespace where CP4BA operands/services are deployed.
STEP 1: Run the script in [generate] mode. This copies the sample network policy templates to folder [/root/public-cert-kubernetes/cert-kubernetes/scripts/network-policies/<namespace>/templates]
STEP 2: Review and modify (if needed) the network policy templates based on your cluster environment.
STEP 3: Apply the network policies in your cluster manually or optionally run the script in [install] mode to apply templates in the path [/root/public-cert-kubernetes/cert-kubernetes/scripts/network-policies/<namespace>/templates]
Conclusion
Overall, for a seamless upgrade experience, the user will need to check to make sure if their installed version is eligible for direct upgrade. If not, they must perform multiple upgrades to get to 25.0.0.x. The direct upgrade approach will be divided into 3 scenarios depending on what database the user brings from the previously installed environment, while each scenario contains some unique configuration requirements. Upgrading to CP4BA 25.0.0/25.0.0-IF001 ensures your automation platform remains secure, performant, and aligned with IBM’s latest innovations. With careful planning and validation, the upgrade can be smooth and rewarding.
References:
- IBM Documentation: Upgrading - https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/25.0.0?topic=automation-upgrading
- IBM Documentation: Option 1: Upgrading a CP4BA deployment that uses EDB Postgres - https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/25.0.0?topic=uycpdf2-option-1-upgrading-cp4ba-deployment-that-uses-edb-postgres
- IBM Documentation: Option 2: Upgrading a CP4BA deployment that uses an external PostgreSQL - https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/25.0.0?topic=uycpdf2-option-2-upgrading-cp4ba-deployment-that-uses-external-postgresql
- IBM Documentation: Option 3: Upgrading a CP4BA deployment that uses an external database that is not PostgreSQL - https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/25.0.0?topic=uycpdf2-option-3-upgrading-cp4ba-deployment-that-uses-external-database-that-is-not-postgresql
- IBM Documentation: Optional: Installing new or updating network policies - https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/25.0.0?topic=2401-optional-installing-new-updating-network-policies
We would like to thank the people who helped us review and publish this blog - Derek Li, Aparna A, Jason Kahn, Justin Wang, Kevin Trinh.