Must-knows of the post-upgrade tasks for Cloud Pak for Data upgrade

In the article Cloud Pak for Data upgrade best practice , I introduced 3 key phrases for the upgrade. Post-upgrade is the last phrase but not the least one. I'd like to share must-knows of the post-upgrade tasks for Cloud Pak for Data upgrade.

1. Validation
After the Cloud Pak for Data upgrade, we must check the cluster state and make sure the cluster is in healthy status. 
1) For Cloud Pak for Data 3.5, we can check the status with cpd-cli.
Run the following command.
cpd-cli status -n your-cpd-project

Make sure that each assembly is in Ready status.

2) The approach for checking the status of Cloud Pak for Data 4.0.X is different from that for 3.5.
Basically, you'll have to check the custom resource status for your services. In IBM documentation, there are dedicated commands for checking the status for each service. 

2. Re-enable the Cloud Pak for Data route disabled in the upgrade phrase (optional)
If you are using a customized Cloud Pak for Data route created by yourself and you disabled (deleted) this route during the upgrade phrase from preventing en-users accessing the cluster, you'll have to create it manually after the upgrade.

3. Re-enable the Cronjob suspended in the upgrade phrase (optional)
If you get cronjobs suspended in the Pre-upgrade phrase, you may need to re-enable these cronjobs after your upgrade.
Check the status of the cronjobs.
oc get cronjobs -n your-cpd-project
If they are still suspended, then run below command.
oc get cronjobs -n your-cpd-project | grep False| grep -v spark | cut -d' ' -f 1 | xargs oc patch cronjobs -p '{"spec" : {"suspend" : false }}'

4. Migration or code changes
Since Cloud Pak for Data 4.0.7, Spark 2.4 has been removed. You can no longer run Spark applications by using Spark 2.4. You must use Spark 3.0.
Besides, Python 3.7 and 3.8 is deprecated and will be removed in an upcoming refresh. So you are recommended to start using Python 3.9 when creating Spark applications or Environment runtimes.

5. Special post-upgrade tasks for WKC
WKC is a very popular service and it's core of the Data Fabric. So I'd like to introduce the special post-upgrade tasks for WKC (3.5 > 4.0.7) as as example.
1) Run an offline backup and restore procedure. It's only needed when you decided to run an offline backup and restore procedure during the upgrade.
2) If your cluster is using the Portworx or OpenShift Container Storage, you must resize the PVC for the InfoSphere Information Server Db2 instance to avoid running out of disk space.
3)You must update the indexes in the XMETA database to ensure good performance of SQL queries.
4)Re-build the index to find assets in the Information assets view and to enable catalog search. 
5)Manual synchronization of assets

The above special post-upgrade tasks are just an example and may change in the future. So I recommend you refer to IBM documentation as the source of truth.

6. Choosing a service upgrade plan
You can choose how the service is upgraded when you install or upgrade a newer version of the operator on the cluster in the future.
Prior to Cloud Pak for Data 4.7, the services will be automatically upgraded by default when you install a newer version of the operator on the cluster.
But if you want to manually upgrade you service when you install a newer version of the operator for the service, you can optionally pin the service's customer resource to a specific version.

7. Sanity tests from end-users is recommended

8. Have an enablement workshop about What's new and Usage for the end-users
There are platform architecture and design changes when upgrading to a new major version. An enablement workshop can get the end-users used to the new version in a shorter time and get better user experience.

Summary
In this article, I introduced must-knows of the post-upgrade tasks for Cloud Pak for Data upgrade. Together with my previous articles, they addressed all the 3 key phrases pre-upgrade, upgrade implementation and post-upgrade. I believe there would be changes regarding Cloud Pak for Data upgrade in the future. But I hope the best practice, experience, lessen learnt and tips I shared could be beneficial to the readers at this time. Thanks!
#CloudPakforDataGroup