Introduction
Starting with IBM Cloud Pak for Integration 2023.4.1, users can now backup and restore components of the Cloud Pak for Integration by using Red Hat OpenShift API for Data Protection (OADP).
The OADP mechanism offers a fast, simple, and consistent way of backing up the Cloud Pak. Currently, this works with:
- Operators (and related artefacts)
- Platform UI
- Declarative API
- Declarative API Product
- Automation assets
- App Connect resources
- Event Streams resources
This initial release is a great introduction to the technology, and I encourage users to try it. Not all components are supported at the moment, but all feedback is greatly encouraged. One component missing is identity and access management, however the above will allow you to restore critical workload.
Throughout this blog I'll go through a example backup and restore, and highlight some key features.
For detailed instructions on how to use OADP with the Cloud Pak, see Backing up and restoring IBM Cloud Pak for Integration.
For more information on OADP, see Introduction to OpenShift API for Data Protection.
Example backup and restore
I will walk through the process of backing up and restoring the Cloud Pak using OADP. I will do it with a fresh install of 2023.4.1.
Configuring OADP
You will need an s3 location for the backups. The simplest to use is IBM Cloud Object Storage service. The "Lite" plan is free to use, and gives you enough storage try the feature.
- Go to IBM Cloud, and create a Cloud Object Storage instance.
- Go to the instance created, and create a bucket. You can use the "Quickly get started" template.
- Make a note of the service credentials (or you can come back here later). You'll need the ones with the "hmac" suffix:
Now you can install and configure OADP on the OpenShift cluster.
- Install the OADP Operator. This can be done by going to OperatorHub, search for "OADP", selecting the RedHat operator, and installing the operator with all the default configuration.
- Create a secret in the openshift-adp namespace with the credentials to your IBM Cloud s3 location:
- Create a DataProtectionApplication resource. This defines your storage locations:
- Validate that the application was created successfully by checking the BackupStorageLocation resource in the openshift-adp namespace. It should have a phase of "Available":
Labelling resources for backup
Now that the OADP application is ready, you need to label all the resources you want to backup. Details on how the labels work can be found in the Label the instances to back up section.
A few considerations for the label commands below:
- I have installed my operators in A single namespace on the cluster mode, which means I also need to label the OperatorGroup resource.
- I am restoring into the same namespace on the same cluster, so I am not labelling the catalog sources for backup. However, labelling the catalog sources is easy to do if you also wanted to restore them as well on a new cluster.
You can do the labelling fast using the CLI:
- Change namespace to the namespace where the Cloud Pak is installed:
oc project <namespace>
- Label all the Subscriptions, I labelled all the subscriptions with the same label:
oc label subscription ibm-integration-platform-navigator backup.integration.ibm.com/component=subscription
oc label subscription ibm-appconnect backup.appconnect.ibm.com/component=subscription
oc label subscription ibm-integration-asset-repository backup.eventstreams.ibm.com/component=subscription
oc label subscription ibm-common-service-operator backup.eventstreams.ibm.com/component=subscription
- Label all the instances that you have in that namespace. In my case, those are:
-
oc label platformnavigator --all backup.integration.ibm.com/component=platformnavigator
oc label assetrepository --all backup.integration.ibm.com/component=assetrepository
oc label dashboard --all backup.appconnect.ibm.com/component=dashboard
oc label designerauthoring --all backup.appconnect.ibm.com/component=designerauthoring
You can also use the Platform UI to label, or check the labels for each instance. For example, in this other environment I can see my Kafka cluster and my Automation assets have the labels:
The labelling experience also has auto-complete to help you re-use backup labels that you might have on other instances:
Backup and restore
Backing up and restoring is as simple as creating a Backup resource, and a Restore resource.
- To create the backup, create this Backup resource.
apiVersion: velero.io/v1
kind: Backup
metadata:
name: integration
namespace: openshift-adp
spec:
ttl: 720h0m0s
defaultVolumesToRestic: false
includeClusterResources: true
includedNamespaces:
- '*'
orLabelSelectors:
- matchExpressions:
- key: backup.integration.ibm.com/component
operator: In
values:
- catalogsource
- operatorgroup
- subscription
- assetrepository
- platformnavigator
- secret
- matchExpressions:
- key: backup.apiconnect.ibm.com/component
operator: In
values:
- api
- product
- matchExpressions:
- key: backup.appconnect.ibm.com/component
operator: In
values:
- catalogsource
- operatorgroup
- subscription
- configuration
- dashboard
- designerauthoring
- integrationruntime
- integrationserver
- switchserver
- matchExpressions:
- key: backup.eventstreams.ibm.com/component
operator: In
values:
- catalogsource
- operatorgroup
- subscription
- eventstreams
- kafkaconnect
- kafkatopic
- kafkauser
- kafkabridge
- kafkaconnector
- kafkarebalance
- This will create a backup in IBM Cloud. You can see the backup files if you check Cloud Object Storage:
- You can also use the Velero CLI to see the backup:
-
velero backup describe integration -n openshift-adp --details
Once the backup is successful, you will see it show on OpenShift as well, alongside any other backups you have in the same s3 bucket. I then proceeded to delete all the instances and operators in my namespace, to simulate a data loss. I removed all the PersistentVolumeClaims as well.
To restore:
- Create the Restore resource:
-
apiVersion: velero.io/v1
kind: Restore
metadata:
name: integration
namespace: openshift-adp
spec:
backupName: integration
includeClusterResources: true
existingResourcePolicy: update
restorePVs: true
restoreStatus:
includedResources:
- api
- product
hooks: {}
includedNamespaces:
- '*'
itemOperationTimeout: 1h0m0s
orLabelSelectors:
- matchExpressions:
- key: backup.integration.ibm.com/component
operator: In
values:
- catalogsource
- operatorgroup
- subscription
- assetrepository
- platformnavigator
- secret
- matchExpressions:
- key: backup.apiconnect.ibm.com/component
operator: In
values:
- api
- product
- matchExpressions:
- key: backup.appconnect.ibm.com/component
operator: In
values:
- catalogsource
- operatorgroup
- subscription
- configuration
- dashboard
- designerauthoring
- integrationruntime
- integrationserver
- switchserver
- matchExpressions:
- key: backup.eventstreams.ibm.com/component
operator: In
values:
- catalogsource
- operatorgroup
- subscription
- eventstreams
- kafkaconnect
- kafkatopic
- kafkauser
- kafkabridge
- kafkaconnector
- kafkarebalance
- You will see operators, instances and pods come back into the same namespace.
Get the new admin password (as identify and access management is not currently backed up and restored), and you can access the Platform UI to see all the workloads running again!
Hopefully the above guide gives a good overview of how to try OADP with the Cloud Pak. All of the steps should be possible in under an hour. You can also do more complex upgrade strategies with:
- Different labels (For example, one label for the production workload, one for the UAT workload)
- Having different backup CRs for different components (For example, one that does operators and another that does instances)
- Using backup schedules
- Integrate this with CICD (For example, backup the operators / stateless components with GitOps and restore the instances with OADP)