Storage Fusion

Introduction 

OpenShift Container Platform provides a built in Container Image Registry which runs as a standard workload on the cluster. A registry is typically used as a publication target for images built on the cluster, as well as a source of images for workloads running on the cluster.

For more information:

https://www.redhat.com/en/blog/configure-the-openshift-image-registry-backed-by-openshift-container-storage

https://docs.openshift.com/container-platform/4.8/registry/configuring-registry-operator.html

In this article, we will take a look at how to backup and restore the OpenShift image registry.

1. Registry Storage Requirements

A registry needs to have storage in order to store its contents. Image data is stored in two locations. The actual image data is stored in a configurable storage location such as

1.        Cloud storage           or
2.        A filesystem volume.

2. How to figure out current configuration in the cluster:

$ oc edit configs.imageregistry.operator.openshift.io/cluster

Look for spec.storage

1.    If you have s3: entry, then you have cloud
2.    If you have pvc: entry, then you have filesystem

3. What to back up, so we can restore in case of disaster?

Let’s review the current Registry settings first.

Use case 1 :  If cloud storage is configured.

$ oc edit configs.imageregistry.operator.openshift.io/cluster

apiVersion: imageregistry.operator.openshift.io/v1
kind: Config
metadata:
  ...
  name: cluster
  ...
spec:
  ...
  storage:
    s3:
      bucket: cluster-image-registry-us-east-1
      encrypt: true
      keyID: ""
      region: us-east-1
      regionEndpoint: ""
...

In case the cluster is configured with the cloud storage, we need to back up the 

1.     secret/image-registry-private-configuration-user (which contains s3 credentials).
2.     configs.imageregistry.operator.openshift.io/cluster

As after the disaster when cluster is recovered, so we can restore these resources.

Note: When a cluster is created or recovered, the Image Registry Operator reconciles and creates the resources in the `openshift-image-registry` namespace. Therefore, please wait until the resource `configs.imageregistry.operator.openshift.io/cluster` is created before running the recipe.

Basically, restore must override the configs.imageregistry.operator.openshift.io/cluster to get the original configuration.

Follow below steps to backup/restore s3 base configuration: 

      1.  Label the secret i.e. secret/image-registry-private-configuration-user

$ oc label secret image-registry-private-configuration-user custom-label=fusion

      2.  Create the policy from the Fusion UI

    From Fusion UI --> Backup & restore --> Policies --> Add policy --> (fill details) --> Create policy

$ oc get fbp -A
NAMESPACE                NAME                                    BACKUPSTORAGELOCATION   SCHEDULE      RETENTION   RETENTIONUNIT
ibm-spectrum-fusion-ns   openshift-image-registry-policy         ashish-bucket           00 0 1 * *    30          days

       3.  Assign the policies to openshift-image-registry namespace.

    From Fusion UI --> Backup & restore --> Backed up applications --> Project apps --> Select a cluster --> Select application --> Next --> Select a backup policy --> Assign

    This will get assigned to the default recipe.

$ oc get fpa -A
NAMESPACE                NAME                    CLUSTER   APPLICATION       BACKUPPOLICY     RECIPE          RECIPENAMESPACE          PHASE      LASTBACKUPTIMESTAMP                     CAPACITY
ibm-spectrum-fusion-ns   <policy-assignment>               <project-name>    <backup-policy>  <Recipe-name>   ibm-spectrum-fusion-ns   Assigned   22h                   154440770

       4.  Apply the below recipe.

$ oc apply -f openshift-image-registry-with-s3-backup-restore.yaml

openshift-image-registry-with-s3-backup-restore.yaml

       5.  Patch the updated recipe.

$ oc get policyassignment -n ibm-spectrum-fusion-ns

$ oc -n ibm-spectrum-fusion-ns patch policyassignment <policy-assignment-name> --type merge -p '{"spec":{"recipe":{"name":" openshift-image-registry-with-s3-backup-restore-recipe", "namespace":"ibm-spectrum-fusion-ns"}}}'

      6.  Now we can take backup.

From Fusion UI --> Backup & restore --> Backed up applications --> Click backed up application --> Actions --> Backup now

Use case 2 :  If a filesystem volume is configured

$ oc edit configs.imageregistry.operator.openshift.io/cluster

apiVersion: imageregistry.operator.openshift.io/v1
kind: Config
metadata:
  ...
  name: cluster
  ...
spec:
  ...
  storage:
    pvc:
      claim: image-registry-storage
...

In case the cluster is configured with a filesystem volume, we need to take a backup of the

1. PVC (Persistent Volume Claim) i.e. image-registry-storage
2. configs.imageregistry.operator.openshift.io/cluster

This ensures that when a restore operation is performed, all the necessary data (i.e. images) will be present.

Follow below steps to backup/restore volume configuration: 

       1. Follow steps 2 and 3 from use case 1.

       2. Apply the below recipe.

$ oc apply -f openshift-image-registry-recipe-with-pvc.yaml

openshift-image-registry-with-pvc-backup-restore.yaml

       3.  Patch the updated recipe.

$ oc get policyassignment -n ibm-spectrum-fusion-ns

$ oc -n ibm-spectrum-fusion-ns patch policyassignment <policy-assignment-name> --type merge -p '{"spec":{"recipe":{"name":"openshift-image-registry-with-pvc-backup-restore-recipe", "namespace":"ibm-spectrum-fusion-ns"}}}'

       4. Now we can take backup.

From Fusion UI --> Backup & restore --> Backed up applications --> Click backed up application --> Actions --> Backup now