Storage Fusion

In the first article we examined why providing data protection continues to play a crucial, central role for complex application platforms which rely on stateful, persistent data storage like IBM Cloud Pak for Data. 

In this article we will examine an IBM Storage Fusion capability called “recipes” which allows the user or the application platform to define a repeatable, orchestrated backup and recovery procedure to ensure that data can be restored properly anywhere at any time. We will start by looking at the constituent parts of a recipe:

• Groups – A set of resources or storage volumes that are processed as a unit in a workflow.
• Hooks – A specific action taken during a backup or restore operation to facilitate the backup or restore operation.
• Workflows – A sequence of actions that constitute a backup or restore operation based on the groups and hooks directives.

In the last article we briefly discussed the reasons why backup and restore operations might need to be orchestrated so we will not revisit that discussion now. Let’s assume that we have already determined that there is a need for backup and restore orchestration and take a closer look with an example using EnterpriseDB (EDB). The diagram below represents a container-native deployment of an EDB application:

In the diagram above, you can see that there are several OpenShift resources that are associated with the EDB application:

• Persistent storage represented by a Ceph storage volume and consumed by EDB through a periststentvolumeclaim (PVC) resource and dependent persistentvolume (pv) and storage class resources.
• Various resources both scoped to the edb namespace and to the cluster (cluster-scoped resources).

Using standard tools such as Velero, OpenShift APIs for Data Protection (OADP), and Restic you could manually create a solution which orchestrates the backup, for example:

1. Backup the namespace-scoped resources and dependent cluster-scoped resource.
2. Make the data consistent for backup.
3. Backup the persistent volumes.

Subsequently, the restore orchestration would follow:

1. Restore the persistent volumes.
2. Restore the resources.

In the previous article we discussed why this simple orchestration might not be sufficient, especially for application platforms that consist of many different applications, for example, IBM Cloud Pak for Data. Let’s discuss how we can use IBM Storage Fusion’s Backup & Restore service recipes to orchestrate backup and recovery of this EDB application.

Grouping resources for the workflow

The first exercise is to group resources together that will be treated similarly in the backup and restore workflows.  This exercise requires knowledge of the application and how the application is deployed and recovered so we will not get into details of how we determined these groupings but for the sake of the exercise we will assume that we have determined the correct groupings. For this example, we will group resources into three groups:

• A group of the persistent volumes that need to be protected (We will call this group edb_volumes)
• A group of resources (cluster-scoped and namespace-scoped) that need to be preserved as part of the backup workflow. (We will call this group edb_resources)
• An implied group of resources (cluster-scoped and namespace-scoped) that do not need to be preserved as part of the backup workflow. These resources will be recreated at restore time as part of the declarative nature of the application.

We can visualize our groupings as in the diagram below:

Workflows – Part 1

Now that we have three groups (two of the groups are explicitly defined) we can start to define a workflow. A workflow in its simplest form is a declaration of the sequence in which the groups are processed.  For example, a backup workflow could be defined as the sequence of first backing up the project resources (edb-resources) and then the persistent volume data (edb-volumes).  A YAML snippet might look something like this:

  workflows:
  - name: backup
    sequence:
  - group: edb-resources
    - group: edb-volumes

Hooks

In almost all workflows, it will be necessary to take a specific action to facilitate the backup or restore.  Some typical examples will be:

•  During the backup operation, prepare the data on the persistent volumes by executing an application specific action such as a checkpoint or a database write suspend.
• During the backup operation, make the data consistent on the persistent volumes by scaling workload resources down to zero.
• During the restore operation, ensure that a condition is met before progressing in the workflow, for example, ensure that the replica count matches the desired declared state.

Hooks allow you to provide this flexibility in the workflows.  There are three basic types of hooks.

• An exec hook which allows you to execute a specific command within a pod as part of the workflow.
• A scale hook which allows you to scale workloads resources down and back up.
• A check hook which allows you to check for a specific condition during a workflow.

Workflows – Part 2

Revisiting our original backup workflow, we could imagine an exec hook to take a checkpoint of the EDB database before backing up the persistent volume data (more specifically before taking the snapshot for the backup). Our new backup workflow might look something like this:

  workflows:
  - name: backup
    sequence:
    - group: edb-resources
    - hook: clusters-pod-exec/checkpoint
    - group: edb-volumes

This is just a simple example to demonstrate the basic concepts of groups, hooks, and workflow which taken together constitute a recipe. In the next article, we will look at a real-life example of a recipe and take a closer look at how the groups, hooks, and workflows are declared within the recipe and how a recipe is associated with an application and backup policy in IBM Storage Fusion.

Read the next blog in this series: How IBM Storage Fusion Simplifies Backup and Recovery of Complex OpenShift Applications - Part 3 “A Deeper Dive into Recipes"