Utilizing OADP and Migration Toolkit for Containers for Efficient, Flexible App Backups in Containers

Introduction

After extensive research, I discovered an ideal backup solution for applications running on Openshift. It is beneficial for any application running on OpenShift. If you're an OpenShift user, I am confident that you'll find this combination optimal. It is simple, intuitive and won't mentally burden the more developer focused Openshift user. Plus, Openshift provides access to MTC and OADP straight out of the box.

Use Case

The OpenShift Operator-managed Red Hat SSO initiates its own Postgresql database, which I needed to restore along with Red Hat SSO's component parts from an Amazon cluster. This task involved utilizing an Amazon native storage class to transfer data to an on-premise cluster that employs storage classes provided by OpenShift Data Foundations. In addition, I aimed to meticulously restore solely the Red Hat SSO/Keycloak Kubernetes components residing in a namespace populated with other applications.

However, finding a straightforward and viable backup solution for restoring my Red Hat Single Sign-On instance from a backup proved challenging. I explored numerous backup solutions, all of which presented certain constraints upon restoration:

1. They failed to convert between different storage classes for the Persistent Volume Claims.
2. They were unable to leverage the dynamic provisioning capability of persistent volumes, choosing instead to restore a PV with the identical name as the source cluster.
3. On its own, OADP essentially required the restoration of persistence (PVs and PVCs) to closely mirror the source cluster. It even created node affinity rules, including those related to region. Although there are ways to bypass some of these challenges, they are laborious and their reliability was not guaranteed.
4. The solutions did not offer an intuitive mechanism for selecting Kubernetes resources to restore. They either restored entire namespaces or employed non-intuitive filtering mechanisms.

The Process

1. Install the Migration Toolkit for Containers Operator on both clusters (source and target)
2. Install the OADP Operator on both clusters (source and target)
3. Schedule backups on the source cluster using OADP Operator resources:

apiVersion: oadp.openshift.io/v1alpha1
kind: DataProtectionApplication
metadata:
  generation: 3
  name: velero-itd-data-protection-app
  namespace: openshift-adp
spec:
  backupLocations:
    - velero:
        config:
          profile: default
          region: us-east-2
        credential:
          key: cloud
          name: cloud-credentials
        default: true
        objectStorage:
          bucket: oadp-backup-interthedungeon
          prefix: velero
        provider: aws
  configuration:
    restic:
      enable: true
    velero:
      defaultPlugins:
        - openshift
        - aws
        - kubevirt
  snapshotLocations:
    - velero:
        config:
          profile: default
          region: us-east-2
        provider: aws

apiVersion: velero.io/v1
kind: Schedule
metadata:
  name: schedule
  namespace: openshift-adp
  resourceVersion: '70733297'
spec:
  schedule: 30 8 * * *
  template:
    defaultVolumesToRestic: true
    hooks: {}
    includedNamespaces:
      - camel-k-itd
      - interthedungeon
      - interthedungeon-landing-page
      - interthedungeon-windows-game
      - wallet-verify
    storageLocation: velero-itd-data-protection-app-1
    ttl: 720h0m0s

4.  Create a State Migration only, Migration Plan that targets just the PVC you want to restore to target cluster

5.  Run Stage on the MTC Migration Plan

6.  Add the same DataProtectionApplication to the target clusters OADP resources

7. Add a restore kind to the target cluster OADP resources:

apiVersion: velero.io/v1
kind: Restore
metadata:
  name: restore-itd
  namespace: openshift-adp
spec:
  backupName: schedule-20230519234608 
  includedResources:
  - Keycloak
  - Secret
  - Service
  - Deployment
  - ReplicaSet
  - ConfigMap
  - PersistentVolumeClaim
  includedNamespaces:
  - interthedungeon
  orLabelSelectors:
  - matchLabels:
      app: keycloak
  - matchLabels:
      app: sso
  namespaceMapping:
    interthedungeon: interthedungeon-game
  restorePVs: true

Short Video Tutorial