Recovering from a Bucket

How save objects and recover them using object storage.

Introduction

A RecoveryBucket resource indicates an Object Storage that will be used to restore original objects in the RecoveryPlan.

Object Storage stores data in an unstructured format in which each entry represents an object. Unlike other storage solutions, there is not a relationship or hierarchy between the data being stored. Organizations can access their files as easy as with traditional hierarchical or tiered storage. Object Storage benefits include virtually infinite scalability and high availability of data.

Many Cloud Providers include their own flavor of Object Storage and most tools and SDKs can interact with them as their share the same interface. Disaster Recovery Operator officially supports the following Object Storage solutions:

AWS Simple Storage Service (S3) Google Cloud Storage

Disaster Recovery Operator can support multiple buckets in different providers as each one is managed independently.

Contents stored in a bucket

A bucket is assigned to a RecoveryPlan spec.resources item. The same bucket can be assigned to multiple resources. It stores every synchronized object in the destination cluster with some internal control annotations added. In the case of a disaster, resources with recoveryProcess.fromOriginal.enabled equal to true will be restored using the bucket configuration.

The path of a stored object is as follows: <recoveryplan_namespace>/<recoveryplan_name>/<object_group-version-resource>/<object_namespace>.<object_name>.

Requirements

  • At least an instance of a ObjectStorage service in one of the supported Cloud Providers. This is commonly known as a bucket and will be referred as so in the documentation.
  • At least one pair of accessKeyID and secretAccessKey that gives both write and read permissions over all objects of the bucket. Refer to the chosen cloud provider documentation to learn how to create and extract them. It is recommended that each access key pair has only access to a single bucket.

Preparing and setting the bucket

Create the secret

Store the following file and apply it into the cluster substituting the template parameters with real ones.

apiVersion: v1
kind: Secret
metadata:
  name: bucket
  namespace: <namespace>
stringData:
  s3.auth.yaml: |
    accessKeyID: <access_key_id>
    secretAccessKey: <secret_access_key>
    useSSL: true

Create the RecoveryBucket

Store the following file and apply it into the cluster substituting the template parameters with real ones.

apiVersion: dr.astronetes.io/v1alpha1
kind: RecoveryBucket
metadata:
  name: bucket
  namespace: <namespace>
spec:
  endpoint: storage.googleapis.com
  bucketName: <bucket_name>
  secretRef:
    name: bucket
    namespace: <namespace>

Create the RecoveryPlan

For how to get started with Recovery Plans check its section. If the Recovery Plan does not set spec.resources[x].recoveryProcess.fromOriginal.enabled equal to true, where x refers to the index of the desired resource, the contents of the bucket will not be used. For the configuration to work, make sure both the bucket reference and recovery process transformations are correctly set.

Indicating which bucket to use can accomplished by configuring the spec.BucketRef like in the following example:

apiVersion: dr.astronetes.io/v1alpha1
kind: RecoveryPlan
metadata:
  name: applications
spec:
  suspend: false
  forceNamespaceCreation: true
  sourceClusterRef:
    name: source
    namespace: dr
  destinationClusterRef:
    name: destination
    namespace: dr
  resources:
    - group: apps
      version: v1
      resource: deployments
      transformation:
        patch:
          - op: replace
            path: /spec/replicas
            value: 0
      recoveryProcess:
        fromOriginal:
          enabled: true
  bucketRef:
    name: <bucket_name>
    namespace: <bucket_namespace>
    objectPrefix: <object_prefix>

Create the secret

Store the following file and apply it into the cluster substituting the template parameters with real ones.

apiVersion: v1
kind: Secret
metadata:
  name: bucket
  namespace: <namespace>
stringData:
  s3.auth.yaml: |
    accessKeyID: <access_key_id>
    secretAccessKey: <secret_access_key>
    useSSL: true

Create the RecoveryBucket

Store the following file and apply it into the cluster substituting the template parameters with real ones.

S3 requires that the region in the endpoint matches the region of the target bucket. It has to be explicitely set as AWS does not infer buckets region e.g. us-east-1 for North Virginia.

apiVersion: dr.astronetes.io/v1alpha1
kind: RecoveryBucket
metadata:
  name: bucket
  namespace: <namespace>
spec:
  endpoint: s3.<aws_region>.amazonaws.com
  bucketName: <bucket_name>
  secretRef:
    name: bucket
    namespace: <namespace>

Create the RecoveryPlan

For how to get started with Recovery Plans check its section. If the Recovery Plan does not set spec.resources[x].recoveryProcess.fromOriginal.enabled equal to true, where x refers to the index of the desired resource, the contents of the bucket will not be used. For the configuration to work, make sure both the bucket reference and recovery process transformations are correctly set.

Indicating which bucket to use can accomplished by configuring the spec.BucketRef like in the following example:

apiVersion: dr.astronetes.io/v1alpha1
kind: RecoveryPlan
metadata:
  name: applications
spec:
  suspend: false
  forceNamespaceCreation: true
  sourceClusterRef:
    name: source
    namespace: dr
  destinationClusterRef:
    name: destination
    namespace: dr
  resources:
    - group: apps
      version: v1
      resource: deployments
      transformation:
        patch:
          - op: replace
            path: /spec/replicas
            value: 0
      recoveryProcess:
        fromOriginal:
          enabled: true
  bucketRef:
    name: <bucket_name>
    namespace: <bucket_namespace>
    objectPrefix: <object_prefix>