Recovering from a Bucket

How save objects and recover them using object storage.

Introduction

A Bucket resource indicates an Object Storage that will be used to restore original objects when recovering from a disaster.

Object Storage stores data in an unstructured format in which each entry represents an object. Unlike other storage solutions, there is not a relationship or hierarchy between the data being stored. Organizations can access their files as easy as with traditional hierarchical or tiered storage. Object Storage benefits include virtually infinite scalability and high availability of data.

Many Cloud Providers include their own flavor of Object Storage and most tools and SDKs can interact with them as their share the same interface. Resiliency Operator officially supports the following Object Storage solutions:

AWS Simple Storage Service (S3) Google Cloud Storage

Resiliency Operator can support multiple buckets in different providers as each one is managed independently.

Contents stored in a bucket

A bucket is assigned to a LiveSynchronization by setting it in a spec.config.bucketName item. It stores every synchronized object in the destination cluster with some internal control annotations added. In the case of a disaster, resources with recoveryProcess.fromOriginal.enabled equal to true will be restored using the bucket configuration.

The path of a stored object is as follows: <bucket_namespace>/<bucket_name>/<object_group-version-resource>/<object_namespace>.<object_name>.

Requirements

At least an instance of a ObjectStorage service in one of the supported Cloud Providers. This is commonly known as a bucket and will be referred as so in the documentation.
At least one pair of accessKeyID and secretAccessKey that gives both write and read permissions over all objects of the bucket. Refer to the chosen cloud provider documentation to learn how to create and extract them. It is recommended that each access key pair has only access to a single bucket.

Preparing and setting the bucket

Create the secret

Store the following file and apply it into the cluster substituting the template parameters with real ones.

apiVersion: v1
kind: Secret
metadata:
  name: bucket-credentials
stringData:
  accessKeyID: <access_key_id>
  secretAccessKey: <secret_access_key>

Create the Bucket

Store the following file and apply it into the cluster substituting the template parameters with real ones.

apiVersion: assets.astronetes.io/v1alpha1
kind: Bucket
metadata:
  name: gcp
  namespace: <namespace>
spec:
  generic:
    endpoint: storage.googleapis.com
    name: <bucket_name>
    useSSL: true
    secretName: bucket-credentials

Create the LiveSynchronization

If the LiveSynchronization does not set spec.resources[x].recoveryProcess.fromOriginal.enabled equal to true, where x refers to the index of the desired resource, the contents of the bucket will not be used. For the configuration to work, make sure both the bucket reference and recovery process transformations are correctly set.

Indicating which bucket to use can accomplished by configuring the spec.config.bucketName like in the following example:

apiVersion: automation.astronetes.io/v1alpha1
kind: LiveSynchronization
metadata:
  name: livesynchronization-sample
spec:
  plugin: kubernetes-objects-to-kubernetes
  config:
    sourceName: source
    destinationName: destination
    bucketName: <bucket_object_name>
    observability:
      enabled: false
    replication:
      resources:
        - group: apps
          version: v1
          resource: deployments
          transformation:
            patch:
              - op: replace
                path: /spec/replicas
                value: 0
          filters:
            namespaceSelector:
              matchLabels:
                env: pre
          recoveryProcess:
            fromPatch:
              - op: replace
                path: /spec/replicas
                value: 1
        - group: apps
          version: v1
          resource: deployments
          transformation:
            patch:
              - op: replace
                path: /spec/replicas
                value: 0
          filters:
            namespaceSelector:
              matchLabels:
                env: pre-second
          recoveryProcess:
            fromPatch:
              - op: replace
                path: /spec/replicas
                value: 1
        - group: ""
          version: v1
          resource: services
          filters:
            namespaceSelector:
              matchLabels:
                env: pre
        - group: ""
          version: v1
          resource: services
          filters:
            namespaceSelector:
              matchLabels:
                env: pre-second
        - group: ""
          version: v1
          resource: secrets
          filters:
            namespaceSelector:
              matchLabels:
                env: pre

Create the secret

Store the following file and apply it into the cluster substituting the template parameters with real ones.

apiVersion: v1
kind: Secret
metadata:
  name: bucket-credentials
stringData:
  accessKeyID: <access_key_id>
  secretAccessKey: <secret_access_key>

Create the Bucket

Store the following file and apply it into the cluster substituting the template parameters with real ones.

S3 requires that the region in the endpoint matches the region of the target bucket. It has to be explicitely set as AWS does not infer buckets region e.g. us-east-1 for North Virginia.

apiVersion: assets.astronetes.io/v1alpha1
kind: Bucket
metadata:
  name: gcp
spec:
  generic:
    endpoint: s3.<bucket-region>.amazonaws.com
    name: <bucket-name>
    useSSL: true
    secretName: bucket-credentials

Create the LiveSynchronization

If the Recovery Plan does not set spec.resources[x].recoveryProcess.fromOriginal.enabled equal to true, where x refers to the index of the desired resource, the contents of the bucket will not be used. For the configuration to work, make sure both the bucket reference and recovery process transformations are correctly set.

Indicating which bucket to use can accomplished by configuring the spec.BucketRef like in the following example:

apiVersion: automation.astronetes.io/v1alpha1
kind: LiveSynchronization
metadata:
  name: livesynchronization-sample
spec:
  plugin: kubernetes-objects-to-kubernetes
  config:
    sourceName: source
    destinationName: destination
    bucketName: <bucket_object_name>
    observability:
      enabled: false
    replication:
      resources:
        - group: apps
          version: v1
          resource: deployments
          transformation:
            patch:
              - op: replace
                path: /spec/replicas
                value: 0
          filters:
            namespaceSelector:
              matchLabels:
                env: pre
          recoveryProcess:
            fromPatch:
              - op: replace
                path: /spec/replicas
                value: 1
        - group: apps
          version: v1
          resource: deployments
          transformation:
            patch:
              - op: replace
                path: /spec/replicas
                value: 0
          filters:
            namespaceSelector:
              matchLabels:
                env: pre-second
          recoveryProcess:
            fromPatch:
              - op: replace
                path: /spec/replicas
                value: 1
        - group: ""
          version: v1
          resource: services
          filters:
            namespaceSelector:
              matchLabels:
                env: pre
        - group: ""
          version: v1
          resource: services
          filters:
            namespaceSelector:
              matchLabels:
                env: pre-second
        - group: ""
          version: v1
          resource: secrets
          filters:
            namespaceSelector:
              matchLabels:
                env: pre