Configuring a LiveSynchronization

How to proctect the platform resources from a disaster

Introduction

A LiveSynchronization resource indicates a set of Kubernetes resource to replicate or synchronize between the source cluster and the destination cluster.

Requirements

A KubernetesCluster resource for source cluster.
A KubernetesCluster resource for destination cluster.

Process

1. Configure the live synchronization

Create the livesynchronization.yaml file according to your requirements. For this example, the goal is to synchronize deployments with the disaster-recovery label set to enabled. It is also desirable that when its replication is completed that no pod is created in the destination cluster and that after a recovery is launched the deployment launches active pods again.

Let’s dissect the following YAML:

apiVersion: automation.astronetes.io/v1alpha1
kind: LiveSynchronization
metadata:
  name: livesynchronization-sample
spec:
  suspend: false
  plugin: kubernetes-objects-to-kubernetes
  config:
    sourceName: source
    destinationName: destination
    observability:
      enabled: false
    replication:
      resources:
        - group: apps
          version: v1
          resource: deployments
          transformation:
            patch:
              - op: replace
                path: /spec/replicas
                value: 0
          filters:
            namespaceSelector:
              matchLabels:
                disaster-recovery: enabled
          recoveryProcess:
            fromPatch:
              - op: replace
                path: /spec/replicas
                value: 1

spec.config.sourceName and spec.config.destinationName refers to the name and namespace of the KubernetesCluster resources for the corresponding clusters.

The spec.config.replication.resources is a list of the set of resources to deploy. A single LiveSynchronization can cover multiple types or groups of resources, although this example only manages deployments.

The type of the resource is defined at spec.config.replication.resources[*].resource. The filters can be located in spec.config.replication.resources[*].filters. In this case, the RecoveryPlan is matching the content of the disaster-recovery label.

The spec.config.replication.resources[*].transformation and spec.config.replication.resources[*].recoveryProcess establish the actions to take after each resource is synchronized and after they are affected by the recovery process respectively. In this case, while being replicated, each deployment will set their replicas to 0 in the destination cluster and will get back to one after a successful recovery. The resource parameters are always left intact in the source cluster.

2. Suspending and resumen a recovery plan

A keen eye might have noticed the spec.suspend parameter. In this example it is set to true to indicate that the recovery plan is inactive. An inactive or suspended recovery plan will not replicate new or existing resources until it is resumed. Resuming a recovery plan can be done by setting spec.suspend to false and applying the changes in yaml. Alternatively, a patch with kubectl will work as well and will not require the original yaml file:

kubectl patch livesynchronization <livesynchronization_name> -p '{"spec":{"suspend":false}}' --type=merge

3. Deploy the Live Synchronization

The live synchronizarion can be deployed as any other Kubernetes resource:

kubectl -n <namespace_name> apply -f livesynchronization.yaml

Live Synchronizations and namespaces

It is only possible to deploy one live synchronization per namespace if they share common resources such as a bucket or a kubernetes cluster.

Additional steps

For more examples, take a look at our samples.

Modifying synchronized resources.

Depending on the use case and the chosen solution for Resiliency Operator, it is convenient that resources synchronized in the destination cluster differ from the original copy. Taking as example a warm standby scenario, in order to optimize infrastructure resources, certain objects such as Deployments or Cronjobs do not need to be actively running until there is a disaster. The standby destination cluster can run with minimal computing power and autoscale as soon as the recovery process starts, reducing the required overhead expenditure.

While a resource is being synchronized into the destination cluster, its properties can be transformed to adapt them to the organization necessities. Then, if and when a disaster occurs, the resource characteristics can be restored to either its original state or an alternative one with the established recover process.

Filters

FIlters are useful to select only the exact objects to synchronize. They are set in the spec.config.replication.resources[*].filters parameter.

Name selector

The nameSelector filters by the name of the resources of the version and type indicated. The following example selects only the Configmaps that follow the regular expression config.*:

apiVersion: automation.astronetes.io/v1alpha1
kind: LiveSynchronization
metadata:
  name: livesynchronization-sample
spec:
  plugin: kubernetes-objects-to-kubernetes
  suspend: false
  config:
    sourceName: source
    destinationName: destination
    observability:
      enabled: false
    replication:
      resources:
      - version: v1
        resource: configmaps
        filters:
          nameSelector:
            regex:
              - "config.*"

This selector can also be used negatively with excludeRegex. The following example excludes every configmap that ends in .test:

apiVersion: automation.astronetes.io/v1alpha1
kind: LiveSynchronization
metadata:
  name: livesynchronization-sample
spec:
  plugin: kubernetes-objects-to-kubernetes
  suspend: false
  config:
    sourceName: source
    destinationName: destination
    observability:
      enabled: false
    replication:
      resources:
      - version: v1
        resource: configmaps
        filters:
          nameSelector:
            excludeRegex:
            - "*.test"

Namespace selector

The namespaceSelector filters resources taking in consideration the namespace they belong to. This selector is useful to synchronize entire applications if they are stored in a namespace. The following example selects every deployment that is placed in a namespace with the label disaster-recovery: enabled:

apiVersion: automation.astronetes.io/v1alpha1
kind: LiveSynchronization
metadata:
  name: livesynchronization-sample
spec:
  plugin: kubernetes-objects-to-kubernetes
  suspend: false
  config:
    sourceName: source
    destinationName: destination
    observability:
      enabled: false
    replication:
      resources:
        - group: apps
          version: v1
          resource: deployments
          filters:
            selector:
              matchLabels:
                disaster-recovery: enabled

Transformations

Transformations are set in the spec.config.replication.resources[*].transformation parameter and are managed through patches.

Patch modifications alter the underlying object definiton using the same mechanism as kubectl patch. As with jsonpatch, the allowed operations are replace, add and remove. Patches are defined in the spec.config.replication.resources[*].transformation.patch list and admits an arbitary number of modifications.

apiVersion: automation.astronetes.io/v1alpha1
kind: LiveSynchronization
metadata:
  name: livesynchronization-sample
spec:
  ...
  config:
    ...
    replication:
      resources:
      - ...
        transformation:
          patch:
            - op: replace
              path: /spec/replicas
              value: 0
            - op: remove
              path: /spec/strategy

Multiple transformations

While Resiliency Operator supports multiple transformations for the same LiveSynchronization, it does not cover having more than one transformation for the same resource group. Transformations that cover different resources of the same resource group should be in different recovery plans. The same resource or resource set can only be affected by up to one transformation and cannot be present in more than one LiveSynchronization.

RecoveryProcess

The RecoveryProcess of a LiveSynchronization is executed in the case of a disaster to recover the original status of the application in the destination cluster. A resource can be either restored from the original definition stored in a bucket or by performing custom patches like with Transformations.

To restore from the original data, read the Recovering from a Bucket section. This option will disregard performed transformations and replace the parameters with those of the source cluster.

Patching when recovering is accessible at spec.config.replication.resources[*].recoveryProcess.fromPatch list and admits an arbitary number of modifications. It will act on the current state of the resource in the destination cluster, meaning it will take into consideration the transformations performed when it was synchronized unlike when recovering from original. As with jsonpatch, the allowed operations are replace, add and remove.

apiVersion: automation.astronetes.io/v1alpha1
kind: LiveSynchronization
metadata:
  name: livesynchronization-sample
spec:
  ...
  config:
    ...
    replication:
      resources:
        - ...
          recoveryProcess:
            fromPatch:
              - op: replace
                path: /spec/replicas
                value: 1