Kubernetes to Kubernetes

• 1: Introduction
• 2: Configuration
• 3: Observability
• 4: API Reference

1 - Introduction

Kubernetes objects can be synchronized between clusters adopting the kubernetes-to-kubernetes plugin.

Synchronization process

Special rules

Additionally, there are some special rules for source kind of objects:

• PersistentVolumeClaim: objects will be updated in the destination cluster only if .spec.resources changes.

• ServiceAccount: If you’re using a Kubernetes cluster version <v1.24, when creating a ServiceAccount, an autogenerated secret will be created. From here, there are two scenarios:

  • Service account with an autogenerated secret: The autogenerated secret will be deleted, and a new one will be generated in the target cluster. The secret is always updated during each synchronization.

  • Service account with a custom secret: The custom secret will remain unchanged.

  To avoid errors, custom secrets must not follow the autogenerated secret naming structure. Secrets are considered autogenerated if they begin with the following prefixes:

  • {serviceAccountName}-token-
  • {serviceAccountName}-dockercfg-

Blacklisted objects

There are some Kubernetes objects that are blacklisted by default and will be ignored by the synchronization process.

This is the list of the blacklisted objects:

• Namespaces which name starts with kube- or openshift , and the Namespace resiliency-operator.
• All namespaced objects inside a blacklisted Namespace.
• ConfigMaps named kube-root-ca.crt or openshift-service-ca.crt.

2 - Configuration

Introduction

The synchronization process can be configured with some specific parameters using the .spec.config attribute.

apiVersion: automation.astronetes.io/v1alpha1
kind: Synchronization
metadata:
  name: example
spec:
  plugin: kubernetes-to-kubernetes
  config:
    ...

Required configuration

Source and destination

The source and the destination clusters can be specified using the .spec.config.sourceName and .spec.config.destinationName properties. Both KubernetesCluster objects should exists in the same Namespace where the synchronization is being created.

Example:

apiVersion: automation.astronetes.io/v1alpha1
kind: Synchronization
metadata:
  name: example
spec:
  plugin: kubernetes-to-kubernetes
  config:
    sourceName: cluster-1
    destinationName: cluster-2
    ...

Selectors

The resources that should be synchronized between clusters can be configured using the .spec.config.selectors property.

Example:

apiVersion: automation.astronetes.io/v1alpha1
kind: Synchronization
metadata:
  name: example
spec:
  plugin: kubernetes-to-kubernetes
  config:
    ...
    selectors:
      - target:
          version: v1
          resources:
            - namespaces
        objectSelector:
          labelSelector:
            matchLabels:
              disaster-recovery: "true"

Optional configuration

Global selectors

Global selectors are used to set the default value on all selectors defined in .spec.config.selectors. They can be configured with the .spec.config.globalSelector property.

There are two options allowed, that can be configured at the same time:

• namespaceSelector: to set e dafult namespace selector for namespaced resources
• objectSelector: to set a default object selector for all resources

Example:

apiVersion: automation.astronetes.io/v1alpha1
kind: Synchronization
metadata:
  name: example
spec:
  plugin: bucket-to-kubernetes
  config:
    ...
    globalSelector:
      namespaceSelector:
        labelSelector:
          matchLabels:
            env: pro
      objectSelector:
        labelSelector:
          matchLabels:
            env: pro

Log level

The log level of the Pod deployed to execute the synchronization, can be configured with the .spec.config.logLevel parameter.

Example:

apiVersion: automation.astronetes.io/v1alpha1
kind: Synchronization
metadata:
  name: example
spec:
  plugin: kubernetes-to-kubernetes
  config:
    ...
    logLevel: warn

Observability

Observability can be enaled using the specific .spec.config.observability parameter.

enabled: false
interval: 60s

Example:

apiVersion: automation.astronetes.io/v1alpha1
kind: Synchronization
metadata:
  name: example
spec:
  plugin: kubernetes-to-kubernetes
  config:
    ...
    observability:
      enabled: true
      interval: 2m

Limit assigned resources

For each synchronization, a new Pod is deployed inside the same Namespace. The limit and requests resources can be set using the .spec.config.resources field.

Example:

apiVersion: automation.astronetes.io/v1alpha1
kind: Synchronization
metadata:
  name: example
spec:
  plugin: kubernetes-to-kubernetes
  config:
    ...
    resources:
      requests:
        cpu: 1
        memory: 2Gi
      limits:
        cpu: 2
        memory: 2Gi

Concurrency

The concurrency parameter can be used to improve the peformance of the synchronization process with .spec.config.concurrency`.

Example:

apiVersion: automation.astronetes.io/v1alpha1
kind: Synchronization
metadata:
  name: example
spec:
  plugin: kubernetes-to-kubernetes
  config:
    ...
    concurrency: 200

Transformations

Kubernetes obejcts from the source cluster can be transformed before being synchronized into the destination cluster.

Example:

apiVersion: automation.astronetes.io/v1alpha1
kind: Synchronization
metadata:
  name: example
spec:
  plugin: kubernetes-to-kubernetes
  config:
    ...
    transformations:
      - resources:
          - group: ""
            version: v1
            resources:
              - namespaces
        namespaceSelector:
          labelSelector:
            matchLabels:
              sync: "true"
        operations:
          - jsonpatch:
              operations:
                - op: add
                  path: /metadata/labels/test-astrosync
                  value: ok

3 - Observability

Introduction

The synchronization process exports many metrics in Prometheus format over HTTP.

Exported metrics

The following metrics are available:

Requirements

• The ServiceMonitor CRD from prometheus (servicemonitors.monitoring.coreos.com) must be enabled in the cluster where the operator is running.

Processes

Enable observability

Update the synchronization configuration, setting the paramente .spec.config.observability.enabled to true.

Once enabled the observability, a ServiceMonitor will be created in the same Namespace of the related synchronization.

Disable observability

Update the synchronization configuration, setting the paramente .spec.config.observability.enabled to false.

4 - API Reference

Config

Configuration for Kubernetes to Kubernetes synchronization

KubernetesGlobalSelector

Global selector is used to set the default value on all selectors

ObjectSelector

NameSelector

Select object by their name

LabelSelector

Select object by their labels

LabelSelectorRequirement

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

LabelSelectorOperator

A label selector operator is the set of operators that can be used in a selector requirement.

NamespaceSelector

ObservabilityConfig

Configure the synchronization process observability using Prometheus ServiceMonitor

Duration

Duration is a wrapper around time.Duration which supports correct marshaling to YAML and JSON. In particular, it marshals into strings, which can be used as map keys in json.

SynchronizationOptions

Customize the synchronization process with special options

KubernetesObjectSelector

GroupVersionResources

Select a set of GroupVersionResource

Transformations

Transformations is a list of operations to modifiy the Kubernetes objects matching the given selectors

TransformationOperation

The operation to execute to transform the objects

OperationJSONPatch

The JSONPatch operation

JSONPatchOperation

JSONPAtch operation