Synchronize Zookeeper clusters

How to synchronize Zookeeper clusters data

Overview

In environments where high availability and disaster recovery are paramount, it is essential to maintain synchronized data across different ZooKeeper clusters to prevent inconsistencies and ensure seamless failover.

In the following tutorial will be explained how to synchronize Zookeeper clusters.

Prerequisites

Install Astronetes Resiliency Operator.
Create a namespace where to store the secrets and run the synchronization between clusters.

Setup

Import the first cluster

Import the first Zookeeper cluster as described in details here:

Define the Database resource with the following YAML, and save it as zookeeper-1.yaml:

apiVersion: assets.astronetes.io/v1alpha1
kind: Database
metadata:
  name: zookeeper-1
spec:
  zookeeper:
    client:
      servers:
        - <zookeeper_ip>:<zookeeper_port>
        - <zookeeper_ip>:<zookeeper_port>
        - <zookeeper_ip>:<zookeeper_port>

Import the resource with the following command:
```
kubectl create -f zookeeper-1.yaml
```

Import the second cluster

Import the second Zookeeper cluster as described in details here:

Define the Database resource with the following YAML, and save it as zookeeper-2.yaml:

apiVersion: assets.astronetes.io/v1alpha1
kind: Database
metadata:
  name: zookeeper-2
spec:
  zookeeper:
    client:
      servers:
        - <zookeeper_ip>:<zookeeper_port>
        - <zookeeper_ip>:<zookeeper_port>
        - <zookeeper_ip>:<zookeeper_port>

Import the resource with the following command:
```
kubectl create -f zookeeper-2.yaml
```

Synchronize the clusters

Create the configuration manifest to synchronize the clusters according to the full documentation is provided here:

In the following example there is the configuration to synchronize every hour all the data in the / path:

Create the synchronization file as zookeeper-sync.yaml with the following content:

apiVersion: automation.astronetes.io/v1alpha1
kind: SynchronizationPlan
metadata:
  name: synchronize-zookeeper
spec:
  schedule: "0 * * * *"
  template:
    spec:
      plugin: zookeeper-to-zookeeper-nodes
      config:
        sourceName: zookeeper-1
        destinationName: zookeeper-2
        rootPath: /
        createRoutePath: true

Custom path

The data to be synchronized between clusters can be specified in spec.template.spec.config.rootPath.

Apply the configuration:
```
kubectl apply -f zookeeper-sync.yaml
```

Operations

Force the synchronization

The synchronization can be run at any time creating a Synchronization object.

In the following example there is the configuration to synchronize all the data in the / path:

Create the synchronization file as zookeeper-sync-once.yaml with the following content:

apiVersion: automation.astronetes.io/v1alpha1
kind: Synchronization
metadata:
  generateName: synchronize-zookeeper-
spec:
  plugin: zookeeper-to-zookeeper-nodes
  config:
    sourceName: zookeeper-1
    destinationName: zookeeper-2
    rootPath: /
    createRoutePath: true

Apply the configuration:

kubectl create -f zookeeper-sync-once.yaml