XDCR is Couchbase’s built-in mechanism for replicating data between clusters, often across different datacenters. It’s not just a simple copy; it’s an active-passive or active-active replication that maintains consistency and allows for disaster recovery and geographical distribution of your data.
Let’s see it in action. Imagine we have two Couchbase clusters, cluster1 in New York and cluster2 in London. We want to replicate a bucket named travel-sample from cluster1 to cluster2.
First, on cluster2 (the target cluster), we need to set up a replication specification. This involves telling cluster2 how to connect to cluster1 and which bucket to replicate.
On cluster2, navigate to Replication in the Couchbase Web Console.
Click Create Replicated Bucket.
Source: cluster1 (this is a logical name you define for the source cluster).
Source Username: Administrator
Source Password: password123
Source Hostname: 192.168.1.100 (IP address of a node in cluster1)
Target Bucket: travel-sample (the bucket on cluster2 where data will be replicated)
Source Bucket: travel-sample (the bucket on cluster1 to replicate from)
Replication Type: Continuous (for ongoing replication)
Click Save.
Couchbase will then establish a connection and start replicating data. You can monitor the progress in the Replication tab, seeing the number of documents replicated and the replication lag.
The mental model for XDCR is a bit like a distributed transaction log. When a document is written or updated in the source cluster, that change is recorded. XDCR then picks up these changes and applies them to the target cluster. The key is that XDCR is designed to be resilient. If a network connection drops, it will queue up changes and resume replication when the connection is restored.
The "magic" of XDCR lies in its conflict resolution. In an active-active setup, where you might be writing to both clusters simultaneously, conflicts can arise if the same document is modified on both sides. Couchbase uses a last-write-wins (LWW) strategy by default, based on the document’s timestamp. However, you can customize this behavior using custom conflict resolvers, which are small JavaScript functions that run on the Couchbase cluster to decide how to merge conflicting versions of a document. This allows for complex business logic to dictate data consistency.
For example, imagine you have a user_profile document. If a user updates their address on cluster1 while simultaneously updating their phone number on cluster2, a standard LWW might overwrite one of the changes. A custom conflict resolver could be written to merge these distinct updates, ensuring both the address and phone number persist.
The _sync bucket is where Couchbase stores metadata related to XDCR, including information about replication status, checkpoints, and any potential errors. While you generally don’t interact with it directly, understanding its existence is crucial for deep troubleshooting. It’s the backbone of XDCR’s state management, holding the distributed pointers that enable it to pick up where it left off after an interruption.
When setting up XDCR, especially for active-active scenarios, carefully consider your network latency and bandwidth. High latency can significantly increase replication lag, impacting the freshness of your data across datacenters. Couchbase offers tuning parameters for XDCR, such as xdcr_batch_size and xdcr_max_concurrent_batches, which can be adjusted to optimize throughput based on your network conditions.
The next step after successfully setting up XDCR is often to configure failover and failback procedures.