Swapping out a Couchbase node is less about replacing hardware and more about orchestrating a controlled migration of data and responsibilities.
Let’s say you’re decommissioning old-node-01 and bringing in new-node-01.
First, add the new node to your cluster. If you’re using couchbase-cli, it’s straightforward:
couchbase-cli server-add \
--cluster couchbase.example.com:8091 \
--user admin \
--password password \
--node-ip old-node-01 \
--new-node-ip new-node-01 \
--new-node-port 8091 \
--new-node-password password
Couchbase will now see new-node-01 as an available server. The critical step is telling it to rebalance. This process distributes the data that was on old-node-01 (and potentially other nodes) across the entire cluster, including new-node-01.
couchbase-cli rebalance \
--cluster couchbase.example.com:8091 \
--user admin \
--password password
During the rebalance, Couchbase is intelligently moving vBuckets (which are how data is sharded) from the nodes being removed to the nodes being added. You can monitor this with:
couchbase-cli rebalance --cluster couchbase.example.com:8091 --user admin --password password --status
You’ll see the percentage complete. It’s crucial not to remove the old node until the rebalance is fully complete and all data has been migrated.
Once the rebalance shows 100% completion, old-node-01 will have had all its data moved off. Now, you can safely remove it from the cluster.
couchbase-cli server-remove \
--cluster couchbase.example.com:8091 \
--user admin \
--password password \
--node-ip old-node-01
The trickiest part is understanding how vBuckets are managed. Each bucket has 1024 vBuckets by default. When you rebalance, Couchbase recalculates the ownership of these vBuckets across all available nodes. It doesn’t copy data; it transfers ownership and then moves the actual data shards (partitions of vBuckets) to the new node. The old node stops serving requests for those vBuckets, and the new node starts.
A common mistake is to immediately shut down the old node after adding the new one. This interrupts the rebalance and can lead to data unavailability if the rebalance doesn’t complete. Always wait for the rebalance --status to report 100%.
Another subtle point: if you have replication enabled between data centers, the rebalance will also account for replicating the data to the new node in the target data center. This can significantly increase the time a rebalance takes.
When you remove a node, Couchbase doesn’t just delete it; it formally tells the remaining nodes that this server is no longer part of the cluster. This is important for cluster membership and internal communication.
The next thing you’ll likely encounter is figuring out how to optimize the vBucket distribution after several rebalances, which leads into understanding bucket-tail settings and auto-failover configurations.