Multi-region deployments in CockroachDB can achieve incredible availability and disaster recovery, but they fundamentally trade geographic proximity for consistency, and that trade-off manifests as latency.

Let’s see this in action. Imagine a simple SELECT statement hitting a row. In a single-region cluster, this might take 1-2ms. Now, spread that data across us-east-1, eu-west-1, and ap-southeast-2. A SELECT for a row in us-east-1 from a client in us-east-1 will still be fast. But what if that client is in eu-west-1? The request has to travel across the Atlantic, hit the CockroachDB node, read the data, and travel back. That’s a minimum of 100ms, just for network round-trip time (RTT).

CockroachDB uses a distributed consensus protocol (Raft) to ensure data consistency across regions. When you write data, that write needs to be acknowledged by a majority of replicas in the leaseholder’s region (by default, this is the region where the leaseholder for that data resides). If your leaseholder is in us-east-1 and your client is in eu-west-1, that write has to:

  1. Travel from eu-west-1 to us-east-1.
  2. Be processed by the leaseholder in us-east-1.
  3. Be replicated to a majority of Raft followers (which might be in us-east-1, eu-west-1, or ap-southeast-2). This involves network hops between regions.
  4. The leaseholder acknowledges the write.
  5. The acknowledgment travels back to the client in eu-west-1.

This round trip, involving multiple cross-region hops for consensus, is the core of multi-region write latency. Reads are generally faster if they can hit a local replica, but if the data isn’t local or if the read requires a strong consistency guarantee that forces a cross-region read, latency increases.

The primary lever you have to manage this is the default_transaction_رى (or transaction_رى at the session level) setting, and how you configure your database schema and application.

  • default_transaction_رى: This setting determines how CockroachDB handles transaction retries and consistency. The default is SERIALIZABLE, which provides the strongest consistency but can incur higher latency in multi-region setups because it might require more coordination. Setting it to READ_COMMITTED can significantly reduce read latency if your application can tolerate potentially stale reads. However, READ_COMMITTED doesn’t offer the same guarantees against race conditions.
  • Leaseholder Rebalancing: By default, CockroachDB tries to place leaseholders (the node that coordinates reads and writes for a given piece of data) in the region where the data was most recently written. You can influence this by setting ALTER RANGE ... CONFIGURE ZONE ... to pin leaseholders to specific regions or to distribute them more evenly. For example, ALTER RANGE default CONFIGURE ZONE USING num_voters=3, num_replicas=5, constraints='{"+region=us-east-1": 2, "+region=eu-west-1": 2, "+region=ap-southeast-2": 1}' attempts to distribute voters across regions. This can help reduce latency for reads originating in a specific region if you can ensure the leaseholder for that data is also in that region.
  • Application Design: The most impactful change often comes from your application. Can you partition your data such that most transactions stay within a single region? For instance, if you have users primarily in Europe, can you ensure their data and the leaseholders for that data are predominantly in eu-west-1? This often involves using REGIONAL BY TABLE or REGIONAL BY STATEMENT table locality settings. REGIONAL BY TABLE places all replicas for a table in a single region, determined by the sql.regional_by_table_default_region cluster setting, effectively making that table region-local for reads and writes if the client is also in that region.
  • Geo-Partitioning: For very large datasets, you can geo-partition tables. This allows you to specify rules for where data resides based on column values. For example, you could have all European customer data in eu-west-1 and all North American customer data in us-east-1. This is powerful but requires careful planning of your data model and query patterns.
  • Read Quiescence: In some extreme multi-region scenarios, if you have a critical read that must be fast and must be consistent, you might consider temporarily quiescing writes to a region or routing traffic to a specific region where the data is known to be fresh and the leaseholder is local. This is a more advanced technique for very specific, high-priority operations.

The underlying mechanism that allows for these trade-offs is CockroachDB’s ability to replicate data across regions and use Raft consensus. When you configure REGIONAL BY TABLE for a table and set sql.regional_by_table_default_region to us-east-1, CockroachDB ensures that all replicas for that table are located within us-east-1. A read originating from us-east-1 for data in this table will hit a local replica with minimal network latency. Writes to this table, coordinated by the leaseholder also in us-east-1, will also experience lower latency compared to a scenario where the leaseholder is in a different region. This is because the Raft consensus group for that table’s data is contained entirely within us-east-1, avoiding cross-region network hops for the consensus process itself.

The next challenge is understanding how to manage distributed transactions that span multiple regional tables or involve operations requiring inter-region coordination.

Want structured learning?

Take the full Cockroachdb course →