Consul datacenters can be federated to create a single, unified service discovery and mesh control plane across multiple geographic regions.
Let’s see this in action. Imagine you have two datacenters, dc1 in New York and dc2 in London, and you want services in dc1 to be discoverable by services in dc2, and vice-versa, with mutual TLS encryption for all mesh traffic.
First, ensure each Consul datacenter has its server agents running and healthy.
In dc1 (New York), on a server agent, you’d have a configuration like this (e.g., in /etc/consul.d/server.hcl):
server = true
datacenter = "dc1"
bind_addr = "192.168.1.10:8300" # Internal IP of the server
client_addr = "0.0.0.0:8500" # For API/UI access
# Enable mesh features
auto_encrypt {
allow_weak = false
}
# Federation configuration
federation {
enabled = true
# List of remote datacenters and their Consul server addresses
# In a real setup, you'd have multiple addresses for HA
allow_unauthenticated = false # Enforce TLS for federation
}
And in dc2 (London), a similar configuration:
server = true
datacenter = "dc2"
bind_addr = "192.168.2.10:8300"
client_addr = "0.0.0.0:8500"
# Enable mesh features
auto_encrypt {
allow_weak = false
}
# Federation configuration
federation {
enabled = true
allow_unauthenticated = false
}
Now, you need to tell each datacenter about the other. This is done by configuring the Consul servers to trust and connect to each other. On a server agent in dc1, you’d add the following to its configuration:
# In dc1's server config:
retry_join = ["192.168.2.10:8300"] # IP of a Consul server in dc2
And on a server agent in dc2:
# In dc2's server config:
retry_join = ["192.168.1.10:8300"] # IP of a Consul server in dc1
Important: The retry_join here is for the server gossip protocol that establishes the basic connectivity between Consul servers in different datacenters. This is separate from the federation configuration itself, which tells Consul how to route requests and manage services across these joined datacenters.
Once the Consul servers have restarted and established gossip, you’ll see them aware of each other. You can verify this with consul members. You should see members from both dc1 and dc2 listed, with their respective datacenter names.
The federation block enables Consul to treat these joined datacenters as part of a single logical cluster for service discovery and intentions. When allow_unauthenticated = false, Consul automatically sets up mutual TLS for the server-to-server communication used for federation. Consul uses its own internal CA to issue certificates for this.
The magic behind this is that Consul’s RPC (Remote Procedure Call) mechanism is extended. When a client agent in dc1 needs to discover a service in dc2, it first queries its local Consul server. If the service isn’t local, the Consul server in dc1 will then RPC to the Consul server in dc2 to fetch the service catalog information. This happens transparently to the client application.
For the service mesh (Consul Connect), when a service in dc1 wants to connect to a service in dc2, the consul connect proxy (sidecar) in dc1 will initiate the connection. Consul’s control plane, aware of both datacenters, will ensure that the appropriate CA certificates are trusted on both ends, and that intentions (service-to-service authorization rules) are enforced across datacenters.
This federation allows for a unified view of services and a single policy domain for service mesh security, even though the Consul servers are physically distributed.
The key levers you control are:
datacenter: The logical name of your Consul deployment region.retry_join: How Consul server agents find each other across networks.federation.enabled: The switch to turn on cross-datacenter communication.federation.allow_unauthenticated: Crucial for securing the federation channel with TLS.auto_encrypt: For enabling TLS on the client-to-server and server-to-client communication within a datacenter, and also plays a role in how certificates are managed for inter-datacenter communication whenallow_unauthenticated = false.
What most people don’t realize is that the retry_join configuration for server agents is what establishes the underlying gossip mesh that allows Consul servers in different datacenters to communicate. Without this foundational gossip, the federation block wouldn’t have any servers to talk to. The federation configuration then builds on top of this by defining how Consul manages and routes information between these established datacenters for service discovery and mesh control.
Once you have successfully federated your datacenters, the next immediate challenge will be managing and synchronizing Consul Intents across these regions to enforce consistent service-to-service authorization policies.