Consul’s WAN gossip is the backbone of multi-datacenter federation, allowing your Consul clusters to discover and communicate with each other across geographically dispersed networks.
Let’s see it in action. Imagine you have two datacenters, dc1 and dc2.
Here’s a snippet of a Consul agent configuration for dc1:
server = true
datacenter = "dc1"
data_dir = "/opt/consul/data"
bind_addr = "192.168.1.10:8300" # Agent's IP on dc1 network
retry_join = ["192.168.2.20:8301"] # IP of a Consul server in dc2
And for dc2:
server = true
datacenter = "dc2"
data_dir = "/opt/consul/data"
bind_addr = "192.168.2.20:8300" # Agent's IP on dc2 network
retry_join = ["192.168.1.10:8301"] # IP of a Consul server in dc1
When these agents start, they’ll attempt to reach out to the specified retry_join addresses. If successful, they’ll establish a WAN gossip connection.
The core problem WAN gossip solves is enabling Consul servers in different datacenters to form a single, logical Consul cluster. Without it, each datacenter would operate as an isolated Consul domain, unable to share service discovery information or health checks across locations. This is crucial for applications that span multiple regions, requiring a unified view of available services and their health status.
Internally, WAN gossip uses a UDP-based protocol (port 8301 by default) for efficient, decentralized communication. When a Consul server in dc1 connects to a server in dc2, they exchange membership information. This includes details about other servers and clients within their respective datacenters. This information is then propagated throughout the WAN gossip pool. Each server maintains a view of the entire federated cluster, including all datacenters and their members.
The key levers you control are:
retry_join: This is the primary mechanism for bootstrapping the WAN connection. You specify the addresses of one or more Consul servers in other datacenters. Consul will repeatedly try to connect to these addresses until a connection is established. Once connected, the gossip protocol takes over, and the new server will learn about other members in the federated cluster.start_join: Similar toretry_join, but these addresses are only used on initial startup. If the join fails, Consul won’t retry them.retry_joinis generally preferred for its resilience.wan_addr: This is the address Consul advertises to other datacenters for WAN gossip. By default, it’s derived frombind_addrand the WAN gossip port (8301). You’d typically only set this if yourbind_addris not directly routable by other datacenters.server: Only Consul servers participate in WAN gossip. Client agents do not initiate or maintain WAN gossip connections.
The most surprising truth about WAN gossip is that it’s actually two distinct gossip protocols: one for LAN (within a datacenter) and one for WAN (between datacenters). They use different ports and have different membership pools. The retry_join directive is specifically for establishing the WAN pool.
Once WAN gossip is established, you can observe the membership of your federated cluster using the Consul CLI: consul members -wan. This command will show you all the nodes participating in the WAN gossip across all federated datacenters.
The next concept you’ll need to understand is how services registered in one datacenter become visible and discoverable in another.