Consul’s gossip protocol, the backbone of its distributed consensus and service discovery, is unencrypted by default, leaving your cluster’s internal communication vulnerable.
Let’s see it in action. Imagine a two-node Consul cluster, node1 and node2, that we want to secure.
First, on node1, we generate a shared secret key. This key will be used by all nodes in the cluster to encrypt and decrypt gossip traffic.
consul tls-certgen -key-file consul-agent-master.key -cert-file consul-agent-master.pem -ca-file consul-ca.pem -name server -domain example.com
This command generates a server certificate (consul-agent-master.pem), its private key (consul-agent-master.key), and uses a Certificate Authority (consul-ca.pem) to sign it. While this is for TLS, we’ll adapt the principle for our pre-shared key scenario. For a true pre-shared key, you’d generate a symmetric key using openssl rand -hex 32 and distribute it securely. Let’s assume we have that key, my_super_secret_key_hex.
Now, we configure Consul on node1 to use this key for gossip encryption.
On node1, in /etc/consul.d/consul.hcl:
server = true
bootstrap_expect = 1
datacenter = "dc1"
data_dir = "/opt/consul"
log_level = "INFO"
node_name = "node1"
bind_addr = "192.168.1.10"
# Gossip encryption configuration
encrypt = "my_super_secret_key_hex"
We start Consul on node1:
consul agent -config-dir=/etc/consul.d/
On node2, we configure it identically, ensuring it uses the same pre-shared key.
On node2, in /etc/consul.d/consul.hcl:
server = false
datacenter = "dc1"
data_dir = "/opt/consul"
log_level = "INFO"
node_name = "node2"
bind_addr = "192.168.1.11"
# Gossip encryption configuration
encrypt = "my_super_secret_key_hex"
And start Consul on node2:
consul agent -config-dir=/etc/consul.d/
Now, if you check the Consul UI or run consul members on either node, you’ll see them communicating. The key insight here is that Consul uses the encrypt configuration value as a secret key to derive symmetric encryption keys for its internal gossip. This isn’t a complex handshake; it’s a direct application of a shared secret for AES-GCM encryption of UDP packets.
The problem this solves is preventing eavesdropping on the cluster’s internal state. Without encryption, anyone on the network segment could potentially see member lists, service registrations, and health check information. By enabling encryption, you ensure that only nodes possessing the correct encrypt key can participate in the gossip protocol and understand the messages.
The exact levers you control are the encrypt configuration parameter in your Consul agent configuration files. This value must be identical across all agents in a given Consul cluster. The value itself is a base64 encoded string representing a symmetric encryption key. Consul uses this key to generate session keys for encrypting gossip traffic using AES-GCM.
What most people don’t realize is that the encrypt key doesn’t need to be rotated as frequently as TLS certificates for agent-to-agent communication. Consul derives the actual encryption keys from this secret, and as long as the secret remains the same, the derived keys will be consistent for the lifetime of the cluster or until the encrypt value is changed. Changing the encrypt value requires a rolling restart of the Consul agents, and all agents must be updated to the new value before the old one is removed.
The next concept you’ll likely encounter is securing the Consul API and UI using TLS certificates.