Consul servers don’t actually do anything with your services; they just coordinate the agents.

Let’s get Consul humming in production. We’ll set up a few servers for high availability and then get our agents talking to them.

Production Server Setup

For a production setup, you always want at least three Consul servers. This gives you a quorum for leader election and fault tolerance. If you lose one server, the remaining two can still elect a leader and keep the cluster running.

Here’s a basic server configuration file, typically server.json:

{
  "server": true,
  "bootstrap_expect": 3,
  "data_dir": "/opt/consul/data",
  "node_name": "consul-server-01",
  "bind_addr": "192.168.1.101",
  "client_addr": "0.0.0.0",
  "retry_join": ["192.168.1.101", "192.168.1.102", "192.168.1.103"],
  "encrypt": "YOUR_GOSSIP_KEY_HERE"
}

Let’s break this down:

  • "server": true: This tells Consul this node is a server.
  • "bootstrap_expect": 3: This is crucial for initial cluster bootstrapping. It tells Consul to wait until it sees 3 servers before forming the initial cluster. You must set this to your desired server count (e.g., 3, 5, 7) and only set it on the initial servers. Once the cluster is formed, you remove this directive from the config files for subsequent restarts.
  • "data_dir": "/opt/consul/data": Where Consul stores its state. Make sure this directory exists and Consul has write permissions.
  • "node_name": "consul-server-01": A unique name for this server. Change this for each server.
  • "bind_addr": "192.168.1.101": The IP address Consul binds to for all inter-node communication (gossip, RPC, etc.). This must be an IP address reachable by other Consul nodes.
  • "client_addr": "0.0.0.0": The address Consul listens on for client connections (API requests, service registrations). 0.0.0.0 means it listens on all network interfaces.
  • "retry_join": A list of IP addresses of other Consul servers that this new server should attempt to join. This is how servers discover each other. You’ll list all your server IPs here.
  • "encrypt": "YOUR_GOSSIP_KEY_HERE": This is your secret symmetric encryption key for gossip traffic. Generate a strong one using consul keygen and distribute it securely to all your servers. This encrypts traffic between Consul nodes, keeping your cluster secure.

To start a server:

  1. Generate your encryption key:
    consul keygen
    
    Copy the output (it’s a long string).
  2. Create the server.json file on each server with its unique node_name and bind_addr. Paste your generated encryption key into the "encrypt" field on all server configurations.
  3. Start Consul:
    consul agent -config-file=/path/to/your/server.json
    

Once all three (or your chosen odd number) servers are up and running with these configurations, you can check their status:

consul members

You should see all your servers listed with status alive.

Production Agent Setup

Agents are the nodes that run your applications and register their services with Consul. They communicate with the Consul servers. You’ll have many more agents than servers.

Here’s a basic agent configuration file, typically agent.json:

{
  "data_dir": "/opt/consul/data",
  "node_name": "web-app-01",
  "bind_addr": "192.168.1.201",
  "client_addr": "127.0.0.1",
  "retry_join": ["192.168.1.101", "192.168.1.102", "192.168.1.103"],
  "encrypt": "YOUR_GOSSIP_KEY_HERE"
}

Key differences from the server config:

  • "server": false (this is the default, so it’s omitted).
  • "node_name": "web-app-01": Unique name for this agent.
  • "bind_addr": "192.168.1.201": The IP address this agent binds to for its own inter-node communication (if it were to become a server, or for specific configurations).
  • "client_addr": "127.0.0.1": This is important. By default, clients only listen on localhost for API requests. This is a security measure. If you need to query Consul from another machine on the same agent host, you’d change this (e.g., to 0.0.0.0 or a specific IP).
  • "retry_join": This is how the agent finds the Consul servers. It will try to join any of the IPs listed.
  • "encrypt": Must match the key used by the servers.

To start an agent:

  1. Ensure the "encrypt" key in the agent’s agent.json matches the one used by the servers.
  2. Start Consul:
    consul agent -config-file=/path/to/your/agent.json
    

From any of your Consul servers, you can verify the agent has joined:

consul members

You should see your web-app-01 node listed.

The "Why" of client_addr and bind_addr

bind_addr is for Consul-to-Consul communication. It’s how nodes find and talk to each other, send gossip, and perform RPCs. It needs to be an IP that’s routable between Consul nodes.

client_addr is for clients of Consul (your applications, CLI tools, other services) to talk to the Consul agent or server. By default, agents bind client_addr to 127.0.0.1 for security. This means only processes running on the same machine can talk to that Consul agent. Servers often bind client_addr to 0.0.0.0 so that any machine in your network can query Consul, but this is less common for agents running application services.

The retry_join mechanism is what allows new nodes to discover existing ones. When a node starts, it pings the IPs in retry_join. If it finds a Consul server, it asks to join the cluster. If it can’t reach any, it will keep retrying at intervals until it succeeds. This is why you don’t need to manually configure every agent to know about every server; a few well-known server IPs are sufficient.

The encrypt key is non-negotiable for production. Without it, all your cluster communication is unencrypted. This means sensitive service data, health check information, and even credentials could be snooped on. It’s a simple step with a massive security payoff.

Once your agents are running and registered, you can start registering services. A common next step is to configure service discovery for an application running on one of your agents.

Want structured learning?

Take the full Consul course →