Consul watches don’t actually trigger actions; they report changes, and you build the action layer yourself.
Let’s see a watch in action. Imagine you have a service registered in Consul, and you want to know when its health status changes.
consul watch -tag 'webserver' -event 'service' -name 'nginx' -type 'critical' -invoke 'echo "NGINX service is CRITICAL!"'
This command will sit there, and if an nginx service tagged as webserver goes into a critical state, it will execute the echo command. The -invoke flag is where the "action" happens, but it’s just a shell command.
Here’s how it all fits together. Consul is a distributed service registry and discovery tool. It keeps track of services, their health, and their configuration. Watches are a mechanism to subscribe to changes in this state. When you set up a watch, you’re telling Consul, "Hey, let me know when something specific changes."
The core components are:
- Consul Agent: Every node running Consul has an agent. This is the process that registers services, performs health checks, and, importantly, handles watches.
- Consul Server: The agents communicate with Consul servers to maintain the cluster’s state.
- Watches: These are configured on the Consul agent. They can monitor changes to:
- Keys: Changes in Consul’s key-value store.
- Services: Changes in service registration, health status, or metadata.
- Nodes: Changes in node status (e.g., a node going down).
- Health: Specific health check statuses.
- Event Handlers: This is the "action" part. When a watch detects a change, it can trigger a pre-defined event. This event can be a simple shell command (
-invoke), a script, or it can send an HTTP request to another service (-http).
The Mental Model: Think of it like a pub/sub system. Consul is the publisher, and your watch is a subscriber. You don’t tell Consul how to react, just what you want to be notified about. The reaction itself is something you orchestrate.
Let’s say you have a load balancer configuration that needs to be updated whenever a webserver service’s health status changes.
-
Define the Watch:
consul watch -type 'service' -name 'nginx' -event 'fail' -invoke '/opt/scripts/update_lb_config.sh'This watch monitors the
nginxservice. If its health status transitions tofail, it will execute theupdate_lb_config.shscript. -
The Script (
/opt/scripts/update_lb_config.sh): This script would then:
This decouples the detection of change (Consul watch) from the reaction to change (your script).
The consul watch command itself is quite powerful. You can specify:
-type:key,services,nodes,health.-key: The specific KV key to watch.-service: The service name to watch.-node: The node name to watch.-tag: Filter services or nodes by tag.-event:any,serf,user,service,node,health,kv. For service/node/health watches, this typically means the state of that item. Forkv, it’s changes to the key.-interval: How often to poll Consul for changes (default is 30s).-timeout: How long to wait for a change before timing out.-handler: A script or command to run when a change occurs. This is the most common way to "trigger" actions.-http: A URL to POST changes to.
What most people miss is that the -event flag isn’t just a simple state name like "critical." For services, it refers to the type of event that triggered the watch. For example, a service watch with -event 'service' will fire whenever a service is added, removed, or its health status changes. If you want to be very specific about health, you’d typically use -type 'health' and then filter by service name and health status.
The most surprising thing about Consul watches is that they are fundamentally blocking operations on the client side. When you run consul watch, that agent process is dedicated to waiting for that specific change. If you need to watch many things or run complex actions, you’ll end up running many consul watch processes or building a more sophisticated event-driven system that listens to Consul’s HTTP API changes directly.
The next logical step after using watches for simple actions is to integrate them with more robust orchestration or configuration management tools.