Redis Sentinel is a crucial component for building highly available Redis deployments, but setting it up correctly for production can be a minefield of subtle misconfigurations.
Let’s see Sentinel in action. Imagine we have a primary Redis instance at 192.168.1.100:6379. We want two Sentinel instances watching it, running on 192.168.1.101:26379 and 192.168.1.102:26379.
Here’s a minimal sentinel.conf for the first Sentinel instance:
port 26379
dir "/tmp/sentinel"
sentinel monitor mymaster 192.168.1.100 6379 2
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 10000
sentinel parallel-syncs mymaster 1
sentinel auth-pass mymaster YourSuperSecretPassword
And for the second Sentinel instance, with the same settings but a different dir to avoid conflicts:
port 26379
dir "/tmp/sentinel2"
sentinel monitor mymaster 192.168.1.100 6379 2
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 10000
sentinel parallel-syncs mymaster 1
sentinel auth-pass mymaster YourSuperSecretPassword
We’d start these like so:
redis-sentinel /etc/redis/sentinel.conf
The sentinel monitor mymaster 192.168.1.100 6379 2 directive tells Sentinel to watch a Redis master named mymaster at 192.168.1.100:6379. The 2 is the quorum, meaning at least two Sentinel instances must agree that the master is down before initiating a failover. This is critical for preventing split-brain scenarios.
The core problem Sentinel solves is detecting when a Redis master becomes unavailable and automatically promoting a replica to become the new master, reconfiguring other replicas and clients to point to the new master. It achieves this by having multiple Sentinel processes monitor the master. If a Sentinel believes the master is down (based on down-after-milliseconds), it asks other Sentinels if they agree. Once enough Sentinels (the quorum) agree, one Sentinel is elected to perform the failover. This elected Sentinel finds the best replica (usually the one with the most up-to-date data) and sends it a REPLICAOF NO ONE command, making it the new master. Then, it tells all other old replicas to replicate from the new master. Finally, it updates its internal configuration and tells other Sentinels about the new master address.
The sentinel down-after-milliseconds setting defines how long a Sentinel must not receive a PING reply from the master (or a valid RDB from a replica) before it considers the instance down. A value like 5000 (5 seconds) is a common starting point for production, balancing responsiveness with avoiding false positives due to network blips.
sentinel failover-timeout dictates the maximum time a failover process can take. If it exceeds this, other Sentinels might try to take over. A value of 10000 (10 seconds) is reasonable.
sentinel parallel-syncs is about how many replicas can be reconfigured to sync with the new master simultaneously. Setting it to 1 serializes the replica reconfigurations, which is safer for very busy replicas or networks, ensuring the new master isn’t overloaded by multiple syncs at once.
The sentinel auth-pass mymaster YourSuperSecretPassword is essential if your Redis master requires authentication. Sentinel needs this password to connect to the master and replicas. Make sure this password is the same as the requirepass directive in your Redis configuration.
A common pitfall is forgetting to configure Sentinel to watch replicas as well. If a replica is promoted but Sentinel doesn’t know about it, it won’t be able to reconfigure other replicas or clients. You achieve this by adding sentinel monitor <master-name> <master-ip> <master-port> <quorum> for every master you want to monitor, and Sentinel automatically discovers replicas of that master.
Another critical aspect is ensuring your Sentinels can communicate with each other. They need to be able to establish TCP connections on their configured ports (26379 in our example). Firewall rules must allow this inter-Sentinel communication.
Most applications connect to Redis using a client library. These libraries often have built-in Sentinel support. When a client connects to a Sentinel, it asks for the current master for a given master name (mymaster). The Sentinel replies with the master’s IP and port. If the master changes, the client can query the Sentinel again to get the new address. This client-side logic is what makes the failover "automatic" from the application’s perspective. If your client library doesn’t support Sentinel, you’ll need to implement this lookup logic yourself or use a proxy.
When a failover occurs, the most surprising thing for many is how Sentinel determines which replica is the "best" to promote. It’s not just about which one responded first. Sentinel evaluates replicas based on their priority (configured via replica-priority in redis.conf), their offset (how far behind the old master they are), and their run ID. A replica with a higher replica-priority value is preferred. If priorities are equal, the replica with the largest offset (most up-to-date) is chosen. This sophisticated selection mechanism ensures data consistency.
The next concept you’ll likely grapple with is how to manage Sentinel configurations across multiple masters and how to handle Sentinel nodes themselves failing over.