The PACELC theorem is more about when you’d choose one trade-off over another, rather than if you can achieve them.
Imagine a distributed database. When things are running smoothly, it’s a simple story: you can have Consistency © and Availability (A) and Partition Tolerance (P) all at once. That’s CAP. But what happens when the network hiccups (a partition)? CAP says you have to pick: either sacrifice Consistency or Availability.
PACELC adds a crucial layer: what happens during normal operation (when there’s no partition)? It says that even when the system is healthy, you still have to make a trade-off, this time between Latency (L) and Consistency © or Availability (A). So, for any distributed system, you have two choices:
- P vs. A: During a network partition, do you prioritize Availability (A) or Consistency ©?
- E vs. C/A: During normal operation (when there’s no partition), do you prioritize Latency (L) or Consistency/Availability (C/A)?
Let’s see this in action with a simplified distributed key-value store. We’ll use a conceptual example to illustrate the trade-offs, as actual implementation details vary wildly between systems like Cassandra, DynamoDB, or etcd.
Consider a system with two nodes, Node A and Node B, storing a simple key-value pair, say {"user_id": "alice", "balance": 100}.
Scenario 1: Prioritizing Availability over Consistency during Partitions (AP)
- Normal Operation (E): Reads and writes are fast. If a client writes to Node A, Node A acknowledges it immediately. If another client reads from Node B, it might get the old value (100) if the write hasn’t replicated yet. This is a latency trade-off for availability.
- Network Partition: Node A and Node B can’t talk to each other.
- A client writes
{"user_id": "alice", "balance": 110}to Node A. Node A accepts the write and confirms. The client sees an updated balance. - Another client writes
{"user_id": "alice", "balance": 120}to Node B. Node B accepts the write and confirms. This client also sees an updated balance. - The Problem: Now, Node A has
balance: 110and Node B hasbalance: 120. If the partition heals and the system needs to reconcile, which value is correct? This system chose to stay available for writes on both sides, sacrificing immediate consistency. It will need a conflict resolution strategy later (e.g., "last write wins" based on timestamp, or manual intervention).
- A client writes
Scenario 2: Prioritizing Consistency over Availability during Partitions (CP)
- Normal Operation (E): To ensure consistency, writes might require acknowledgment from a majority of nodes. If a client writes
{"user_id": "alice", "balance": 100}to Node A, Node A might wait for Node B to confirm receipt before telling the client the write succeeded. This introduces latency for the client’s write operation. - Network Partition: Node A and Node B can’t talk.
- A client tries to write
{"user_id": "alice", "balance": 110}to Node A. Node A cannot get confirmation from Node B (or a majority of nodes if there were more). Node A rejects the write. The client gets an error. - The client tries to read. Node A might serve the old value (100) if it has it cached, or it might refuse to serve data if it can’t guarantee it’s up-to-date with the (unreachable) majority.
- The Outcome: The system prioritizes ensuring that any data it does serve is consistent across the available nodes, even if it means rejecting writes or reads entirely.
- A client tries to write
Scenario 3: Prioritizing Latency over Consistency during Normal Operation (EL)
This is essentially what we saw in Scenario 1 during normal operation. Reads and writes are fast because they don’t wait for cross-node consensus. The system aims for "eventual consistency."
Scenario 4: Prioritizing Consistency over Latency during Normal Operation (EC)
This is what we saw in Scenario 2 during normal operation. Writes are slower because they involve synchronous replication or consensus protocols. Reads might also be slower if they need to query multiple nodes to ensure they get the most up-to-date version.
The PACELC "Theorem" in Practice
Most modern distributed systems are configurable to lean one way or the other. For example, Cassandra famously allows you to set a "consistency level" per-operation.
CONSISTENCY LEVEL = ONE: This means for a read or write, you only need acknowledgment from one node. This prioritizes Latency (L) and Availability (A). If a partition occurs, you can still read/write to the available node. This is AP.CONSISTENCY LEVEL = QUORUM: This means you need acknowledgment from a majority of nodes. For a 3-node cluster, quorum is 2. This prioritizes Consistency ©. If a partition occurs (e.g., 1 node is isolated), you can’t form a quorum, so writes and reads to the isolated node will fail. This is CP.
The "E" in PACELC refers to the "Every" state, meaning normal, non-partitioned operation. So, when E is true (no partition), you still make a choice:
P(Partition tolerance) is always assumed in distributed systems.- When partitioned, you pick
A(Availability) orC(Consistency). - When not partitioned (
Eis true), you pickL(Latency) orC/A(Consistency/Availability).
The counterintuitive part of PACELC is that even when your system is perfectly healthy, you’re still making a fundamental trade-off between how fast operations are (Latency) and how strictly consistent or available they are. Many engineers tend to focus only on the CAP trade-off during partitions, forgetting that the "normal" state also has its own set of critical choices that impact user experience and system behavior.
The next logical step is understanding how different replication strategies and consensus algorithms (like Raft or Paxos) map onto these PACELC choices.