Cassandra on Kubernetes, when managed by StatefulSets, isn’t just about running a database in a container; it’s about orchestrating a distributed system where node identity and stable storage are paramount.
Here’s Cassandra running in a Kubernetes cluster, managed by a StatefulSet. Notice the persistent volumes, one for each Cassandra pod, ensuring data survives pod restarts.
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: cassandra
spec:
serviceName: cassandra
replicas: 3
selector:
matchLabels:
app: cassandra
template:
metadata:
labels:
app: cassandra
spec:
containers:
- name: cassandra
image: cassandra:4.0.6
ports:
- containerPort: 9042
name: cql
- containerPort: 7000
name: intra-node
- containerPort: 7001
name: tls-intra-node
- containerPort: 7199
name: jmx
env:
- name: CASSANDRA_SEEDS
value: "cassandra-0.cassandra.default.svc.cluster.local,cassandra-1.cassandra.default.svc.cluster.local,cassandra-2.cassandra.default.svc.cluster.local"
- name: CASSANDRA_CLUSTER_NAME
value: "MyCassandraCluster"
- name: CASSANDRA_DC
value: "datacenter1"
- name: CASSANDRA_RACK
value: "rack1"
volumeMounts:
- name: cassandra-data
mountPath: /var/lib/cassandra
volumeClaimTemplates:
- metadata:
name: cassandra-data
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 100Gi
storageClassName: standard
This setup addresses the core challenges of running a distributed database like Cassandra in a dynamic environment. The StatefulSet provides stable network identities (e.g., cassandra-0, cassandra-1) and stable persistent storage, which are critical for Cassandra’s peer-to-peer discovery and data durability. Each pod gets its own PersistentVolumeClaim (cassandra-data) that is reattached to the same volume even if the pod is rescheduled. The CASSANDRA_SEEDS environment variable is configured to point to the DNS names of the initial pods, allowing new nodes to discover existing ones.
The problem this solves is maintaining Cassandra’s distributed state and ensuring data availability and consistency when individual nodes (pods) fail or are rescheduled. Unlike stateless applications, Cassandra requires each instance to have a unique identity and its own data storage. Kubernetes’ StatefulSet is designed precisely for this, guaranteeing ordered, graceful deployment and scaling, and providing stable identifiers.
Internally, the StatefulSet controller ensures that pods are created, updated, and deleted in a predictable order. For example, when scaling up, it will create cassandra-0, then cassandra-1, then cassandra-2. When scaling down, it will terminate them in reverse order: cassandra-2, then cassandra-1, then cassandra-0. This ordered operation is crucial for Cassandra’s bootstrapping and gossip protocol. The serviceName field creates a Headless Service, which provides DNS records for each pod (e.g., cassandra-0.cassandra.default.svc.cluster.local), enabling stable peer discovery.
The CASSANDRA_SEEDS configuration is vital. It uses the stable DNS names provided by the Headless Service. When a new Cassandra pod starts, it contacts these seed nodes to discover other nodes in the cluster and join the gossip protocol. The CASSANDRA_CLUSTER_NAME, CASSANDRA_DC, and CASSANDRA_RACK environment variables are standard Cassandra configurations that help define the cluster’s topology and data distribution.
One thing that often trips people up is the interaction between Cassandra’s own internal anti-entropy mechanisms (like anti-compaction and repairs) and Kubernetes’ rescheduling. If a pod is rescheduled and its PersistentVolume is attached to a new node, Cassandra’s gossip will eventually detect it. However, data consistency relies on regular repair operations. If a node is down for an extended period, data on other nodes for the ranges that node owned can become stale. Kubernetes doesn’t automatically trigger Cassandra repairs; you need to manage that separately, either through external cron jobs, dedicated operators, or by configuring Cassandra’s built-in repair scheduler if the nodes are consistently available.
The next concept you’ll likely encounter is managing Cassandra’s resource requests and limits effectively, particularly CPU and memory, to ensure stable performance and prevent OOMKills.