Spot nodes can slash your EKS costs by up to 90%, but they’re notoriously difficult to manage without automation.
Here’s what a typical EKS cluster looks like when you’re trying to use Spot instances manually:
{
"apiVersion": "v1",
"kind": "Pod",
"metadata": {
"name": "my-app-pod",
"labels": {
"app": "my-app"
}
},
"spec": {
"containers": [
{
"name": "my-app-container",
"image": "nginx:latest",
"resources": {
"requests": {
"cpu": "100m",
"memory": "128Mi"
},
"limits": {
"cpu": "200m",
"memory": "256Mi"
}
}
}
]
}
}
You’ve got your pods, and you need nodes to run them. Traditionally, you’d create an EKS managed node group or an Auto Scaling Group (ASG) and tell it to launch EC2 instances, perhaps a mix of On-Demand and Spot. The problem? When Spot instances get interrupted (which happens with little notice), your pods get evicted. If you don’t have enough fallback capacity or your Spot ASG doesn’t scale up quickly enough with On-Demand instances, your application goes down. It’s a constant game of whack-a-mole.
Karpenter changes this entirely. It’s a Kubernetes-native node autoscaler that makes running mixed instance types, including Spot, seamless. Instead of pre-provisioning nodes or relying on the native Kubernetes Cluster Autoscaler (which only scales existing node groups), Karpenter watches for unscheduled pods and launches exactly the right EC2 instance to run them, often within seconds.
Let’s see Karpenter in action. Imagine you have an EKS cluster with Karpenter installed. You deploy a new pod that requires a GPU:
apiVersion: apps/v1
kind: Deployment
metadata:
name: gpu-app
spec:
replicas: 1
selector:
matchLabels:
app: gpu-app
template:
metadata:
labels:
app: gpu-app
spec:
containers:
- name: gpu-container
image: nvidia/cuda:11.0-base-ubuntu20.04
resources:
limits:
nvidia.com/gpu: 1
This pod won’t schedule because there are no GPU-enabled nodes in your cluster. Karpenter sees this unscheduled pod. It looks at the pod’s requirements (specifically, nvidia.com/gpu: 1) and its own configuration (which includes a list of allowed instance types and your Spot provisioning preferences).
Karpenter then makes a decision: "I need to launch a GPU instance to satisfy this pod." It might choose an g4dn.xlarge instance type. Crucially, it will check your Karpenter Provisioner configuration. If your Provisioner is configured to prioritize Spot, Karpenter will attempt to launch a g4dn.xlarge Spot instance.
apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
name: default
spec:
cluster:
name: my-eks-cluster
requirements:
- key: karpenter.sh/architecture
operator: In
values: ["amd64"]
- key: karpenter.sh/instance-category
operator: In
values: ["g"] # For GPU instances
- key: kubernetes.io/arch
operator: In
values: ["amd64"]
- key: kubernetes.io/os
operator: In
values: ["linux"]
limits:
cpu: 1000 # Max 1000 CPU cores across all nodes launched by this provisioner
provider:
aws:
instanceTypes:
- g4dn.xlarge
- g5.xlarge
subnetSelectorTerms:
- tags:
eksctl.cluster.k8s.io/v1alpha1/cluster-name: my-eks-cluster
securityGroupSelectorTerms:
- tags:
eksctl.cluster.k8s.io/v1alpha1/cluster-name: my-eks-cluster
# This is where the Spot magic happens
onDemandBaseCapacity: 0 # Don't use On-Demand for initial capacity
onDemandAboveBaseCapacity: 0 # Don't use On-Demand for scaling up
spotMaxPrice: 0.9 # Max price for Spot instances (optional, but good for cost control)
interruptionQueue: my-interruption-queue # SQS queue for Spot interruptions
Notice the onDemandBaseCapacity and onDemandAboveBaseCapacity set to 0. This tells Karpenter to only use Spot instances if possible for this provisioner. If it can’t find a Spot instance for g4dn.xlarge (perhaps due to price or availability), it will try g5.xlarge as a Spot instance. If all Spot options are exhausted, and you had onDemandBaseCapacity: 1, it might then provision an On-Demand instance. But with 0s, it will simply wait for a Spot instance to become available or fail to schedule the pod.
The interruptionQueue is crucial for managing Spot interruptions. Karpenter integrates with EC2 Spot Fleet interruption notices. When a Spot instance is about to be terminated, EC2 sends a message to the specified SQS queue. Karpenter reads this message, identifies the affected node, and then gracefully drains the pods from that node before it’s terminated. It then launches a replacement node (again, ideally a Spot instance) to pick up the workload.
The "right-sizing" aspect comes from Karpenter’s intelligence. It doesn’t just launch generic instance types. When a pod requests resources (e.g., cpu: 100m, memory: 128Mi), Karpenter considers not only the pod’s request but also the overhead of Kubernetes itself, the instance type’s available resources, and your Provisioner’s instanceTypes and requirements. It aims to find the smallest instance that can satisfy the pod’s requirements and any other unscheduled pods in the same zone. This prevents over-provisioning, where you pay for more CPU or memory than your applications actually use.
For instance, if you have many small pods requesting 100m CPU and 128Mi memory, Karpenter might launch a t3.small instance. If you had a single large pod requesting 4CPU and 8Gi memory, it might launch a m5.xlarge. It intelligently consolidates pods onto nodes to maximize resource utilization.
The most surprising thing about Karpenter is how it handles fragmentation. Unlike traditional autoscalers that might spin up a whole new node group just for a few pods that fit a specific instance type, Karpenter can launch a single instance of any type you’ve allowed, in any zone, to satisfy immediate pod scheduling needs. This means you can run a diverse set of workloads (CPU-intensive, memory-intensive, GPU-accelerated) on a single EKS cluster using primarily Spot instances, without needing to pre-configure multiple, rigid node groups.
Once Karpenter is managing your Spot nodes and right-sizing them, the next challenge you’ll face is managing the lifecycle of your Spot-provisioned instances, especially when you need to upgrade Kubernetes or change instance types.