CoreDNS and kube-proxy are essential components for EKS cluster networking, but managing them as EKS Managed Add-ons can sometimes lead to unexpected behavior.
CoreDNS and kube-proxy as EKS Managed Add-Ons
The most surprising thing about EKS Managed Add-ons for CoreDNS and kube-proxy is that while AWS manages their lifecycle (creation, updates, deletion), you still retain significant control over their configuration, often through mechanisms that can feel a bit like a shell game.
Let’s see what this looks like in practice. Imagine you’ve just launched an EKS cluster and want to inspect the CoreDNS configuration. You’d typically look for a ConfigMap.
kubectl get configmap coredns -n kube-system -o yaml
This will show you the default CoreDNS configuration provided by EKS. It might look something like this:
apiVersion: v1
data:
Corefile: |
.:53 {
errors
health {
lameduck 5s
}
ready
kubernetes cluster.local in-addr.arpa ip6.arpa {
pods insecure
upstream
fallthrough in-addr.arpa ip6.arpa
}
prometheus :9153
forward . /etc/resolv.conf {
max_concurrent 1000
}
cache 30
loop
reload
loadbalance
}
kind: ConfigMap
metadata:
creationTimestamp: "2023-10-27T10:00:00Z"
name: coredns
namespace: kube-system
resourceVersion: "12345"
uid: abcdef12-3456-7890-abcd-ef1234567890
Now, if you want to tweak this, say, to change the upstream DNS servers that CoreDNS forwards to, you can edit this ConfigMap.
kubectl edit configmap coredns -n kube-system
You’d modify the forward directive. For instance, to use Google’s public DNS servers (8.8.8.8 and 8.8.4.4) instead of /etc/resolv.conf:
forward . 8.8.8.8 8.8.4.4 {
max_concurrent 1000
}
After saving, CoreDNS will automatically pick up this change due to the reload plugin configured in the Corefile. You can verify the pods have restarted or the new configuration is active by checking the CoreDNS logs:
kubectl logs -n kube-system -l k8s-app=kube-dns
Similarly, for kube-proxy, while it doesn’t have a directly editable ConfigMap in the same way as CoreDNS for its core parameters, its configuration is managed through the EKS add-on API. You can view its current configuration and desired version:
aws eks describe-addon --cluster-name your-cluster-name --addon-name kube-proxy
This command will output details about the kube-proxy add-on, including its status, version, and potentially a configurationValues field if you’ve explicitly set any custom parameters. If you need to change kube-proxy’s behavior, like its cluster-cidr or iptables-sync-period, you’d typically do so via the aws eks update-addon command, providing a configuration-values-json file.
For example, to update the sync period:
aws eks update-addon --cluster-name your-cluster-name --addon-name kube-proxy --addon-version <new-version> --configuration-values-json '{"kubeProxyAgent": {"clusterCIDR": "10.100.0.0/16", "iptablesSyncPeriod": "10s"}}'
This tells EKS to update the kube-proxy add-on to the specified version and apply these specific configuration overrides. EKS then handles the rollout of the new kube-proxy pods with the updated configuration.
The mental model here is that EKS Managed Add-ons are a layer of abstraction. For CoreDNS, the abstraction is that EKS ensures the coredns ConfigMap is present and the kube-dns Deployment is managed, but you can directly manipulate the ConfigMap. For kube-proxy, the abstraction is stronger; you interact with the add-on API to effect changes, and EKS orchestrates the underlying pod updates.
The key problem this solves is providing a managed, up-to-date, and secure set of core networking components without requiring users to manually deploy and manage the Kubernetes DNS and service proxy. EKS handles the patching and upgrades of these components, reducing operational burden.
When you modify the CoreDNS ConfigMap directly, you’re essentially telling the managed add-on to use your provided configuration. EKS will still manage the lifecycle of the CoreDNS pods (e.g., ensuring they are running, updating them to new versions), but it will respect the Corefile you’ve defined. The reload plugin in CoreDNS is crucial here; it watches the ConfigMap for changes and reloads the configuration without requiring a pod restart, making your edits nearly instantaneous.
For kube-proxy, the managed add-on approach means you don’t directly edit a Deployment or ConfigMap. Instead, you use the AWS CLI or SDK to tell EKS what configuration you want. EKS then takes that desired state and applies it to the kube-proxy pods it manages. This ensures that the kube-proxy version and configuration are always consistent with what EKS expects for that add-on version.
One aspect that often trips people up is that if you create a new ConfigMap for CoreDNS with a different name, EKS might not pick it up because it specifically looks for the coredns ConfigMap in the kube-system namespace to manage the add-on. Similarly, if you try to manually update the kube-proxy Deployment, EKS will likely overwrite your changes to enforce the managed add-on configuration. You must work with the managed add-on’s intended configuration points.
The next logical step after ensuring your CoreDNS and kube-proxy are correctly configured is to understand how to monitor their health and performance within the EKS environment.