CoreDNS doesn’t just blindly forward your DNS queries; it needs to know who to ask. This is where upstream DNS resolvers come in, and configuring them correctly is crucial for your cluster’s connectivity.

Let’s see CoreDNS in action, resolving a simple google.com query when configured with Google’s public DNS servers (8.8.8.8 and 8.8.4.4) as upstreams.

# Assuming CoreDNS is running and you can query it via kubectl
kubectl exec -n kube-system <coredns-pod-name> -- nslookup google.com

# Expected Output (will vary based on actual IP, but shows resolution):
# Server:    10.43.0.10
# Address:   10.43.0.10#53
#
# Non-authoritative answer:
# Name:	google.com
# Address: 142.250.190.14

The forward plugin in CoreDNS is your primary tool here. It tells CoreDNS where to send queries it can’t answer itself.

Here’s a typical Corefile snippet for configuring upstream resolvers:

.:53 {
    errors
    health {
       lameduck 5s
    }
    ready
    kubernetes cluster.local in-addr.arpa ip6.arpa {
       pods insecure
       fallthrough in-addr.arpa ip6.arpa
    }
    prometheus :9153
    forward . 8.8.8.8 8.8.4.4 {
       # Optional: Policy for selecting upstreams
       policy random
       # Optional: Timeout for upstream queries
       timeout 5s
       # Optional: Health check interval for upstreams
       health_check 10s
    }
    cache 30
    loop
    reload
    loadbalance
}

In this example:

  • . signifies all domain names.
  • 8.8.8.8 and 8.8.4.4 are the IP addresses of the upstream DNS servers.
  • policy random means CoreDNS will pick one of the upstreams at random for each query. Other policies include round_robin (sequential) and sequential (tries in order).
  • timeout 5s sets a 5-second limit for waiting for an upstream response.
  • health_check 10s tells CoreDNS to periodically check if the upstream servers are still responsive.

The kubernetes plugin handles your internal cluster DNS resolution (services, pods). When a query doesn’t match the cluster.local domain, CoreDNS moves on to the next plugin, which is forward. The forward plugin then takes that query and sends it to the configured upstream resolvers. If an upstream server responds, CoreDNS relays that answer back to the client. If no upstream responds within the timeout, CoreDNS will return a SERVFAIL or NXDOMAIN, depending on the situation.

The kubernetes plugin’s fallthrough directive is important. It means if the kubernetes plugin doesn’t have an answer for a query (e.g., it’s not an internal cluster name), it will pass that query down to the next plugin in the chain, which is forward. This is how external DNS resolution works.

The cache plugin, placed after forward, stores the responses from upstreams for a configurable duration (30 seconds here). This significantly speeds up subsequent requests for the same domain and reduces load on your upstreams. Without a cache, every single external DNS query would go through the forward plugin and out to your upstreams.

When you configure forward . <IP_ADDRESS>, you’re essentially telling CoreDNS, "For any domain name that I don’t explicitly handle (like cluster.local), send it to these IP addresses." The . is a wildcard for all domains. If you wanted to forward only specific domains to specific upstreams, you’d use multiple forward blocks, like forward example.com 1.1.1.1 and forward another.com 9.9.9.9.

Many cloud providers offer specific DNS server IPs for their managed Kubernetes services. For example, AWS EKS often uses the VPC’s DNS resolver, which can be found via the DHCP options set for your VPC (typically 169.254.169.253). If you’re running on-premises, you’d typically forward to your existing internal DNS servers or public resolvers like 1.1.1.1 or 8.8.8.8.

You might encounter situations where CoreDNS needs to resolve its own upstream servers. In such cases, it might recursively try to resolve the upstream server’s IP itself, leading to a loop if not handled carefully. The loop plugin is designed to detect and break these infinite recursion loops. If CoreDNS detects that it’s trying to resolve a name that’s already in its current query path, it will stop and return an error.

The reload plugin is essential for dynamically updating your Corefile without restarting the CoreDNS pods. When you change your Corefile (e.g., to update upstream IPs), CoreDNS will detect the change and reload its configuration. This means your upstream DNS configuration changes are applied live.

The health_check directive within the forward plugin is how CoreDNS ensures it’s not wasting time on unresponsive upstream servers. It periodically sends a small query (like a ping) to each upstream IP. If an upstream fails a health check, CoreDNS will temporarily stop sending queries to it. Once it starts responding again, it’s added back into the rotation. This prevents your cluster from experiencing DNS resolution delays or failures because of a single, temporarily down upstream DNS server.

After fixing your upstream DNS resolvers, you’ll likely run into issues with your network policies if they aren’t correctly configured to allow egress traffic to your chosen upstream DNS servers.

Want structured learning?

Take the full Coredns course →