Drone CI failed to deploy a Helm chart because the Kubernetes API server was unreachable from the Drone agent.
This usually happens for one of a few reasons:
-
Incorrect Kubernetes Context/Credentials: The most common culprit is that the
kubectlorhelmcommand inside your Drone pipeline doesn’t have the right information to talk to your Kubernetes cluster. This means the service account your Drone agent is using, or the kubeconfig file it’s trying to read, is misconfigured or missing critical permissions.- Diagnosis: Run
kubectl config get-contextsandkubectl config current-contextwithin a similar environment (or locally if you can replicate the kubeconfig). Ensure the context points to your target cluster and that you cankubectl get nodessuccessfully. In Drone, check your environment variables and secrets to ensureKUBECONFIG(if used) is set correctly or that the service account token mounted by the pod has the necessary permissions. - Fix: If using a service account, ensure it has the
cluster-adminrole or at least roles grantingcreate,update,patch, anddeletepermissions ondeployments,services,ingresses,secrets, andconfigmapswithin the target namespace. If using a kubeconfig file, ensure it’s securely passed as a Drone secret and that theKUBECONFIGenvironment variable is set to point to it. - Why it works: Kubernetes relies on authenticated and authorized API calls. Without the correct credentials or permissions, the agent can’t even initiate a conversation with the cluster.
- Diagnosis: Run
-
Network Policies Blocking Egress: Your Kubernetes cluster might have Network Policies configured that prevent pods in the namespace where the Drone agent runs from making outbound connections to the Kubernetes API server’s IP address and port (usually 443).
- Diagnosis: Check for Network Policies in the namespace where your Drone agent is deployed. Use
kubectl get networkpolicy -n <drone-agent-namespace>and examine their rules. - Fix: Add an egress rule to the Network Policy that allows outbound traffic to the Kubernetes API server’s IP and port. This might involve allowing traffic to
0.0.0.0/0on port443if the API server is publicly accessible, or to a specific CIDR range if it’s internal. - Why it works: Network Policies act as firewalls within Kubernetes. If the policy doesn’t explicitly permit egress to the API server, the connection will be silently dropped.
- Diagnosis: Check for Network Policies in the namespace where your Drone agent is deployed. Use
-
DNS Resolution Issues: The Drone agent pod might not be able to resolve the hostname of your Kubernetes API server. This can happen if the cluster’s DNS service (like CoreDNS) isn’t configured correctly or if the agent pod’s DNS configuration is broken.
- Diagnosis: Exec into the Drone agent pod during a failed pipeline run and try to
pingornslookupyour Kubernetes API server’s hostname. You can find the API server hostname by looking at yourkubectl config viewoutput or by checking your Kubernetes control plane configuration. - Fix: Ensure the
kube-dnsorcorednsservice in thekube-systemnamespace is running and healthy. If you’re running Drone in a managed Kubernetes service (EKS, GKE, AKS), check their specific documentation for DNS setup. For self-hosted clusters, verify yourcorednsdeployment and its ConfigMap. - Why it works: DNS is how names are translated into IP addresses. If this translation fails, the agent can’t find the API server to connect to.
- Diagnosis: Exec into the Drone agent pod during a failed pipeline run and try to
-
RBAC Permissions for Helm Tiller (if applicable) or Helm Commands: While Helm v3 doesn’t use Tiller, older versions did. If you’re on Helm v2, the service account needs permissions for Tiller. Even with Helm v3, the service account needs permissions to interact with Kubernetes resources that Helm will manage (deployments, services, etc.).
- Diagnosis: If using Helm v2, check if Tiller is running and if its associated service account has the correct RBAC roles. For Helm v3, check the RBAC roles bound to the service account your Drone agent uses. Run
kubectl auth can-i --listfor the service account to see its granted permissions. - Fix: For Helm v2, grant
cluster-adminor equivalent RBAC permissions to the Tiller service account. For Helm v3, ensure the service account has permissions tocreate,get,list,watch,update,patch, anddeleteondeployments,services,ingresses,secrets,configmaps, andjobsin the target namespace. - Why it works: Helm is just a client that talks to the Kubernetes API. The API server enforces permissions based on the identity (service account) of the client making the request.
- Diagnosis: If using Helm v2, check if Tiller is running and if its associated service account has the correct RBAC roles. For Helm v3, check the RBAC roles bound to the service account your Drone agent uses. Run
-
Incorrect Kubernetes API Server Address: The Helm
values.yamlor pipeline configuration might be pointing to the wrong Kubernetes API server address, or the address itself is inaccessible from the Drone agent’s network.- Diagnosis: Check the
kubernetes.io/client-certificate-data,kubernetes.io/client-key-data, andclusters.cluster.serverfields in your kubeconfig, or any explicitserversettings in your Helm chart’svalues.yamlor pipeline configuration. Verify that this server address is reachable from where your Drone agent is running. - Fix: Update the kubeconfig or Helm values to use the correct Kubernetes API server address. Ensure that if the API server is internal, the Drone agent has network access to it.
- Why it works: You need to tell Helm/kubectl precisely where to find the API server. If that address is wrong or unreachable, the connection will fail.
- Diagnosis: Check the
-
Resource Constraints on the Drone Agent Pod: The Drone agent pod might be starved of CPU or memory, preventing its network stack from functioning correctly or causing the Helm/kubectl process to crash before it can establish a connection.
- Diagnosis: Check the resource requests and limits for your Drone agent pod. Monitor the pod’s resource utilization in Kubernetes. Look for
OOMKilledevents or high CPU/memory usage inkubectl top pod <drone-agent-pod-name> -n <namespace>. - Fix: Increase the CPU and memory requests/limits for the Drone agent deployment in your Kubernetes cluster.
- Why it works: Insufficient resources can lead to general instability and unresponsiveness in the pod, including its networking capabilities.
- Diagnosis: Check the resource requests and limits for your Drone agent pod. Monitor the pod’s resource utilization in Kubernetes. Look for
The next error you’ll likely encounter after fixing connectivity is a Forbidden error, indicating that while the API server is reachable, the service account lacks the necessary RBAC permissions to perform the requested Helm operations.