A Kubernetes manifest validation failure means your kubectl or Argo CD is refusing to apply a configuration because it violates the Kubernetes API server’s rules.
Common Causes and Fixes
-
Invalid
apiVersionorkind: The API server doesn’t recognize the combination ofapiVersionandkindin your manifest. This often happens when you’re using a feature from a newer Kubernetes version on an older cluster, or if you’ve mistyped the name.-
Diagnosis:
kubectl api-resources | grep <your_kind>This command lists all available resource types and their API versions. Check if your
apiVersionandkindmatch an entry here. For example, if you havekind: DeploymentandapiVersion: apps/v1,kubectl api-resourcesshould showdeploymentswithapps/v1. -
Fix: Update the
apiVersionorkindin your manifest to match what your cluster supports. For instance, if you seeapiVersion: networking.k8s.io/v1beta1forIngressbut your cluster only supportsnetworking.k8s.io/v1, change it.# Incorrect: apiVersion: networking.k8s.io/v1beta1 kind: Ingress # ... # Correct: apiVersion: networking.k8s.io/v1 kind: Ingress # ... -
Why it works: The Kubernetes API server uses
apiVersionandkindto route your request to the correct controller and validate the object’s schema. If it doesn’t recognize these, it can’t even begin to process the request.
-
-
Schema Validation Errors: The manifest contains fields that are not defined in the schema for the specified
apiVersionandkind, or the values provided are of the wrong type or format.-
Diagnosis: Use
kubectl explain <kind>.<field_path>to understand the expected schema. For example:kubectl explain deployment.spec.template.spec.containers kubectl explain service.spec.ports.protocolCompare the output with your manifest.
-
Fix: Correct the field names, data types, or values according to the schema. For instance, if you specified
replicaCountfor a Deployment (common in Helm charts but not a direct Kubernetes field), you should usereplicas.# Incorrect: apiVersion: apps/v1 kind: Deployment spec: replicaCount: 3 # Not a valid Kubernetes field for Deployment template: spec: containers: - name: my-container image: nginx # ... # Correct: apiVersion: apps/v1 kind: Deployment spec: replicas: 3 # Use 'replicas' for Deployment template: spec: containers: - name: my-container image: nginx # ... -
Why it works: The API server validates incoming objects against a predefined OpenAPI schema. If your manifest deviates, it’s rejected before being stored in etcd.
-
-
Invalid Resource Names or Labels: Resource names or labels might contain invalid characters or exceed length limits. Kubernetes names generally must be DNS-1123 compliant (lowercase alphanumeric characters,
-, and., starting and ending with an alphanumeric character) and have length constraints.-
Diagnosis: Examine the
metadata.nameandmetadata.labelsfields in your manifest. Check against Kubernetes naming conventions.# Example check for a name echo "my-invalid-resource-name!" | grep -E '^[a-z0-9]([-a-z0-9]*[a-z0-9])?$' echo "my-invalid-resource-name!" | wc -c # Check length -
Fix: Rename resources and labels to conform to the required format. For example, change
my-app-servicetomy-app-service(no exclamation mark) and ensure it’s within the 253-character limit.# Incorrect: apiVersion: v1 kind: Service metadata: name: my-service-! # Invalid character '!' # ... # Correct: apiVersion: v1 kind: Service metadata: name: my-service-valid # Valid name # ... -
Why it works: Resource names and labels are used by Kubernetes for identification and selection. Invalid characters or lengths can break internal indexing and lookup mechanisms.
-
-
Incorrectly Structured
selectororlabelsMismatch: For controllers like Deployments, StatefulSets, or Services, theselectorin the controller’s spec must match thelabelson the Pod template.-
Diagnosis: Compare
spec.selector.matchLabels(ormatchExpressions) of the controller withspec.template.metadata.labelsof the Pod template.# Example for Deployment apiVersion: apps/v1 kind: Deployment spec: selector: matchLabels: app: my-app # This must match template labels template: metadata: labels: app: my-app # Correctly matches selector spec: containers: - name: my-container image: nginx -
Fix: Ensure the labels in
spec.selector.matchLabelsexactly match the labels defined inspec.template.metadata.labels.# Incorrect: apiVersion: apps/v1 kind: Deployment spec: selector: matchLabels: app: wrong-app # Mismatch template: metadata: labels: app: my-app # This is what pods will have # ... # Correct: apiVersion: apps/v1 kind: Deployment spec: selector: matchLabels: app: my-app # Now matches template: metadata: labels: app: my-app # ... -
Why it works: The controller uses its
selectorto find and manage Pods. If they don’t match, the controller can’t find the Pods it’s supposed to manage, leading to reconciliation loops or outright validation errors indicating a broken relationship.
-
-
Missing Required Fields: Some resource types or specific configurations require certain fields to be present.
-
Diagnosis: Use
kubectl explain <kind> --recursiveorkubectl explain <kind>.spec.<field>to identify mandatory fields. The documentation will often explicitly state if a field is required. -
Fix: Add the missing required fields. For example, a
PersistentVolumeClaimneedsspec.resources.requests.storageifstorageClassNameis not set toNoneand the cluster doesn’t have a default StorageClass.# Incorrect (missing storage request): apiVersion: v1 kind: PersistentVolumeClaim metadata: name: my-pvc spec: accessModes: - ReadWriteOnce # resources: # requests: # storage: 1Gi # Missing # ... # Correct: apiVersion: v1 kind: PersistentVolumeClaim metadata: name: my-pvc spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi # Added # ... -
Why it works: The API server enforces that all mandatory fields are populated to ensure the object can be correctly created and managed by Kubernetes.
-
-
Invalid Enum Values: Fields that expect a specific set of predefined values (enums) might be given an unsupported value.
-
Diagnosis: Check
kubectl explain <kind>.<field>for the allowed values. For instance,Service.spec.typecan beClusterIP,NodePort,LoadBalancer, orExternalName. -
Fix: Change the invalid enum value to one of the allowed options.
# Incorrect: apiVersion: v1 kind: Service spec: selector: app: my-app ports: - protocol: TCP port: 80 targetPort: 9376 type: MyCustomType # Invalid enum value # ... # Correct: apiVersion: v1 kind: Service spec: selector: app: my-app ports: - protocol: TCP port: 80 targetPort: 9376 type: ClusterIP # Valid enum value # ... -
Why it works: Enum fields are strictly validated to ensure that the component consuming them can correctly interpret the value.
-
The next error you’ll likely hit is a ControllerNotFound or a ResourceQuotaExceeded if all your manifest syntax is correct but the cluster is constrained.