Crossplane providers are just normal Kubernetes controllers, but they need to talk to cloud provider APIs to manage cloud resources. To do this, they need credentials.
Here’s how you manage those credentials securely.
The Problem: How Do Providers Get Cloud Credentials?
Providers need API keys, service account tokens, or other credentials to authenticate with cloud APIs. If you hardcode these into the provider deployment, or store them insecurely, you’re creating a massive security risk. Anyone who can read the provider’s deployment manifest or its running pods can steal those credentials and gain access to your cloud environment.
The Solution: Kubernetes Secrets
The idiomatic Kubernetes way to handle sensitive information like API keys is using Secrets. Crossplane providers are designed to consume these Secrets.
-
Create a Kubernetes Secret: First, you create a
Secretobject in your Kubernetes cluster that holds the cloud provider credentials. The key within the secret must match what the specific provider expects.For example, for an AWS provider, you might create a secret like this:
apiVersion: v1 kind: Secret metadata: name: aws-credentials namespace: crossplane-system # Or wherever your provider is installed type: Opaque data: # The key 'creds' is a common convention, but check provider docs creds: ewogICJhcm4iOiAiYXJuOmF3czppYW06OjEyMzQ1Njc4OTA5MDpSZXNvdXJjZUFjY291bnQvYWRlZmF1bHQvODc2NTQzMjEwOTg3OjEyMzQ1Njc4OTAxMjM0NTY3ODkwIgp9Self-correction: The above
credsvalue is a Base64 encoded JSON string. The actual content should be the AWS access key ID and secret access key. A more typical secret for AWS looks like this:apiVersion: v1 kind: Secret metadata: name: aws-credentials namespace: crossplane-system type: Opaque data: # Base64 encoded AWS_ACCESS_KEY_ID AWS_ACCESS_KEY_ID: QUtJQU5ZRVhZOVVWV09EQ0hNTQ== # Base64 encoded AWS_SECRET_ACCESS_KEY AWS_SECRET_ACCESS_KEY: c2VjcmV0LWV4YW1wbGUtYWRtaW5pc3RyYXRvci1hY2Nlc3Mta2V5Wait, how do I get that Base64 string? You can use the
kubectlcommand:# For access key ID echo -n "YOUR_ACCESS_KEY_ID" | base64 # For secret access key echo -n "YOUR_SECRET_ACCESS_KEY" | base64Replace
YOUR_ACCESS_KEY_IDandYOUR_SECRET_ACCESS_KEYwith your actual AWS credentials. -
Configure the Provider to Use the Secret: When you install a Crossplane provider (e.g.,
provider-aws), you specify whichSecretit should use for credentials. This is done within theProviderConfigcustom resource.Here’s an example
ProviderConfigfor AWS:apiVersion: aws.upbound.io/v1beta1 kind: ProviderConfig metadata: name: default spec: credentials: source: Secret secretRef: namespace: crossplane-system name: aws-credentials key: AWS_ACCESS_KEY_ID # This is the *key name* in the secret, not the secret's data keySelf-correction: The
spec.credentials.secretRef.keyfield needs to be more nuanced. It often refers to a specific key within the secret that contains the entire credential payload, or it can refer to individual keys if the provider supports it. The AWS provider, for instance, can readAWS_ACCESS_KEY_IDandAWS_SECRET_ACCESS_KEYdirectly from the secret if they exist. If you had a single key likecredsthat contained a JSON blob withaccessKeyIdandsecretAccessKey, you’d specify that.Let’s refine the
ProviderConfigfor the common case where the secret has individual keys:apiVersion: aws.upbound.io/v1beta1 kind: ProviderConfig metadata: name: default spec: credentials: source: Secret secretRef: namespace: crossplane-system name: aws-credentials # No 'key' specified here means the provider will look for standard keys like # AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY within the secret named 'aws-credentials'.If your secret was structured differently, for example, using a single
credskey containing a JSON payload:# Secret manifest apiVersion: v1 kind: Secret metadata: name: aws-creds-json namespace: crossplane-system type: Opaque data: creds: eyJhY2Nlc3NLZXlJZCI6ICJBS0lBTllFWjlVVldPRENIS01CIiwic2VjcmV0QWNjZXNLZXkiOiAiYWRtaW5pc3RyYXRvci1hY2Nlc3Mta2V5LXdpdGgtbG9uZy1zZWNyZXQifQ== # Base64 of {"accessKeyId": "AKI...","secretAccessKey": "admin..."}Then your
ProviderConfigwould specify thekey:apiVersion: aws.upbound.io/v1beta1 kind: ProviderConfig metadata: name: default spec: credentials: source: Secret secretRef: namespace: crossplane-system name: aws-creds-json key: creds # This tells the provider to read the 'creds' key from the secret
Why This is Secure
- Separation of Concerns: The
Secretis a Kubernetes object for managing sensitive data. TheProviderConfigis a Crossplane object for configuring providers. They are distinct. - Access Control: Kubernetes RBAC can restrict who can read or create
Secrets. You can ensure only specific service accounts (like the one the Crossplane provider runs as) or authorized users can access the credentialSecret. - No Hardcoding: Credentials are never written directly into YAML manifests for the provider deployment. They are managed as a separate, protected resource.
- Dynamic Updates: If credentials need to be rotated, you update the
Secretobject. The Crossplane provider will automatically pick up the new credentials (after a short delay, or potentially a restart if the provider doesn’t dynamically re-read).
Other Cloud Providers
The principle is the same for other providers (GCP, Azure, etc.), but the structure of the secret and the expected keys will differ.
-
GCP: You’ll typically use a service account key, often provided as a JSON file. The
Secretwould contain this JSON, and theProviderConfigwould point to theSecretand the key holding the JSON.# Example GCP Secret apiVersion: v1 kind: Secret metadata: name: gcp-credentials namespace: crossplane-system type: Opaque data: service-account-key.json: eyJ0eXBlIjogInNlcnZpY2VlQWNjb3VudCIsICJwcm9qZWN0X2lkIjogIm15LXByb2plY3QiLCAic...# Example GCP ProviderConfig apiVersion: gcp.upbound.io/v1beta1 kind: ProviderConfig metadata: name: default spec: credentials: source: Secret secretRef: namespace: crossplane-system name: gcp-credentials key: service-account-key.json # The key in the secret holding the JSON -
Azure: You’ll use a Service Principal. The
Secretwill containAZURE_TENANT_ID,AZURE_CLIENT_ID, andAZURE_CLIENT_SECRET.# Example Azure Secret apiVersion: v1 kind: Secret metadata: name: azure-credentials namespace: crossplane-system type: Opaque stringData: # Using stringData for clarity, kubectl will convert to base64 AZURE_TENANT_ID: "YOUR_TENANT_ID" AZURE_CLIENT_ID: "YOUR_CLIENT_ID" AZURE_CLIENT_SECRET: "YOUR_CLIENT_SECRET"# Example Azure ProviderConfig apiVersion: azure.upbound.io/v1beta1 kind: ProviderConfig metadata: name: default spec: credentials: source: Secret secretRef: namespace: crossplane-system name: azure-credentials # The provider looks for AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET in the secret.
The Next Step: Storing Secrets in a Vault
While Kubernetes Secrets are good, for highly sensitive environments, you might want to store your cloud credentials in a dedicated secrets management system like HashiCorp Vault or AWS Secrets Manager. Crossplane providers can be configured to pull credentials directly from these external vaults, meaning the credentials never touch Kubernetes Secrets at all. This is often done via the credentials.source: ExternalSecret option in the ProviderConfig.