SOPS and ArgoCD bring secret management to GitOps, but they don’t magically make secrets disappear from your Git history.
Here’s how you can use SOPS to encrypt secrets and manage them securely with ArgoCD:
The Core Idea
Instead of storing plain-text secrets in Kubernetes manifests (which is a terrible idea), you encrypt them using SOPS. These encrypted files live in your Git repository. ArgoCD, with a little help, can then decrypt these secrets before applying them to your cluster.
Setting Up SOPS
-
Install SOPS: You’ll need SOPS installed locally for encrypting and on your ArgoCD cluster (or the ArgoCD Application Controller pod) for decryption.
# Example for macOS with Homebrew brew install sops -
Choose an Encryption Method: SOPS supports several backends: KMS, GCP KMS, Azure Key Vault, AWS Secrets Manager, age, and PGP. For simplicity and common use cases, let’s focus on KMS or PGP.
-
AWS KMS: Requires an AWS profile configured with permissions to use KMS.
# Create a KMS key if you don't have one aws kms create-key --description "SOPS encryption key" # Note down the KeyId (e.g., arn:aws:kms:us-east-1:123456789012:key/abcdef12-3456-7890-abcd-abcdef123456) # Encrypt a file sops --kms arn:aws:kms:us-east-1:123456789012:key/abcdef12-3456-7890-abcd-abcdef123456 --encrypt secrets.yaml > secrets.encrypted.yaml -
PGP: Requires you to have PGP keys generated.
# Generate a PGP key if you don't have one gpg --full-generate-key # Note down your PGP Key ID (e.g., 1234ABCD5678EFGH) # Encrypt a file sops --pgp 1234ABCD5678EFGH --encrypt secrets.yaml > secrets.encrypted.yaml
-
-
Create a SOPS Configuration File (
.sops.yaml): This file tells SOPS which encryption method to use by default and which files to encrypt. Place this in the root of your Git repository.# .sops.yaml creation_rules: - kms: arn:aws:kms:us-east-1:123456789012:key/abcdef12-3456-7890-abcd-abcdef123456 # Replace with your KMS key ARN # OR # - pgp: 1234ABCD5678EFGH # Replace with your PGP Key ID # Optionally, specify which files to encrypt # editable: # - secrets/**/*.yaml # - secrets/**/*.json -
Commit Encrypted Secrets: Add your
.sops.yamlfile and the encrypted secrets file (e.g.,secrets.encrypted.yaml) to Git. Never commit unencrypted secrets.git add .sops.yaml secrets.encrypted.yaml git commit -m "Add encrypted secrets" git push
Integrating SOPS with ArgoCD
ArgoCD needs to be able to decrypt these secrets when it applies them. The most common way to achieve this is by using a SOPS decryption plugin for ArgoCD.
-
Install the SOPS Plugin for ArgoCD: This involves adding a plugin configuration to your ArgoCD installation. You can do this via the ArgoCD CLI or by modifying the ArgoCD deployment.
-
Using ArgoCD CLI:
argocd plugin add sops-plugin \ --url https://github.com/argoproj-labs/sops-plugin \ --version v0.3.0 # Use the latest stable version -
Manual Deployment (Example for ArgoCD running in-cluster): You’ll need to modify the
argocd-application-controllerdeployment to include the SOPS plugin as a sidecar or an init container, or configure it as an external plugin. The official ArgoCD SOPS plugin documentation provides detailed instructions for various installation methods (e.g., Helm, Kustomize, manual deployment).A common approach is to configure ArgoCD to use an external plugin. You’d create a
ConfigMapandSecretfor the plugin and then reference them in theargocd-application-controller’s arguments.Key Configuration Points:
- The plugin needs access to the decryption keys. For KMS, this means the ArgoCD controller pod needs appropriate AWS credentials. For PGP, you’ll need to provide the private key securely (e.g., via a Kubernetes Secret).
- You’ll configure the plugin in
argocd-cmConfigMapunderplugins.yaml.
# Example snippet for argocd-cm ConfigMap (plugins.yaml) apiVersion: v1 kind: ConfigMap metadata: name: argocd-cm namespace: argocd # Or your ArgoCD namespace data: plugins.yaml: | apiVersion: argoproj.io/v1alpha1 kind: ArgoCDPlugin metadata: name: sops spec: # Define how ArgoCD discovers and runs the plugin # This is a simplified representation; refer to ArgoCD docs for full details discover: find: # Match files ending with .encrypted.yaml or .encrypted.json fileName: regex: ".*\\.encrypted\\.(yaml|json)" # How to execute the plugin generate: # Use the sops-plugin image exec: image: ghcr.io/argoproj-labs/sops-plugin:v0.3.0 # Use the version you installed command: ["python3", "plugin.py"] args: ["generate"]
-
-
Configure ArgoCD Application to Use the Plugin: In your ArgoCD
Applicationmanifest, you’ll tell it to use the SOPS plugin for specific files.apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: my-app namespace: argocd spec: project: default source: repoURL: YOUR_GIT_REPO_URL targetRevision: HEAD path: path/to/your/manifests # Specify the plugin to use for files matching a pattern plugin: name: sops # Optional: Pass arguments to the plugin if needed # args: # - "--some-flag" destination: server: https://kubernetes.default.svc namespace: my-app-namespace syncPolicy: automated: prune: true selfHeal: true -
How it Works:
- ArgoCD fetches your Git repository.
- When it encounters a file matching the plugin’s
fileNameregex (e.g.,secrets.encrypted.yaml), it invokes the SOPS plugin. - The SOPS plugin uses the configured decryption method (KMS, PGP, etc.) to decrypt the file in memory.
- The decrypted content is then passed to ArgoCD’s standard manifest processing pipeline.
- ArgoCD applies the decrypted Kubernetes manifests (including your secrets) to the cluster.
Managing Decryption Keys Securely in ArgoCD
- AWS KMS: Ensure the
argocd-application-controllerpod has an IAM role attached with permissions tokms:Decryptyour specific KMS key. This is the most secure method as it avoids storing keys directly in Kubernetes. - PGP: Store your PGP private key in a Kubernetes
Secretin the same namespace as your ArgoCD controller. Mount this secret as a volume into theargocd-application-controllerpod, and configure the SOPS plugin to read the key from that mounted file. - Other Backends: Follow the specific instructions for your chosen backend (GCP KMS, Azure Key Vault, etc.) for how to grant the ArgoCD controller access.
The "Magic" is in the Plugin
The SOPS plugin acts as an intermediary. It intercepts files ArgoCD would normally apply, decrypts them on the fly using your pre-configured secrets management backend, and then hands the plain-text Kubernetes objects back to ArgoCD. Your Git repository only ever stores encrypted blobs, and the decryption happens in the secure environment of your ArgoCD controller pod.
The next step is often integrating this with your CI/CD pipeline to automatically encrypt secrets before committing them.