Vault isn’t just about storing secrets; it’s about making them dynamic and auditable, fundamentally changing how we think about application credentials in a CI/CD pipeline.
Let’s say you have a web app that needs to talk to a database. Traditionally, you’d hardcode the database password in a config file, or worse, in the application code itself. This is a nightmare for security and rotation.
Here’s a simplified Python Flask app that needs a database password:
from flask import Flask, request, jsonify
import psycopg2 # Example for PostgreSQL
import os
app = Flask(__name__)
# This is where the magic happens - fetching from Vault
def get_db_password():
# In a real app, this would use the hvac library to talk to Vault
# For demonstration, we'll simulate fetching it.
# In production, Vault would be queried via its API.
return os.environ.get("DB_PASSWORD", "default_insecure_password")
@app.route('/users', methods=['GET'])
def get_users():
db_password = get_db_password()
try:
conn = psycopg2.connect(
dbname="mydatabase",
user="myuser",
password=db_password,
host="db.example.com"
)
cur = conn.cursor()
cur.execute("SELECT id, name FROM users;")
users = cur.fetchall()
cur.close()
conn.close()
return jsonify(users)
except psycopg2.Error as e:
return jsonify({"error": f"Database error: {e}"}), 500
if __name__ == '__main__':
app.run(debug=True)
Now, let’s see how Vault fits in. Vault provides a secret engine, often the Key/Value (KV) v2 engine, to store secrets.
A typical Vault KV v2 structure looks like this:
secret/
├── data/
│ └── myapp/
│ └── database/
│ └── config
└── metadata/
└── myapp/
└── database/
└── config
When you write a secret to secret/data/myapp/database/config, Vault stores it with versioning. The data path is where the actual secret content lives, and metadata tracks versions, timestamps, and other info.
Here’s how you’d write a secret using the Vault CLI:
vault kv put secret/myapp/database/config username="appuser" password="supersecretpassword123" dbname="mydatabase"
This command writes the username, password, and dbname into the secret/data/myapp/database/config path. Vault automatically assigns it a version.
To retrieve this secret, your application (or the CI/CD system it’s running in) would authenticate with Vault and then make a read request.
Vault’s primary value proposition is dynamic secrets. Instead of a static password that never changes, Vault can generate credentials on demand for systems like databases, cloud providers, or Kubernetes.
For example, you can configure Vault to dynamically generate PostgreSQL credentials. When your application requests a database secret for the myapp role, Vault’s PostgreSQL secrets engine will connect to your PostgreSQL server, create a new user with a randomly generated password, grant it specific privileges for a limited time, and then return those credentials to your application. When the lease expires, Vault automatically revokes the generated user and password.
This is a game-changer. It means you don’t have to worry about password rotation for these dynamically generated secrets; Vault handles it. Your application only needs to know how to authenticate to Vault, not how to generate or manage database users.
The authentication methods are crucial. Your application or CI/CD agent needs a way to prove its identity to Vault. Common methods include:
- AppRole: A role ID and secret ID combination, suitable for applications and services.
- Kubernetes Auth: Uses Kubernetes service account tokens to authenticate.
- AWS IAM Auth: Uses EC2 instance metadata or IAM role credentials.
- Token Auth: The simplest but least secure for long-term use, often used for initial bootstrapping.
Let’s say your Flask app is running inside Kubernetes. You’d configure the Kubernetes Auth method in Vault and then create an auth_method for your app’s Kubernetes service account. The app, when starting, would read its service account token from /var/run/secrets/kubernetes.io/serviceaccount/token and send it to Vault’s Kubernetes auth endpoint. If Vault verifies the token and the service account is authorized, it issues a Vault token to the application. This Vault token is then used for subsequent secret reads.
The hvac Python library is the standard way to interact with Vault from Python applications.
import hvac
import os
def get_db_password_from_vault():
client = hvac.Client(url=os.environ.get("VAULT_ADDR", "http://127.0.0.1:8200"))
# Authenticate (example using AppRole)
# In a real scenario, this would be managed more securely, e.g., via environment variables
# or a dedicated Kubernetes service account
try:
client.auth.approle.login(
role_id=os.environ.get("VAULT_ROLE_ID"),
secret_id=os.environ.get("VAULT_SECRET_ID")
)
except hvac.exceptions.InvalidRequest as e:
print(f"Vault authentication failed: {e}")
return None
if not client.is_authenticated():
print("Vault client not authenticated.")
return None
# Read the secret
try:
secret = client.secrets.kv.v2.read_secret_version(
path='secret/myapp/database/config'
)
return secret['data']['data']['password']
except Exception as e:
print(f"Failed to read secret from Vault: {e}")
return None
# In your Flask app:
# db_password = get_db_password_from_vault()
# if db_password:
# # ... proceed with database connection
The most surprising aspect of Vault is how it encourages a "zero-trust" approach to secrets, even within your own infrastructure. Instead of securing a central secret store with network firewalls and complex access controls, Vault’s dynamic secrets and short-lived tokens mean that even if a Vault token is compromised, its usefulness is severely limited by its short TTL and the granular permissions assigned to its authentication method.
Beyond dynamic secrets, Vault also offers encryption as a service. You can use Vault to encrypt and decrypt data without ever exposing the decryption key to your application. This is useful for encrypting sensitive fields in a database before they are written, or for encrypting sensitive files.
The next step after mastering KV secrets and dynamic secrets is often exploring Vault’s Transit secrets engine for encryption-as-a-service.