Consul and Vault can dynamically inject secrets into your applications, eliminating the need to store credentials directly in your code or configuration files.
Here’s how it works:
Imagine you have a web application that needs to connect to a PostgreSQL database. Instead of hardcoding the database username and password, you want Vault to generate them on demand and Consul to make them available to your application.
1. Vault: The Secret Generator
First, you need to configure Vault to manage database credentials. Let’s say you’re using the database secrets engine for PostgreSQL:
# Enable the database secrets engine at the 'pg' path
vault secrets enable database
# Configure the PostgreSQL database connection
vault write database/config/connection \
plugin_name=postgresql \
connection_url="postgresql://vault:vaultpassword@localhost:5432/myappdb?sslmode=disable" \
allowed_roles="myapp-role"
# Create a role named 'myapp-role'
vault write database/roles/myapp-role \
db_name=myappdb \
creation_statements="CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}'; GRANT SELECT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";" \
default_ttl=1h \
max_ttl=24h
When your application requests credentials for myapp-role, Vault will:
- Generate a unique username and a strong, random password.
- Execute the
creation_statementsto create a new database user with those credentials and grant it specific privileges. - Return the generated username and password to the requester.
- Schedule the database user for deletion after the
default_ttlexpires.
2. Consul: The Secret Distributor
Consul acts as a service discovery and configuration registry. You can use its Key-Value (KV) store to hold references to Vault secrets or even the secrets themselves, though the latter is less common for dynamic secrets. A more robust pattern is to have your application query Vault directly, but Consul can orchestrate this.
A common pattern is to use Consul’s service registration to inform applications about services that can provide secrets, and then have the application logic connect to Vault. However, for simpler cases or to inject static config that points to dynamic secrets, Consul KV is useful.
Let’s say your application is registered in Consul as a service named webapp. You can define a configuration entry for it:
// Consul KV: /services/webapp/config
{
"vault_role": "myapp-role",
"vault_path": "database/creds/myapp-role"
}
Your webapp application, upon startup, would:
- Query Consul for its own configuration.
- Read
vault_roleandvault_path. - Use the Vault agent or the Vault client library to request credentials from Vault at the specified path.
3. The Application: Dynamic Injection
Your application code would contain logic like this (using the Vault Go client as an example):
package main
import (
"fmt"
"log"
"os"
"github.com/hashicorp/vault/api"
)
func main() {
// Initialize Vault client
config := api.DefaultConfig()
// In a real app, you'd get the address from config or env var
config.Address = "http://127.0.0.1:8200"
client, err := api.NewClient(config)
if err != nil {
log.Fatalf("unable to initialize Vault client: %v", err)
}
// Authenticate (e.g., using AppRole, Kubernetes auth, etc.)
// For simplicity, assuming token is available via env var or file
client.SetToken(os.Getenv("VAULT_TOKEN"))
// Read configuration from Consul (simulated here)
vaultRole := "myapp-role"
vaultPath := "database/creds/myapp-role"
// Request dynamic credentials from Vault
secret, err := client.Logical.Read(vaultPath)
if err != nil {
log.Fatalf("unable to read secret from Vault: %v", err)
}
if secret == nil || secret.Data == nil {
log.Fatalf("no data received from Vault")
}
// Extract username and password
username, ok := secret.Data["username"].(string)
if !ok {
log.Fatalf("username not found in secret data")
}
password, ok := secret.Data["password"].(string)
if !ok {
log.Fatalf("password not found in secret data")
}
fmt.Printf("Successfully retrieved dynamic credentials:\n")
fmt.Printf("Username: %s\n", username)
// In a real app, DO NOT print the password!
// fmt.Printf("Password: %s\n", password)
// Use these credentials to connect to your database
// dbConnectionString := fmt.Sprintf("postgresql://%s:%s@localhost:5432/myappdb?sslmode=disable", username, password)
// ... use dbConnectionString to connect
}
The most surprising true thing about this setup is that Vault doesn’t just store secrets; it actively manages their lifecycle. It can create, rotate, and revoke credentials based on defined policies and time-to-live (TTL) settings, acting more like a credential issuance service than a simple password vault.
When your application starts, it doesn’t fetch a static secret. Instead, it initiates a request to Vault for a role. Vault then performs its magic: creating a temporary database user, granting it just enough privileges for the application’s immediate needs, and returning these temporary credentials. The crucial part is that Vault also sets an expiration time for these credentials. When that time is up, Vault automatically cleans up by revoking the credentials and deleting the associated database user. This ensures that even if an application is compromised, the leaked credentials are only valid for a short, predefined period.
The interaction between Consul and Vault here is about orchestration and discovery. Consul can tell your application where to find Vault and which role to request credentials from, but the actual dynamic secret generation and management are solely Vault’s responsibility. This separation of concerns allows each system to do what it does best: Consul for service discovery and configuration, and Vault for secure secrets management and dynamic credential issuance.
A subtle but powerful aspect is how Vault’s database secrets engine allows you to define creation_statements. These are SQL commands that Vault executes before issuing the credentials. This means you can precisely control the privileges granted to the dynamically generated user, adhering to the principle of least privilege. For instance, you could grant read-only access to specific tables, or permissions to only a particular schema, ensuring the application can only access what it absolutely needs.
The next step in mastering dynamic secrets is to explore Vault’s various authentication methods, such as Kubernetes Service Account authentication, which allows pods to authenticate with Vault without needing pre-shared tokens.