Audit logging in Couchbase isn’t just about recording what happened; it’s about proving who did what, when, and to which data, even if the system itself is compromised.
Let’s fire up a demo cluster and see this in action. Imagine we’re running a small e-commerce backend.
# Start a Couchbase cluster (simplified for demo)
docker run -d --name couchbase -p 8091:8091 -p 11210:11210 couchbase/server:7.1.0
# Access the UI and set up a user with audit permissions
# Navigate to Settings -> Audits
# Enable Audit Logging: Check the "Enable Audit Logging" box.
# Set Audit Log Size: Set to 100 MB (this is the max size before rotation).
# Set Audit Log Path: /opt/couchbase/var/lib/couchbase/audit/
# Select Audit Events:
# - Data Access: CHECKED
# - Data Modification: CHECKED
# - Security: CHECKED
# - User Management: CHECKED
# - Bucket Management: CHECKED
# - Service Management: CHECKED
# Click "Save"
Now, let’s perform some actions and then inspect the logs.
First, add a user:
curl -X POST -u Administrator:password \
http://localhost:8091/api/v1/settings/audit \
-d '{"username":"auditor","password":"password123","roles":["auditor"]}'
Next, let’s access some data. We’ll add a simple document to a bucket named my_bucket.
# Assuming you have a document named doc1.json with {"key": "value"}
curl -X POST -u Administrator:password \
http://localhost:8091/document/my_bucket/doc1 \
-d '{"key": "value"}'
Finally, let’s change a bucket setting, say, disable compaction for my_bucket.
curl -X POST -u Administrator:password \
http://localhost:8091/pools/default/buckets/my_bucket/update \
-d 'compactionMode=disabled'
Now, let’s look at the audit logs. These are generated as JSON files in the specified directory. On a real system, you’d typically use a log forwarder (like Filebeat or Fluentd) to send these to a central logging system (like Elasticsearch or Splunk). For this demo, we’ll docker exec into the container and cat the file.
docker exec couchbase cat /opt/couchbase/var/lib/couchbase/audit/audit.log
You’ll see entries like this (simplified for clarity):
{"timestamp": "2023-10-27T10:30:01.123Z", "type": "user", "operation": "create", "user": "Administrator", "target": "auditor", "status": "success"}
{"timestamp": "2023-10-27T10:31:15.456Z", "type": "data", "operation": "write", "user": "Administrator", "bucket": "my_bucket", "key": "doc1", "status": "success"}
{"timestamp": "2023-10-27T10:32:40.789Z", "type": "bucket", "operation": "update", "user": "Administrator", "bucket": "my_bucket", "setting": "compactionMode", "status": "success"}
The core problem audit logging solves is non-repudiation. Without it, if a critical security incident occurs, you have no verifiable record of who performed the actions that led to it. This is crucial for compliance frameworks like GDPR, HIPAA, SOC 2, and PCI DSS, which mandate logging of sensitive operations.
Internally, Couchbase’s audit logging mechanism works by intercepting specific operational events within the server. When an event occurs that matches the configured audit filters, the server formats it into a structured JSON record and appends it to the audit log file. The log file is managed with a rolling policy: when it reaches the configured size limit (e.g., 100 MB), it’s rotated, and a new log file is started. Older files are typically deleted based on a retention policy, though this is often handled by the external log management system.
The levers you control are:
- Enable/Disable: The primary switch.
- Log Size: Controls how often log rotation happens, impacting disk space and log management frequency.
- Log Path: Where the logs are written to disk. Ensure this path has sufficient space and appropriate permissions.
- Audit Events: This is the most granular control. You select which categories of events to log. Common categories include:
Data Access: Reads of documents.Data Modification: Writes, updates, deletes of documents.Security: Authentication attempts (success/failure), certificate operations.User Management: Creating, deleting, modifying users and roles.Bucket Management: Creating, deleting, modifying buckets.Service Management: Starting/stopping services, cluster configuration changes.Query Management: Execution of N1QL queries.Analytics Management: Execution of Analytics queries.Eventing Management: Eventing function deployment and execution.Full Text Search Management: FTS index creation/deletion.
The one thing most people don’t realize is that audit events are not logged by default, and enabling them can have a performance impact, especially if you log very granular events like every single document read. The JSON format is designed for machine parsing, but it’s also human-readable, making it versatile. For compliance, the key is not just collecting these logs, but ensuring they are stored securely and retained for the required period, often necessitating an external SIEM (Security Information and Event Management) system.
The next hurdle you’ll face is configuring a log rotation and retention strategy that meets your organization’s compliance mandates and operational capacity.