Express APIs need audit logging to meet compliance requirements, but it’s often bolted on as an afterthought, leading to gaps in visibility.
Let’s see what that looks like in practice. Imagine a simple Express app:
const express = require('express');
const app = express();
const port = 3000;
app.use(express.json()); // Middleware to parse JSON bodies
app.get('/users/:id', (req, res) => {
const userId = req.params.id;
// In a real app, this would fetch from a database
const user = { id: userId, name: 'Alice' };
console.log(`Fetched user ${userId}`);
res.json(user);
});
app.post('/users', (req, res) => {
const newUser = req.body;
// In a real app, this would save to a database
console.log('Created new user:', newUser);
res.status(201).json(newUser);
});
app.listen(port, () => {
console.log(`App listening at http://localhost:${port}`);
});
Now, let’s add audit logging. The goal isn’t just to log that something happened, but who did it, what they did, and when. For compliance, we typically need to record:
- Timestamp: When the event occurred.
- User Identity: Who initiated the request (e.g., authenticated user ID, IP address if unauthenticated).
- Action/Operation: What API endpoint was accessed and what HTTP method was used.
- Resource: Which specific resource was affected (e.g.,
/users/123). - Payload/Parameters: Relevant parts of the request body or query parameters.
- Status/Outcome: Whether the request succeeded or failed, and the HTTP status code.
A common approach is to use Express middleware. This lets us hook into every request before it hits your route handlers. We can use a library like winston for robust logging and body-parser (though express.json() is built-in now) if we need to inspect request bodies.
Here’s an enhanced version:
const express = require('express');
const app = express();
const port = 3000;
const winston = require('winston'); // For structured logging
// Configure Winston logger
const logger = winston.createLogger({
level: 'info',
format: winston.format.combine(
winston.format.timestamp(),
winston.format.json() // Log in JSON format for easier parsing
),
transports: [
new winston.transports.Console(), // Log to console
// In production, you'd likely use:
// new winston.transports.File({ filename: 'audit.log' })
],
});
app.use(express.json()); // Middleware to parse JSON bodies
// Audit logging middleware
app.use((req, res, next) => {
const start = Date.now();
const { method, url, ip, user } = req; // 'user' would come from auth middleware
const requestId = Math.random().toString(36).substring(2, 9); // Simple unique ID
// Capture request details
const requestDetails = {
requestId,
timestamp: new Date().toISOString(),
method,
url,
ip,
user: user ? user.id : 'anonymous', // Assuming 'user' object with 'id' is attached by auth
headers: req.headers, // Log relevant headers, sanitize sensitive ones
query: req.query,
body: req.body // Be cautious logging entire bodies in production!
};
// Log successful responses
const originalSend = res.send;
res.send = function (body) {
const duration = Date.now() - start;
const responseDetails = {
requestId,
statusCode: res.statusCode,
responseBody: body, // Again, be cautious
duration
};
logger.info('API Request Completed', {
request: requestDetails,
response: responseDetails
});
originalSend.call(this, body);
};
// Log errors (e.g., 4xx, 5xx status codes)
res.on('finish', () => {
if (res.statusCode >= 400) {
const duration = Date.now() - start;
const errorDetails = {
requestId,
statusCode: res.statusCode,
statusMessage: res.statusMessage,
duration
};
logger.error('API Request Failed', {
request: requestDetails,
response: errorDetails
});
}
});
next();
});
app.get('/users/:id', (req, res) => {
const userId = req.params.id;
const user = { id: userId, name: 'Alice' };
res.json(user);
});
app.post('/users', (req, res) => {
const newUser = req.body;
res.status(201).json(newUser);
});
app.listen(port, () => {
console.log(`App listening at http://localhost:${port}`);
});
This middleware captures the method, url, ip, and potentially user (if authentication is in place). It also wraps res.send to log the response body and status code. For errors, it hooks into the finish event. The requestId is crucial for correlating logs across different services if your API becomes more complex.
The most surprising thing about audit logging is how often its true purpose gets diluted by simply logging "everything" without specific compliance goals in mind. The goal isn’t just to have logs, but to have auditable logs – logs that can be queried, filtered, and presented to satisfy specific regulatory or internal security requirements.
Here’s a request to /users/123 with the above setup:
curl http://localhost:3000/users/123
And the corresponding log output (simplified for clarity, actual output is JSON):
{
"level": "info",
"message": "API Request Completed",
"timestamp": "2023-10-27T10:30:00.123Z",
"meta": {
"request": {
"requestId": "abc123xyz",
"timestamp": "2023-10-27T10:30:00.000Z",
"method": "GET",
"url": "/users/123",
"ip": "::ffff:127.0.0.1",
"user": "anonymous",
"headers": {
"host": "localhost:3000",
"connection": "keep-alive",
// ... other headers
},
"query": {},
"body": {}
},
"response": {
"requestId": "abc123xyz",
"statusCode": 200,
"responseBody": {
"id": "123",
"name": "Alice"
},
"duration": 123
}
}
}
And a failed request:
curl -X POST -H "Content-Type: application/json" -d '{"name": "Bob"}' http://localhost:3000/users
The log might look like this if an error occurred during processing (e.g., validation failure, though not implemented here):
{
"level": "error",
"message": "API Request Failed",
"timestamp": "2023-10-27T10:31:00.456Z",
"meta": {
"request": {
"requestId": "def456uvw",
"timestamp": "2023-10-27T10:31:00.300Z",
"method": "POST",
"url": "/users",
"ip": "::ffff:127.0.0.1",
"user": "anonymous",
"headers": {
"host": "localhost:3000",
"connection": "keep-alive",
"content-type": "application/json",
// ... other headers
},
"query": {},
"body": {
"name": "Bob"
}
},
"response": {
"requestId": "def456uvw",
"statusCode": 400, // Example error status
"statusMessage": "Bad Request",
"duration": 156
}
}
}
The core challenge is balancing detail with privacy and performance. Logging sensitive data (like passwords in request bodies) is a major security risk. You’ll need to sanitize req.body and potentially req.headers to exclude PII or secrets. Libraries like express-audit-log can abstract some of this, but understanding the underlying principles is key. The user field is paramount for compliance – ensure your authentication middleware reliably attaches user identity to req.user. Without it, logs are incomplete for auditing purposes.
The next challenge is managing and querying these logs effectively for actual audits.