API Gateway can send logs to CloudWatch, but it’s off by default and requires explicit configuration.
Here’s how to set it up and understand what you’re getting.
First, let’s see API Gateway in action with logging enabled. Imagine a simple API that echoes back a request.
{
"body": "{\"message\": \"Hello, world!\"}",
"resource": "/echo",
"path": "/echo",
"httpMethod": "POST",
"requestContext": {
"accountId": "123456789012",
"apiId": "abcdef123",
"domainName": "abcdef123.execute-api.us-east-1.amazonaws.com",
"domainPrefix": "abcdef123",
"httpMethod": "POST",
"requestId": "a1b2c3d4-e5f6-7890-1234-567890abcdef",
"requestTime": "10/Oct/2023:10:00:00 +0000",
"requestTimeEpoch": 1696944000000,
"resourcePath": "/echo",
"stage": "prod"
},
"headers": {
"Accept": "*/*",
"Content-Length": "23",
"Content-Type": "application/json",
"Host": "abcdef123.execute-api.us-east-1.amazonaws.com",
"User-Agent": "curl/7.64.1",
"X-Amzn-Trace-Id": "Root=1-65257b20-1234567890abcdef1234567890",
"X-Forwarded-For": "192.0.2.1",
"X-Forwarded-Port": "443",
"X-Forwarded-Proto": "https"
},
"queryStringParameters": null,
"pathParameters": null,
"stageVariables": null,
"isBase64Encoded": false
}
When we enable logging, API Gateway will send details about this request, and its response, to a CloudWatch Logs log group.
To enable this, you need to configure logging settings on your API Gateway stage. This involves two main parts: enabling execution logging and enabling access logging.
Execution Logging: This captures detailed information about the request processing pipeline within API Gateway itself. It’s invaluable for debugging integration issues, Lambda errors, or mapping template problems.
- Navigate to your API Gateway stage: In the AWS Management Console, go to API Gateway, select your API, and then click on the stage (e.g.,
prod). - Go to the "Logs/Tracing" tab.
- Enable CloudWatch Logs: Toggle "Enable CloudWatch Logs" to ON.
- Log level: Select
INFOfor detailed logs.ERRORis also an option, butINFOis generally more useful for debugging. - Log group name: This is the name of the CloudWatch Log Group where your logs will be sent. You can create a new one or select an existing one. A common convention is
api-gateway/<api-id>/<stage-name>, e.g.,api-gateway/abcdef123/prod. - Include tags: You can optionally include request/response tags.
- Save.
Access Logging: This captures information about incoming requests and outgoing responses, similar to traditional web server access logs. It’s great for understanding traffic patterns, identifying sources of requests, and basic request/response data.
- On the same "Logs/Tracing" tab, scroll down to "Access logging".
- Enable Access Logging: Toggle "Enable Access Logging" to ON.
- Log format: You can choose a predefined format like
JSONorHTTP(common log format), or create a custom one.JSONis highly recommended for programmatic analysis. - Log destination ARN: This is the ARN of the CloudWatch Log Group you configured for execution logging, or another log group if you prefer. It will look like
arn:aws:logs:us-east-1:123456789012:log-group:api-gateway/abcdef123/prod:*. - Save.
Once configured, you’ll see log streams appear in your specified CloudWatch Log Group. Each stream typically corresponds to an API Gateway instance.
For example, a typical execution log entry might look like this:
{
"context": {
"requestId": "a1b2c3d4-e5f6-7890-1234-567890abcdef",
"apiId": "abcdef123",
"resourcePath": "/echo",
"httpMethod": "POST",
"stage": "prod",
"identity": {
"sourceIp": "192.0.2.1"
},
"requestTime": "10/Oct/2023:10:00:00 +0000"
},
"message": "Execution failed due to an internal error."
}
And an access log entry (using JSON format) might look like:
{
"requestId": "a1b2c3d4-e5f6-7890-1234-567890abcdef",
"method": "POST",
"path": "/echo",
"requestTime": "10/Oct/2023:10:00:00 +0000",
"status": "200",
"integrationStatus": "200",
"latency": 150,
"userAgent": "curl/7.64.1",
"sourceIp": "192.0.2.1",
"error": "None"
}
The "Log format" for access logs is highly customizable. You can use placeholders like $context.requestId, $context.identity.sourceIp, $context.httpMethod, $context.path, $context.status, $context.latency, etc., to build your own format. For instance, a custom JSON format might look like:
{"requestId":"$context.requestId","clientIp":"$context.identity.sourceIp","httpMethod":"$context.httpMethod","path":"$context.path","status":"$context.status","latency":"$context.latency"}
This gives you precise control over what data you capture.
The most surprising thing about API Gateway logging is how granular you can get with access logs, allowing you to construct logs that perfectly match your existing observability stack’s expectations or your specific debugging needs, without needing to parse opaque request/response bodies unless you explicitly configure it to do so. For example, you can include $request.body or $response.body directly in the log format, though this can quickly increase log volume and cost.
After enabling logging, the next thing you’ll likely want to do is set up CloudWatch Alarms based on your logs, or use CloudWatch Logs Insights to query them efficiently.