Caddy’s default access log format is a bit of a black box if you’re trying to ingest it into a modern log analysis system.
{
"log_level": "info",
"msg": "request completed",
"request": {
"method": "GET",
"remote_addr": "192.168.1.100:54321",
"uri": "/index.html",
"headers": {
"User-Agent": ["Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"]
}
},
"response": {
"status_code": 200,
"size": 1024,
"headers": {
"Content-Type": ["text/html; charset=utf-8"]
}
},
"duration": "12.345ms",
"common_log_format": "192.168.1.100 - - [2023-10-27T10:30:00+00:00] \"GET /index.html HTTP/1.1\" 200 1024",
"bytes_written": 1024,
"remote_ip": "192.168.1.100",
"user_id": ""
}
This is a typical Caddy access log entry, showing a successful GET request for /index.html. Notice how the structured data is nested within the request and response fields, while also providing a common_log_format string for backward compatibility.
To get this into a system like Elasticsearch, Splunk, or even just a structured file for jq processing, you’ll want to customize the log format. Caddy uses the log directive for this, and its format subdirective is where the magic happens.
Here’s a Caddyfile snippet demonstrating a JSON-formatted log that’s much more amenable to machine parsing:
{
servers.logs.output stdout
}
:80 {
log {
output stdout
format json {
time_format 2006-01-02T15:04:05Z07:00
level {key}
message {key}
remote_ip {remote_ip}
remote_port {remote_port}
method {method}
uri {uri}
protocol {protocol}
host {host}
user_agent {user_agent}
request_header {header.*}
response_header {header.*}
status_code {status}
bytes_written {bytes_written}
duration {duration}
}
}
respond /hello "Hello, world!"
}
Let’s break down what’s happening here.
First, we ensure Caddy is configured to output logs. In this example, servers.logs.output stdout at the top level tells Caddy to send its internal logs (like startup messages) to stdout. The log { output stdout } within the site block directs the access logs to stdout as well. You could easily change stdout to a file path like /var/log/caddy/access.log.
The core of the customization is the format json directive. This tells Caddy to emit logs in JSON format. Inside this, we list the fields we want to include.
time_format 2006-01-02T15:04:05Z07:00: This sets the timestamp format. The Gotimepackage uses this specific reference time (2006-01-02T15:04:05Z07:00) to define layouts. This ISO 8601-like format is standard and easily parsed.level {key}andmessage {key}: These map Caddy’s internal log level and message fields to JSON keys namedlevelandmessage.remote_ip {remote_ip}: This maps the client’s IP address to a JSON keyremote_ip.remote_port {remote_port}: Maps the client’s port.method {method}: Maps the HTTP request method (GET, POST, etc.) tomethod.uri {uri}: Maps the requested URI path touri.protocol {protocol}: Maps the HTTP protocol (HTTP/1.1, HTTP/2) toprotocol.host {host}: Maps theHostheader value tohost.user_agent {user_agent}: Maps theUser-Agentheader touser_agent.request_header {header.*}: This is a powerful wildcard. It captures all request headers and nests them under arequest_headerkey in the JSON output. So, aUser-Agentrequest header would appear asrequest_header.User-Agent.response_header {header.*}: Similar to request headers, this captures all response headers.status_code {status}: Maps the HTTP status code tostatus_code.bytes_written {bytes_written}: Maps the number of bytes sent back to the client tobytes_written.duration {duration}: Maps the request processing time toduration.
When Caddy runs with this configuration and receives a request, the output on stdout will look something like this:
{
"time": "2023-10-27T10:30:00+00:00",
"level": "info",
"message": "request completed",
"remote_ip": "192.168.1.100",
"remote_port": 54321,
"method": "GET",
"uri": "/hello",
"protocol": "HTTP/1.1",
"host": "localhost",
"user_agent": "curl/7.68.0",
"request_header": {
"User-Agent": ["curl/7.68.0"],
"Accept": ["*/*"]
},
"response_header": {
"Content-Type": ["text/plain; charset=utf-8"],
"Server": ["Caddy"]
},
"status_code": 200,
"bytes_written": 13,
"duration": "1.234ms"
}
This JSON output is now perfectly structured for ingestion by log aggregation tools. Each field is a distinct key-value pair, making it easy to filter, search, and analyze your Caddy access logs. For instance, you could easily query for all requests with a status_code of 404, or analyze the distribution of user_agent strings.
The format directive supports several other built-in formats like common_log and combined_log for compatibility, but json is the most flexible for modern systems. You can also create custom text formats using placeholders similar to how you’d build a JSON structure.
The most surprising thing about Caddy’s logging configuration is how granular you can get with the format json directive, allowing you to explicitly include or exclude almost any piece of request or response metadata, and even nest headers.
The next step in log analysis is often setting up a dedicated log shipper (like Filebeat or Fluentd) to collect these JSON logs from stdout or a file and forward them to your central logging system.