Caddy’s default configuration is surprisingly opinionated about resource usage, often leaving performance on the table in production environments.
Let’s see Caddy handle a bit of traffic. Imagine we have a simple static site and Caddy serving it.
# Start Caddy with a basic Caddyfile
caddy run --config Caddyfile.prod
And our Caddyfile.prod looks like this:
:80 {
root * /var/www/html
file_server
}
This is fine for development, but for production, we need to get more aggressive.
The core problem Caddy solves is efficiently serving HTTP requests, especially for static files and acting as a reverse proxy. It does this with a pluggable architecture, but its defaults are conservative to ensure it runs "everywhere" without issues. For production, we want it to run fast.
The primary lever you have is Caddy’s configuration, primarily through the Caddyfile. You can also influence performance via environment variables and the underlying operating system’s tuning.
Here’s a breakdown of key areas to tune:
Concurrency and Worker Pools
Caddy uses goroutines extensively. The default number of goroutines can be insufficient for high-volume traffic. You can increase the number of OS threads Caddy can use.
Diagnosis: Monitor system CPU and I/O wait. If you see high CPU but Caddy isn’t saturating all cores, or if requests are queuing, concurrency might be an issue.
Check: ulimit -u will show the maximum number of processes/threads. Caddy’s default GOMAXPROCS might be lower than what your system can handle.
Fix: Set the GOMAXPROCS environment variable. For a server with 16 CPU cores, you might set:
export GOMAXPROCS=16
caddy run --config Caddyfile.prod
This tells the Go runtime to use up to 16 OS threads for executing goroutines, allowing Caddy to handle more concurrent requests efficiently.
Keep-Alive Connections and Buffering
HTTP keep-alive is crucial for performance, reducing the overhead of establishing new TCP connections. Caddy’s buffering can also impact throughput.
Diagnosis: Use tools like netstat -anp | grep caddy to see the number of established connections. High connection churn or slow response times under load suggest keep-alive or buffering issues.
Check: Caddy’s default read_buffer_size and write_buffer_size are often small.
Fix: Increase buffer sizes in your Caddyfile. For example, to 64KB:
:80 {
root * /var/www/html
file_server {
read_buffer_size 65536
write_buffer_size 65536
}
}
This allows Caddy to read and write larger chunks of data at once, reducing system call overhead and improving throughput for large files or slow clients.
TLS Configuration
For HTTPS, Caddy’s TLS settings are critical. Default cipher suites might not be the fastest or most secure.
Diagnosis: Use SSL Labs’ Server Test to analyze your TLS configuration. Look for slow handshakes or weak cipher suites.
Check: Caddy’s default tls directive has a broad set of enabled cipher suites.
Fix: Explicitly define preferred cipher suites and protocols.
:443 {
tls your.email@example.com {
protocols tls1.2 tls1.3
ciphers tls_aes_128_gcm_sha256 tls_chacha20_poly1305_sha256 tls_aes_256_gcm_sha384
}
root * /var/www/html
file_server
}
This prioritizes modern, faster cipher suites and protocols, reducing TLS handshake latency and improving overall connection setup speed.
File System Caching and mmap
Caddy’s file_server can utilize mmap for efficient file serving. However, OS-level caching plays a significant role.
Diagnosis: Use iotop or iostat to monitor disk I/O. If disk I/O is consistently high for static files, OS caching might not be optimal.
Check: Ensure your OS’s page cache is adequately sized and that Caddy is configured to leverage mmap (which is often the default, but worth verifying).
Fix: Tune your OS’s kernel parameters for file system caching. For Linux:
# Example: Increase shared memory limits and file handle limits
sudo sysctl -w fs.file-max=200000
sudo sysctl -w vm.max_map_count=262144
Then, ensure Caddy’s file_server is configured to use mmap (usually default). The mmap system call allows Caddy to map file contents directly into its address space, allowing the OS to manage caching and read operations more efficiently than traditional read() calls.
Connection Limits and Timeouts
Aggressive connection limits and timeouts can prevent resource exhaustion but also starve legitimate traffic.
Diagnosis: Look for context deadline exceeded or similar errors in Caddy logs, or observe slow responses during peak load.
Check: Default timeouts in Caddyfile.
Fix: Adjust timeouts to be more generous for production.
:80 {
timeouts {
read 1m
write 1m
idle 5m
}
root * /var/www/html
file_server
}
This provides clients more time to send requests and receive responses, preventing premature connection drops for slower clients or complex requests, while still protecting against hung connections.
HTTP/2 and HTTP/3 Prioritization
Caddy enables HTTP/2 by default. HTTP/3 can offer further improvements, especially on lossy networks.
Diagnosis: Monitor network latency and packet loss. If these are high, HTTP/3 can significantly outperform HTTP/1.1 and HTTP/2.
Check: Caddy’s default enables HTTP/2. HTTP/3 support requires specific configuration and QUIC.
Fix: Explicitly enable HTTP/3.
:443 {
protocols h1 h2 h3
root * /var/www/html
file_server
}
This allows Caddy to negotiate HTTP/3 with compatible clients, leveraging UDP and reducing head-of-line blocking, leading to faster and more resilient connections.
When you’ve tuned these aspects, you’ll find Caddy much more capable of handling demanding production loads. The next challenge you’ll likely face is managing and monitoring these tuned configurations across a fleet of servers.