FastAPI logs, by default, are just strings. Structlog transforms them into structured data, making them machine-readable and infinitely more useful for debugging and analysis in production.
Let’s see structlog in action with FastAPI. Imagine this main.py:
from fastapi import FastAPI
import logging
import structlog
import sys
app = FastAPI()
# Configure structlog
structlog.configure(
processors=[
structlog.stdlib.add_logger_name,
structlog.stdlib.add_log_level,
structlog.processors.TimeStamper(fmt="iso"),
structlog.processors.StackInfoRenderer(),
structlog.processors.format_exc_info,
structlog.processors.UnicodeDecoder(),
structlog.stdlib.ProcessorFormatter.wrap_for_formatter,
],
logger_factory=structlog.stdlib.LoggerFactory(),
wrapper_class=structlog.stdlib.BoundLogger,
cache_logger_on_first_use=True,
)
# Get a structlog-compatible logger
logger = structlog.get_logger("my_fastapi_app")
@app.get("/")
async def read_root():
logger.info("Root endpoint hit", user_id=123, request_method="GET")
return {"Hello": "World"}
@app.get("/items/{item_id}")
async def read_item(item_id: int, q: str | None = None):
if item_id == 999:
try:
result = 1 / 0
except ZeroDivisionError:
logger.error("Division by zero encountered", item_id=item_id, query_param=q, exc_info=True)
else:
logger.debug("Item endpoint hit", item_id=item_id, query_param=q)
return {"item_id": item_id, "q": q}
# Configure standard library logging to use structlog's formatter
formatter = structlog.stdlib.ProcessorFormatter(
processor=structlog.dev.ConsoleRenderer(colors=True) # Use ConsoleRenderer for development, JSONRenderer for production
)
handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(formatter)
root_logger = logging.getLogger()
root_logger.addHandler(handler)
root_logger.setLevel(logging.DEBUG)
When you run this with uvicorn main:app --reload and then hit the / endpoint, you’ll see output like this:
{
"timestamp": "2023-10-27T10:30:00.123456Z",
"logger_name": "my_fastapi_app",
"event": "Root endpoint hit",
"user_id": 123,
"request_method": "GET",
"level": "info"
}
And if you hit /items/999, you’ll get:
{
"timestamp": "2023-10-27T10:30:01.678901Z",
"logger_name": "my_fastapi_app",
"event": "Division by zero encountered",
"item_id": 999,
"query_param": null,
"level": "error",
"exception": "Traceback (most recent call last):\n File \"/path/to/your/main.py\", line 31, in read_item\n result = 1 / 0\nZeroDivisionError: division by zero",
"stack_info": "Traceback (most recent call last):\n File \"/path/to/your/main.py\", line 31, in read_item\n result = 1 / 0\nZeroDivisionError: division by zero\n\nDuring handling of the above exception, another exception occurred:\nTraceback (most recent call last):\n File \"/path/to/your/main.py\", line 33, in read_item\n logger.error(\"Division by zero encountered\", item_id=item_id, query_param=q, exc_info=True)\n..."
}
The core problem structlog solves is the "text-based log" problem. In production, you’re not manually reading log files. You’re shipping them to a centralized logging system (like ELK, Splunk, Datadog) that expects structured data (usually JSON) to filter, search, and alert effectively. String logs are opaque; structured logs are transparent.
Structlog works by chaining processors. Each processor takes the event dictionary and transforms it. add_logger_name adds the logger’s name, add_log_level adds the severity, TimeStamper adds a timestamp, format_exc_info takes the exception details and formats them nicely, and wrap_for_formatter prepares the event for the final renderer. The ProcessorFormatter then uses a renderer (ConsoleRenderer for dev, JSONRenderer for prod) to turn that processed dictionary into a string.
The structlog.stdlib.BoundLogger is key. It wraps the standard library logging module, allowing you to use familiar logger.info(), logger.error(), etc., while internally structlog intercepts these calls, processes them, and then passes them to the standard library’s handlers. This means you can integrate structlog with existing Python logging configurations.
The structlog.dev.ConsoleRenderer is great for local development because it provides colored, human-readable output. For production, you’d swap this for structlog.processors.JSONRenderer() to ensure your logs are emitted as JSON, which is what most log aggregation tools expect.
The exc_info=True argument in logger.error() is crucial. It tells structlog to capture the current exception information and include it in the log event, which is invaluable for debugging. Without it, you’d just see "Division by zero encountered" without the traceback.
When you set up the standard library logger (root_logger), you’re telling Python’s built-in logging system to use structlog’s formatter for any log records it receives. This bridges the gap between the standard library and structlog, ensuring that structlog’s processed output is what actually gets sent to your handlers (like StreamHandler to stdout).
The real power emerges when you start using structlog.processors.JSONRenderer() in production. Combined with a log shipping agent (like Filebeat, Fluentd) that picks up these JSON lines and sends them to a system like Elasticsearch, you can then build dashboards and alerts based on specific fields like user_id, item_id, or request_method directly in your logging platform, rather than relying on brittle regex matching against plain text.
The structlog.stdlib.ProcessorFormatter.wrap_for_formatter processor is essential because it ensures that the event dictionary processed by structlog is in the correct format for the final renderer, especially when you’re integrating with the standard library’s logging system. It handles details like ensuring all keys are strings and that the final event is a dictionary ready for serialization.