Django’s logging can be surprisingly opaque in production until you realize its core mechanism isn’t about what you log, but how you structure it for machines to read.
Let’s see it in action. Imagine a simple Django view:
# views.py
import logging
from django.http import HttpResponse
logger = logging.getLogger(__name__)
def my_view(request):
user_id = request.user.id if request.user.is_authenticated else 'anonymous'
request_id = request.META.get('HTTP_X_REQUEST_ID', 'no-id')
data = {'message': 'User accessed view', 'user_id': user_id, 'request_id': request_id}
logger.info("User access event", extra=data)
return HttpResponse("Hello!")
# urls.py
from django.urls import path
from . import views
urlpatterns = [
path('my-view/', views.my_view, name='my_view'),
]
If you just dump this to a plain text file, it’s a mess. But with structured logging, that extra dictionary becomes key-value pairs embedded directly in your log output, typically in JSON.
{"message": "User access event", "user_id": "anonymous", "request_id": "no-id", "levelname": "INFO", "name": "myapp.views", "asctime": "2023-10-27 10:30:00,123", "module": "views", "funcName": "my_view", "lineno": 11, "process": 12345, "thread": 67890}
This JSON output is what makes Django logging production-ready. Tools like Elasticsearch, Splunk, or even simple log aggregation services can parse this directly. They can filter by user_id, group by request_id, or alert on specific message content without complex regex.
The problem this solves is moving from "grep-able text files" to "query-able databases of events." In development, a print() statement or a simple logger.debug("something happened") is fine. But in production, with thousands of requests per second across multiple servers, you need to quickly find specific events. Structured logging provides the precise data points needed for that.
Internally, Django’s logging relies on Python’s standard logging module. The magic for structure comes from dictConfig and extra dictionaries. You define a LOGGING dictionary in your settings.py that specifies handlers (where logs go) and formatters (how logs look). For structured logging, you’ll use a formatter that outputs JSON. The extra parameter in your logging calls is where you attach custom, structured data.
The most common way to achieve this is by using a third-party library like python-json-logger. You’d add it to your INSTALLED_APPS and configure your LOGGING dictionary.
# settings.py
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'json': {
'()': 'pythonjsonlogger.jsonlogger.JsonFormatter',
'format': '%(asctime)s %(levelname)s %(name)s %(message)s %(user_id)s %(request_id)s',
},
},
'handlers': {
'console': {
'class': 'logging.StreamHandler',
'formatter': 'json',
},
},
'loggers': {
'': { # Root logger
'handlers': ['console'],
'level': 'INFO',
},
},
}
This configuration tells Django to use a JsonFormatter for all logs handled by the console handler (which, in production, you’d likely redirect to a file or send to a log aggregator). The format string specifies which fields, including custom ones like user_id and request_id (which come from the extra dict), should be included in the JSON output.
When you use logger.info("User access event", extra={'user_id': user_id, 'request_id': request_id}), the JsonFormatter automatically picks up user_id and request_id from the extra dictionary and includes them as top-level JSON keys. If a log record doesn’t have one of these fields (e.g., user_id is missing), the formatter typically omits it or represents it as null, depending on its configuration.
The most surprising part is how seamlessly standard Python logging integrates with dictConfig and external formatters. You don’t need to monkey-patch Django or write custom middleware for basic structured logging. The extra dictionary is a standard feature of Python’s logging module, and dictConfig allows you to plug in any formatter, including those that generate JSON.
The next step after mastering structured logging is to implement context managers for automatically adding request-specific data to every log message within a request’s lifecycle.