Datadog Log Analytics, when wielded effectively, doesn’t just store logs; it transforms them into a dynamic, queryable knowledge base that can answer questions you didn’t even know you had.
Imagine you’re tracking user sign-ups across a distributed system. You’ve got logs from your web servers, your API gateway, your authentication service, and your database. Without proper analytics, this is just a firehose of text.
Here’s how it looks in action. Let’s say you’re using Datadog’s agent to collect logs. A simple datadog-agent.conf might look like this:
logs:
enabled: true
logs_dd_url: "https://http-intake.logs.datadoghq.com/api/v2/logs"
container_collect_all: true
tcp_port: 10514
udp_port: 10514
And your application’s log output, structured as JSON, might look like this:
{"timestamp": "2023-10-27T10:00:00Z", "level": "info", "message": "User signup initiated", "user_id": "user123", "request_id": "req-abc"}
{"timestamp": "2023-10-27T10:00:01Z", "level": "debug", "message": "Validating email format", "user_id": "user123", "request_id": "req-abc"}
{"timestamp": "2023-10-27T10:00:02Z", "level": "info", "message": "User signup successful", "user_id": "user123", "request_id": "req-abc", "signup_duration_ms": 2000}
{"timestamp": "2023-10-27T10:01:00Z", "level": "error", "message": "Database connection failed", "user_id": "user456", "request_id": "req-def", "error_code": 503}
Datadog automatically parses this JSON. Now, the magic of facets and measures comes in.
Facets are attributes that you can group, filter, and visualize your logs by. Think of them as dimensions in a data cube. In our example, user_id, request_id, level, and error_code are prime candidates for facets. Datadog’s processing pipeline automatically identifies common attributes and makes them available as facets. You can also define custom facets if your logs have unique, important fields.
When you click on "Log Patterns" or "Indexes" in Datadog, you’ll see a list of available facets. You can then use these in your queries. For instance, to see all successful sign-ups:
service:my-app status:success
Or to count errors by error_code:
status:error
| stats count() by error_code
Measures are numerical values within your logs that you can aggregate. These are your metrics extracted directly from log lines. In our example, signup_duration_ms is a perfect measure. You can use it to calculate average sign-up times, percentiles, and more.
To find the average sign-up duration:
service:my-app status:success
| stats avg(signup_duration_ms) by service
Or to find the 95th percentile duration:
service:my-app status:success
| stats p95(signup_duration_ms) by service
The real power comes from combining them. You can facet by user_id and measure signup_duration_ms to see which users are experiencing the longest sign-up times.
service:my-app status:success
| stats avg(signup_duration_ms) by user_id
| sort user_id asc
This allows you to move beyond simply searching for specific errors to understanding trends, performance bottlenecks, and user behavior. You can build dashboards that show the rate of successful sign-ups per minute, faceted by the region of the user, and overlayed with the average sign-up duration.
One of the most impactful, yet often overlooked, aspects of Datadog Log Analytics is the ability to define processing rules that enrich your logs before they are indexed. This means you can parse nested JSON, extract specific regex matches, drop sensitive fields, or even add new attributes based on existing ones. For example, if your logs contain a user agent string, you could use a processing rule to extract the browser type and operating system, making them available as facets without needing to modify your application code. This capability is crucial for turning messy, unstructured, or semi-structured logs into a rich source of analytical data.
Once you’ve mastered facets and measures, the next logical step is to explore Log Patterns for anomaly detection and Live Tail for real-time debugging.