Correlate Elastic APM Traces with Logs Using trace.id (2026)

Logs are actually the output of your traces, not just a separate, parallel system.

Let’s say you’ve got an incoming request hitting your web server. Elastic APM captures this as a trace, a hierarchical breakdown of all the work done to fulfill that request. Think of it as a detailed itinerary for that single request. Each stop on the itinerary, like a database query or an external API call, is a "span."

Here’s a trace in the APM UI, showing a request and its constituent spans:

{
  "trace_id": "a1b2c3d4e5f67890",
  "transaction_id": "0987654321fedcba",
  "span_id": "1234567890abcdef",
  "name": "GET /users/:id",
  "type": "request",
  "start_time": "2023-10-27T10:00:00.000Z",
  "duration": 150,
  "parent_id": null,
  "service": {
    "name": "my-web-app"
  },
  "http": {
    "request": {
      "method": "GET",
      "url": "/users/123"
    }
  },
  "tags": {
    "http.status_code": 200
  }
}

Now, your application is also logging useful information. When a span is executing, your application code might emit log messages. Crucially, if you configure your logging agent (like Filebeat, Fluentd, or the Elastic Agent) correctly, you can inject the trace.id from the current APM trace into those log messages.

Imagine your application sees an error within that GET /users/:id trace. It might log something like this:

{
  "@timestamp": "2023-10-27T10:00:00.100Z",
  "log.level": "error",
  "message": "Failed to fetch user data from database",
  "trace": {
    "id": "a1b2c3d4e5f67890"
  },
  "user": {
    "id": "123"
  },
  "ecs": {
    "version": "1.6.0"
  }
}

See that trace.id field in the log message? That’s the magic. It’s a direct link back to the APM trace.

How to make this happen:

APM Agent Configuration: Ensure your APM agent is configured to capture and propagate trace context. For most languages, this is enabled by default. The agent will add trace.id and span.id to the current execution context.

Application Logging: In your application code, you need to access this trace context and include it in your log messages. Many logging libraries have integrations or mechanisms to automatically include contextual information.

Java (Logback example): You might use a MDC (Mapped Diagnostic Context) or a custom appender.

// Inside your code that's part of an APM trace
String traceId = Tracer.currentTraceId(); // Example, actual API varies by agent
MDC.put("trace.id", traceId);
logger.error("Database error occurred");
MDC.remove("trace.id"); // Clean up

Python (Loguru example):

from elasticapm.contrib.flask import backend
from loguru import logger

# Assuming you have flask and elasticapm configured
@app.route('/users/<user_id>')
def get_user(user_id):
    trace_id = backend.get_current_trace_id() # Example, actual API varies
    logger.bind(trace_id=trace_id).error(f"Fetching user {user_id}")
    # ... rest of your logic

Node.js (Winston example):

const apm = require('elastic-apm-node').start(); // Assuming initialized

app.get('/users/:id', (req, res) => {
  const traceId = apm.currentTraceIds.traceId;
  const logger = req.log.child({ trace_id: traceId }); // Assuming structured logging setup
  logger.error('Error fetching user');
  // ...
});

Logging Agent Configuration: Your logging agent needs to be configured to capture the trace.id from your application logs and send it to Elasticsearch. The key is to ensure the field containing the trace ID is correctly parsed and mapped.
- Elastic Agent (Logs integration):
```
# elastic-agent.yml snippet
logs:
  - type: file
    paths:
      - /var/log/my-app/*.log
    processors:
      - json:
          keys_under_root: true
          message_key: message
      - script:
          lang: javascript
          source: >
            function process(event) {
              if (event.data.trace && event.data.trace.id) {
                // Ensure it's correctly mapped to the ECS trace.id field
                event.data['trace.id'] = event.data.trace.id;
              }
            }
```
  Correction: The above script processor is generally not needed if your application logs are already structured JSON and the trace.id is in a field that the Elastic Agent’s JSON processor can pick up directly. If your application logs plain text and you’re using a grok or regex processor to extract fields, then you’d use a script to map the extracted trace ID to trace.id. For JSON logs, ensure the trace.id is directly accessible, e.g., {"trace": {"id": "..."}} or "trace.id": "...". The agent will then often map it automatically if ECS is enabled.
- Filebeat (with ingest pipeline):
```
# filebeat.yml snippet
filebeat.inputs:
- type: log
  paths:
    - /var/log/my-app/*.log
  json.from_string: true # If logs are JSON

output.elasticsearch:
  pipeline: ${user.name}-trace-log-pipeline

# ingest-pipelines/trace-log-pipeline.json
{
  "description": "Pipeline to add trace.id to logs",
  "processors": [
    {
      "set": {
        "field": "trace.id",

        "value": "{{ trace.id }}", # Adjust path if your trace ID is nested differently

        "if": "ctx?.trace?.id != null"
      }
    }
  ]
}
```
  Correction: For JSON logs, Filebeat often handles common ECS fields automatically if json.keys_under_root is true or if the structure matches. If your trace.id is nested like {"trace": {"id": "..."}}, you might need a jq processor or an ingest pipeline as shown. If it’s already {"trace.id": "..."}, it’s likely automatic.

The core idea: The APM agent makes trace.id available to your application’s execution context. Your application must then log this trace.id alongside its log messages. Finally, your logging agent must be configured to ingest these logs and ensure the trace.id field is correctly mapped to the ECS trace.id field in Elasticsearch.

Once this is set up, you can go to the APM UI, select a trace, and click "View Logs." This will open Kibana with a pre-filtered search for all log messages that share the trace.id of your selected trace. You can also go to Kibana Discover, search for a specific trace.id (e.g., trace.id : "a1b2c3d4e5f67890"), and see both the APM trace details and the corresponding log messages side-by-side.

The next logical step is to visualize the performance of those logs by correlating their timestamps and counts within the trace context.