OpenTelemetry tracing doesn’t add overhead; it reveals the overhead you already have.
Let’s see it in action. Imagine a simple Flask app that talks to a database:
from flask import Flask, request
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.sdk.resources import SERVICE_NAME, Resource
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.instrumentation.flask import FlaskInstrumentor
from opentelemetry.instrumentation.requests import RequestsInstrumentor
import requests
import sqlite3
# Configure OpenTelemetry
resource = Resource(attributes={SERVICE_NAME: "my-flask-app"})
provider = TracerProvider(resource=resource)
# For local testing, you might send to an OTLP collector running on default port
# For production, configure your exporter appropriately (e.g., Jaeger, Zipkin, cloud provider)
span_processor = BatchSpanProcessor(OTLPSpanExporter(insecure=True))
provider.add_span_processor(span_processor)
trace.set_tracer_provider(provider)
app = Flask(__name__)
# Instrument Flask and requests
FlaskInstrumentor().instrument_app(app)
RequestsInstrumentor().instrument_requests()
db_conn = None
def get_db():
global db_conn
if db_conn is None:
db_conn = sqlite3.connect("data.db")
db_conn.execute("CREATE TABLE IF NOT EXISTS items (id INTEGER PRIMARY KEY, name TEXT)")
return db_conn
@app.route('/')
def index():
return "Hello, World!"
@app.route('/items')
def get_items():
db = get_db()
cursor = db.cursor()
cursor.execute("SELECT * FROM items")
items = cursor.fetchall()
return {"items": items}
@app.route('/fetch-external')
def fetch_external():
try:
# This request will now be automatically traced
response = requests.get("https://httpbin.org/delay/1", timeout=5)
return {"external_data": response.json()}
except requests.exceptions.RequestException as e:
return {"error": str(e)}, 500
if __name__ == '__main__':
# Ensure the database is initialized
get_db()
app.run(debug=True, port=5000)
When you run this and hit /items, you’ll see a trace in your OpenTelemetry collector (like Jaeger or Tempo) showing the GET /items request, the SQL query execution, and the database connection. Hitting /fetch-external will show the incoming Flask request and the outgoing requests.get call as separate, linked spans, including the 1-second delay on httpbin.org.
The core problem OpenTelemetry tracing solves is distributed visibility. In a microservices world, a single user request can traverse dozens of services, network hops, and asynchronous queues. Without tracing, debugging a slow or failing request is like trying to find a needle in a haystack blindfolded. You can see the service logs, but you don’t know how they connect or where the latency is accumulating.
OpenTelemetry provides a standardized way to generate, collect, and export trace data. The trace module is your entry point. TracerProvider is the central component that manages tracers and span processors. Resource attaches metadata (like the service name) to all telemetry data from that instance. SpanProcessor decides what to do with spans as they are completed; BatchSpanProcessor buffers spans and sends them in batches, which is more efficient. OTLPSpanExporter is just one way to send data, using the OpenTelemetry Protocol (OTLP) over gRPC.
The magic for Flask and requests comes from FlaskInstrumentor and RequestsInstrumentor. These are automatic instrumentation libraries. When FlaskInstrumentor().instrument_app(app) runs, it attaches hooks to Flask’s request lifecycle. For every incoming request, it starts a new span. If that request involves an outgoing HTTP call made via the requests library, RequestsInstrumentor (which is also initialized) automatically creates a child span for that outgoing request, linking it back to the original Flask request span. This creates the parent-child relationships that define a trace.
You control tracing by configuring the TracerProvider and its SpanProcessor. You can add custom attributes to spans, create manual spans for specific code blocks, and set sampling rates to control how many traces are exported. For example, to add a user ID to every span in a request:
from flask import g
@app.before_request
def add_user_id():
# Assume user is authenticated and ID is available in session or g
user_id = "user-123" # Replace with actual user ID retrieval
span = trace.get_current_span()
if span:
span.set_attribute("user.id", user_id)
This allows you to filter and analyze traces based on user activity.
The one thing most people don’t realize is that the requests library’s default timeout is None, meaning it will wait forever. When you instrument requests with OpenTelemetry, the span for that outgoing request will also wait forever if no timeout is set, potentially holding up your entire trace and blocking resources. Always set a sensible timeout on your requests calls when using automatic instrumentation, or your "unseen" infinite waits will become very visible and problematic.
The next concept to explore is correlating traces with logs and metrics, often using the same Resource attributes and trace IDs.