Explore OpenTelemetry and its role in observability

What this covers

Observability means you can explain what a distributed AI app is doing, where time goes and why a request failed, across APIs, queues, workers and databases. On AI-200 that story starts with OpenTelemetry instrumentation in your code, lands in Azure Monitor Application Insights, is stored in a Log Analytics workspace and is analyzed with KQL. This page sets up the vocabulary used by every other page in this path.

Exam weight: Secure, monitor, and troubleshoot Azure solutions (20–25%)

Skills measured

• "Trace distributed systems by using OpenTelemetry SDKs"
• "Write KQL queries to analyze logs and metrics"

Key concepts

OpenTelemetry

OpenTelemetry (OTel) is a vendor-neutral, open-source standard hosted by the CNCF for generating telemetry: traces, metrics and logs. It is not Azure-specific. You instrument once and pick an exporter for the backend you want (Azure Monitor, Jaeger, Prometheus, Grafana, Datadog and others) without changing instrumentation code.

OTel has four building blocks:

It gives you a consistent API across Python, .NET, Java, Node.js and Go, auto-instrumentation for common libraries, and manual spans for AI-specific work such as embedding generation, RAG retrieval and model calls.

Use case: an inference API calls Cosmos DB for vector search, then a model endpoint. OTel spans show which hop is slow.

The three pillars

A handy summary: metrics tell you that something changed, traces tell you where, logs tell you why.

Traces, spans and context propagation

A trace is the end-to-end record of one request. It is a tree of spans, where each span is a named, timed unit of work. Every span carries a trace ID (32 hex characters, shared by the whole trace), its own span ID (16 hex), a parent span ID (empty for the root span), a name, start and end timestamps, attributes (key-value context) and a status.

Context propagation carries the trace ID and the caller's span ID across process boundaries so Service B's spans join Service A's trace. OTel uses the W3C Trace Context header:

traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01
             ^version ^trace-id (32 hex)          ^parent span (16 hex) ^flags (01 = sampled)

Azure Monitor and Application Insights

You explore the data in portal experiences (Application Map, Failures, Performance, Transaction search) or by writing KQL in Logs.

Vocabulary mapping (OTel to Azure)

How it works

The same flow as plain text:

Your App (OpenTelemetry SDK)
      │  spans / traces / metrics / logs
      ▼
Azure Monitor Exporter
      │
      ▼
Application Insights (ingestion)
      │  stored as tables
      ▼
Log Analytics Workspace  →  queried with KQL

Microsoft packages the SDK, the Azure Monitor exporter, common instrumentation libraries and Azure resource detectors as the Azure Monitor OpenTelemetry Distro, so one call at startup wires the whole pipeline. See Add the OpenTelemetry SDK to an application.

Hands-on

Azure portal

1. Open your Application Insights resource.
2. Select Application Map to see one node per cloud role (service.name) and edges for calls between them.
3. Select Investigate > Performance, pick an operation, then a sample to open the end-to-end transaction waterfall.
4. Select Logs and confirm the schema pane lists the standard tables.

Azure CLI

az monitor app-insights component show \
  --app <app-insights-name> \
  --resource-group <resource-group> \
  --query "{conn:connectionString, workspace:workspaceResourceId}" -o json

Python

from azure.monitor.opentelemetry import configure_azure_monitor
from opentelemetry import trace

configure_azure_monitor()  # reads APPLICATIONINSIGHTS_CONNECTION_STRING
tracer = trace.get_tracer("rag-api")

with tracer.start_as_current_span("rag_query") as span:
    ctx = span.get_span_context()
    print(format(ctx.trace_id, "032x"))  # equals OperationId in App Insights

Exam traps

When to use OpenTelemetry + App Insights vs something else:

Quick review

• OpenTelemetry is vendor-neutral: the same code can export to Azure Monitor, Jaeger, Datadog and others.
• Three signals: traces, metrics, logs.
• Application Insights is the APM layer of Azure Monitor; its data lives in a Log Analytics workspace.
• Trace ID maps to operation_Id (classic) or OperationId (workspace).
• SERVER/CONSUMER spans become requests; CLIENT/PRODUCER/INTERNAL spans become dependencies.
• Span attributes surface as customDimensions (classic) or Properties (workspace).
• traceparent = version, 32-hex trace ID, 16-hex parent span ID, flags.

Practice

Practice: OpenTelemetry questions

Official references

• Explore OpenTelemetry and its role in observability (Learn unit)
• OpenTelemetry concepts
• Application Insights overview
• AI-200 study guide