deepeval-otel

What this skill does

# DeepEval OpenTelemetry Export

Use this skill to instrument an **AI application** — an LLM app, agent, RAG
pipeline, or chatbot — with **raw OpenTelemetry** so its traces land in
**Confident AI's Observatory**. No `deepeval` package is needed — it works with
any OTLP-capable OpenTelemetry SDK. The job is exactly two things: export to
the correct Confident AI OTLP endpoint, and set the `confident.*` attributes
Confident AI reads off each span.

## Scope: AI Applications Only

This skill instruments **AI applications only**. The `confident.*` attributes
and span types — `agent`, `llm`, `retriever`, `tool` — describe AI components,
and Confident AI's Observatory is built to evaluate and monitor AI behavior.

Instrument only the AI parts of the system: agent loops and planning, LLM
calls, retrieval / vector search, and tool calls. Do **not** apply `confident.*`
attributes to non-AI software (web servers, CRUD backends, database layers,
infrastructure) or to non-AI spans inside an otherwise-AI app — that data does
not belong in Confident AI and will not render meaningfully. If the target has
no LLM, agent, retrieval, or tool-calling component, this skill does not apply.

## When to Use vs the `deepeval` Skill

Use **this skill** for vendor-neutral OTLP export to Confident AI — pointing an
OpenTelemetry exporter at Confident AI and setting `confident.*` attributes.

Use the **`deepeval` skill** when the user wants to build a Python pytest eval
suite, generate datasets or goldens, write metrics, run `deepeval test run`, or
instrument with the `deepeval` SDK's `@observe` decorator. The two skills are
complementary, not alternatives.

## Prerequisites

- A Confident AI account and a `CONFIDENT_API_KEY`.
- An OpenTelemetry SDK for the application's language. For Python:
  `opentelemetry-sdk` and `opentelemetry-exporter-otlp-proto-http`.
- The Confident AI OTLP endpoint accepts **HTTP only** — never gRPC.

## How It Works

Confident AI exposes an OTLP/HTTP traces endpoint. Point any OpenTelemetry span
exporter at it with the `x-confident-api-key` header. Confident AI's exporter
then reads `confident.*` attributes off each span to build the trace and span
structure. Parent/child nesting comes from native OpenTelemetry span context,
not from any attribute.

## Workflow

1. Confirm the target is an AI application (it has LLM calls, an agent loop,
   retrieval, or tool calls). If it has none of these, stop — this skill does
   not apply. Then inspect for an existing OpenTelemetry setup (a
   `TracerProvider`, span exporters, or an OpenTelemetry Collector) and prefer
   repointing what exists over adding a parallel pipeline.
2. Choose the endpoint from the API key's region prefix. Read
   `references/endpoint-and-exporter.md`.
3. Wire (or repoint) an OTLP/HTTP span exporter with the `x-confident-api-key`
   header. For Python, start from `templates/confident_otel_setup.py`.
4. If the process runs other OpenTelemetry instrumentation or an APM agent
   (auto-instrumentation for HTTP/DB, Datadog, etc.), isolate the Confident AI
   export so only AI spans reach it — a dedicated pipeline or a span filter.
   Read "Export Only AI Spans" in `references/endpoint-and-exporter.md`.
5. Set `confident.span.*` attributes on spans; set `confident.trace.*` for
   trace-wide fields. Read `references/span-attributes.md` and
   `references/trace-attributes.md`.
6. Honor the OTLP data-type rules: JSON-encode dicts/metadata, use native
   arrays for string lists. See the Data-Type Rules in `span-attributes.md`.
7. If the app already emits OpenTelemetry GenAI semantic conventions, read
   `references/gen-ai-fallbacks.md` before adding redundant attributes.
8. Verify traces appear in the Confident AI Observatory.

## Core Principles

1. Instrument AI components only — agent, LLM, retriever, and tool spans.
   Never apply `confident.*` attributes to non-AI software or non-AI spans.
2. Export only AI spans. If the process has other OpenTelemetry
   instrumentation or an APM agent, isolate the Confident AI pipeline (a
   dedicated provider or a span filter) so non-AI spans — HTTP requests, DB
   queries, infra — are never exported to Confident AI.
3. Prefer repointing an existing OTLP exporter over adding a parallel one.
4. The `confident.*` attribute keys are the entire contract — they are the
   same in every language, so language choice is irrelevant.
5. Always use OTLP/HTTP. Confident AI's endpoint does not accept gRPC.
6. Honor OTLP data-type rules: attribute values must be primitives or
   homogeneous primitive lists; JSON-encode dicts and metadata.
7. Set `confident.span.type` explicitly when it is known; rely on `gen_ai.*`
   inference only as a fallback.
8. Never put secrets, credentials, or raw sensitive data into span attributes.

## References

| Topic | File |
| --- | --- |
| Endpoints, region selection, auth, exporter wiring | `references/endpoint-and-exporter.md` |
| Trace-level `confident.trace.*` attributes | `references/trace-attributes.md` |
| Span-level `confident.span.*` attributes and data-type rules | `references/span-attributes.md` |
| Standard OTel `gen_ai.*` fallback behavior | `references/gen-ai-fallbacks.md` |

## Templates

| Purpose | Template |
| --- | --- |
| Minimal Python OTLP exporter setup + example trace | `templates/confident_otel_setup.py` |