sentry-observability

What this skill does

# Sentry Observability Integration

## Overview

Wire Sentry into your logging, metrics, APM, and dashboard toolchain so every error carries full context and every metric correlates back to root-cause events. This skill covers three integration layers: structured logging (winston, pino, structlog) with Sentry event ID correlation, business metrics with error-rate tracking, and cross-tool linking via Sentry Discover, Grafana webhooks, and APM tools.

See also: [Logging integration details](references/logging-integration.md) | [Metrics patterns](references/metrics-integration.md) | [APM tool cross-linking](references/apm-tool-integration.md)

## Prerequisites

- Sentry SDK v8+ installed (`@sentry/node` for Node.js, `sentry-sdk` for Python)
- At least one structured logger configured (winston, pino, or structlog)
- Sentry project DSN available in environment (`SENTRY_DSN`)
- Dashboard platform accessible (Sentry Discover, Grafana, or Datadog)
- Alert routing strategy decided (who gets paged, where warnings go)

## Instructions

### Step 1 — Attach Sentry Event IDs to Structured Logs

The core pattern: every log line that triggers a Sentry event carries the event ID, and every Sentry event carries the log context. This creates a two-way link between your log aggregator and Sentry.

**Winston (Node.js) — custom transport:**

```typescript
import winston from 'winston';
import * as Sentry from '@sentry/node';

class SentryTransport extends winston.Transport {
  log(info: any, callback: () => void) {
    setImmediate(callback);

    if (info.level === 'error' || info.level === 'fatal') {
      const error = info.error instanceof Error
        ? info.error
        : new Error(info.message);

      Sentry.withScope((scope) => {
        scope.setTag('logger', 'winston');
        scope.setContext('log_entry', {
          level: info.level,
          timestamp: info.timestamp,
          service: info.service,
        });
        const eventId = Sentry.captureException(error);
        info.sentry_event_id = eventId;
        info.sentry_url = `https://${process.env.SENTRY_ORG}.sentry.io/issues/?query=${eventId}`;
      });
    }
  }
}

const logger = winston.createLogger({
  defaultMeta: { service: 'api-gateway' },
  transports: [
    new winston.transports.Console({ format: winston.format.json() }),
    new SentryTransport(),
  ],
});
```

**Pino (Node.js) — hooks pattern:**

```typescript
import pino from 'pino';
import * as Sentry from '@sentry/node';

const logger = pino({
  hooks: {
    logMethod(inputArgs, method, level) {
      if (level >= 50) { // 50 = error, 60 = fatal
        const [obj, msg] = typeof inputArgs[0] === 'object'
          ? [inputArgs[0], inputArgs[1]]
          : [{}, inputArgs[0]];

        Sentry.withScope((scope) => {
          scope.setTag('logger', 'pino');
          const eventId = Sentry.captureException(
            obj.err instanceof Error ? obj.err : new Error(String(msg))
          );
          if (typeof inputArgs[0] === 'object') {
            inputArgs[0].sentry_event_id = eventId;
          }
        });
      }
      return method.apply(this, inputArgs);
    },
  },
});
```

For Python structlog integration, see [logging-integration.md](references/logging-integration.md).

**Request ID correlation middleware:**

```typescript
import { randomUUID } from 'crypto';
import * as Sentry from '@sentry/node';

app.use((req, res, next) => {
  const requestId = (req.headers['x-request-id'] as string) || randomUUID();
  req.requestId = requestId;
  res.setHeader('x-request-id', requestId);
  Sentry.setTag('request_id', requestId);
  req.log = logger.child({ requestId, path: req.path });
  next();
});
```

### Step 2 — Correlate Errors with Business Metrics and APM

Connect Sentry events to your metrics pipeline and decide when Sentry performance monitoring is sufficient versus when to add Datadog or New Relic.

**Sentry custom metrics (built-in, no extra tools):**

```typescript
import * as Sentry from '@sentry/node';

// Counter — track error rates alongside business events
Sentry.metrics.increment('checkout.attempted', 1, {
  tags: { payment_provider: 'stripe', plan: 'enterprise' },
});

Sentry.metrics.increment('checkout.failed', 1, {
  tags: { payment_provider: 'stripe', failure_reason: 'timeout' },
});

// Distribution — track latency with error correlation
Sentry.metrics.distribution('api.response_time', responseTimeMs, {
  tags: { endpoint: '/api/orders', status_code: String(res.statusCode) },
  unit: 'millisecond',
});

// Gauge — track queue depth, connection pool size
Sentry.metrics.gauge('db.pool.active', pool.activeCount, {
  tags: { database: 'primary' },
});

// Set — track unique affected users during incidents
Sentry.metrics.set('incident.affected_users', userId, {
  tags: { incident: 'payment-outage-2026-03' },
});
```

For Prometheus dual-write patterns, see [metrics-integration.md](references/metrics-integration.md).

**When to use Sentry performance vs Datadog/New Relic:**

| Scenario | Use Sentry Performance | Use Datadog/New Relic |
|----------|----------------------|----------------------|
| Frontend + backend in one view | Yes — unified error + perf traces | Overkill if Sentry covers your stack |
| Infrastructure metrics (CPU, memory) | No — Sentry does not collect infra | Yes — native host agent collection |
| 100+ custom metric series | Limited query constraints | Yes — built for high-cardinality |
| Budget-constrained, < 5 services | Yes — one tool, one bill | Unnecessary cost |

**Datadog + Sentry cross-linking via `beforeSend`:**

```typescript
import tracer from 'dd-trace';
import * as Sentry from '@sentry/node';

// dd-trace MUST be initialized before @sentry/node
Sentry.init({
  dsn: process.env.SENTRY_DSN,
  beforeSend(event) {
    const span = tracer.scope().active();
    if (span) {
      const traceId = span.context().toTraceId();
      event.tags = { ...event.tags, 'dd.trace_id': traceId };
      event.contexts = {
        ...event.contexts,
        datadog: {
          trace_url: `https://app.datadoghq.com/apm/trace/${traceId}`,
          trace_id: traceId,
        },
      };
    }
    return event;
  },
});
```

For New Relic correlation patterns, see [apm-tool-integration.md](references/apm-tool-integration.md).

### Step 3 — Build Dashboards and Connect External Tools

Use Sentry Discover for error analytics, set up Grafana webhooks for unified dashboards, and link Sentry events to external tools via `setContext`.

**Linking all tools via `Sentry.setContext('monitoring', ...)`:**

```typescript
import * as Sentry from '@sentry/node';

function setMonitoringContext(req: Request) {
  const traceId = Sentry.getActiveSpan()?.spanContext().traceId;
  const spanId = Sentry.getActiveSpan()?.spanContext().spanId;
  const requestId = req.headers['x-request-id'] as string || crypto.randomUUID();

  // setContext creates a named section in the Sentry event sidebar
  Sentry.setContext('monitoring', {
    traceId,
    spanId,
    requestId,
    grafana_dashboard: `https://grafana.example.com/d/abc123?var-trace_id=${traceId}`,
    kibana_logs: `https://kibana.example.com/app/logs?query=request_id:${requestId}`,
    datadog_trace: traceId
      ? `https://app.datadoghq.com/apm/trace/${traceId}`
      : undefined,
  });

  Sentry.setTag('request_id', requestId);
  Sentry.setTag('trace_id', traceId || 'none');
  Sentry.setTag('deployment', process.env.DEPLOYMENT_ID || 'unknown');
}

app.use((req, res, next) => {
  setMonitoringContext(req);
  next();
});
```

**Grafana integration via Sentry webhooks:**

Configure in Settings > Integrations > Internal Integrations. Point the webhook URL at a receiver that transforms Sentry events into Grafana annotations:

```typescript
// Receive Sentry webhook, create Grafana annotation
app.post('/sentry-to-grafana', async (req, res) => {
  const { event } = req.body;
  if (!event) return res.status(200).send('ignored');

  await fetch(`${process.env.GRAFANA_URL}/api/ann