dotnet-profiling

What this skill does

# dotnet-profiling

Diagnostic tool guidance for investigating .NET performance problems. Covers real-time metric monitoring with dotnet-counters, event tracing and flame graph generation with dotnet-trace, and memory dump capture and analysis with dotnet-dump. Focuses on interpreting profiling data (reading flame graphs, analyzing heap dumps, correlating GC metrics) rather than just invoking tools.

**Version assumptions:** .NET SDK 8.0+ baseline. All three diagnostic tools (dotnet-counters, dotnet-trace, dotnet-dump) ship with the .NET SDK -- no separate installation required.

**Out of scope:** OpenTelemetry metrics collection and distributed tracing setup -- see [skill:dotnet-observability]. Microbenchmarking setup (BenchmarkDotNet) is owned by this epic's companion skill -- see [skill:dotnet-benchmarkdotnet]. Performance architecture patterns (Span\<T\>, ArrayPool, sealed devirtualization) are owned by this epic's companion skill -- see [skill:dotnet-performance-patterns]. Continuous benchmark regression detection in CI -- see [skill:dotnet-ci-benchmarking]. Architecture patterns (caching, resilience) -- see [skill:dotnet-architecture-patterns].

Cross-references: [skill:dotnet-observability] for GC/threadpool metrics interpretation and OpenTelemetry correlation, [skill:dotnet-benchmarkdotnet] for structured benchmarking after profiling identifies hot paths, [skill:dotnet-performance-patterns] for optimization patterns to apply based on profiling results.

---

## dotnet-counters -- Real-Time Metric Monitoring

### Overview

`dotnet-counters` provides real-time monitoring of .NET runtime metrics without modifying application code. Use it as a first-pass triage tool to identify whether a performance problem is CPU-bound, memory-bound, or I/O-bound before reaching for heavier instrumentation.

### Monitoring Running Processes

```bash
# List running .NET processes
dotnet-counters ps

# Monitor default runtime counters for a process
dotnet-counters monitor --process-id <PID>

# Monitor with a specific refresh interval (seconds)
dotnet-counters monitor --process-id <PID> --refresh-interval 2
```

### Key Built-In Counter Providers

| Provider | Counters | What It Tells You |
|----------|----------|-------------------|
| `System.Runtime` | CPU usage, GC heap size, Gen 0/1/2 collections, threadpool queue length, exception count | Overall runtime health |
| `Microsoft.AspNetCore.Hosting` | Request rate, request duration, active requests | HTTP request throughput and latency |
| `Microsoft.AspNetCore.Http.Connections` | Connection duration, current connections | WebSocket/SignalR connection load |
| `System.Net.Http` | Requests started/failed, active requests, connection pool size | Outbound HTTP client behavior |
| `System.Net.Sockets` | Bytes sent/received, datagrams, connections | Network I/O volume |

### Monitoring Specific Providers

```bash
# Monitor runtime and ASP.NET counters together
dotnet-counters monitor --process-id <PID> \
  --counters System.Runtime,Microsoft.AspNetCore.Hosting

# Monitor only GC-related counters
dotnet-counters monitor --process-id <PID> \
  --counters System.Runtime[gc-heap-size,gen-0-gc-count,gen-1-gc-count,gen-2-gc-count]
```

### Custom EventCounters

Applications can publish custom counters for domain-specific metrics:

```csharp
using System.Diagnostics.Tracing;

[EventSource(Name = "MyApp.Orders")]
public sealed class OrderMetrics : EventSource
{
    public static readonly OrderMetrics Instance = new();

    private EventCounter? _orderProcessingTime;
    private IncrementingEventCounter? _ordersProcessed;

    private OrderMetrics()
    {
        _orderProcessingTime = new EventCounter("order-processing-time", this)
        {
            DisplayName = "Order Processing Time (ms)",
            DisplayUnits = "ms"
        };
        _ordersProcessed = new IncrementingEventCounter("orders-processed", this)
        {
            DisplayName = "Orders Processed",
            DisplayRateTimeScale = TimeSpan.FromSeconds(1)
        };
    }

    public void RecordProcessingTime(double milliseconds)
        => _orderProcessingTime?.WriteMetric(milliseconds);

    public void RecordOrderProcessed()
        => _ordersProcessed?.Increment();

    protected override void Dispose(bool disposing)
    {
        _orderProcessingTime?.Dispose();
        _ordersProcessed?.Dispose();
        base.Dispose(disposing);
    }
}
```

Monitor custom counters:

```bash
dotnet-counters monitor --process-id <PID> --counters MyApp.Orders
```

### Interpreting Counter Data

Use counter values to direct further investigation. See [skill:dotnet-observability] for correlating these runtime metrics with OpenTelemetry traces:

| Symptom | Counter Evidence | Next Step |
|---------|------------------|-----------|
| High CPU usage | `cpu-usage` > 80%, `threadpool-queue-length` low | CPU profiling with dotnet-trace |
| Memory growth | `gc-heap-size` increasing, frequent Gen 2 GC | Memory dump with dotnet-dump |
| Thread starvation | `threadpool-queue-length` growing, `threadpool-thread-count` at max | Check for sync-over-async or blocking calls |
| Request latency | `request-duration` high, `active-requests` normal | Trace individual requests with dotnet-trace |
| GC pauses | High `gen-2-gc-count`, `time-in-gc` > 10% | Allocation profiling with dotnet-trace gc-collect |

### Exporting Counter Data

```bash
# Export to CSV for analysis
dotnet-counters collect --process-id <PID> \
  --format csv \
  --output counters.csv \
  --counters System.Runtime

# Export to JSON for programmatic consumption
dotnet-counters collect --process-id <PID> \
  --format json \
  --output counters.json
```

---

## dotnet-trace -- Event Tracing and Flame Graphs

### Overview

`dotnet-trace` captures detailed event traces from a running .NET process. Traces can be analyzed as flame graphs to identify CPU hot paths, or configured for allocation tracking to find GC pressure sources.

### CPU Sampling

CPU sampling records stack frames at a fixed interval to build a statistical profile of where the application spends time:

```bash
# Collect a CPU sampling trace (default profile)
dotnet-trace collect --process-id <PID> --duration 00:00:30

# Collect with the cpu-sampling profile (explicit)
dotnet-trace collect --process-id <PID> \
  --profile cpu-sampling \
  --output cpu-trace.nettrace
```

### CPU Sampling vs Instrumentation

| Approach | Overhead | Best For | Tool |
|----------|----------|----------|------|
| CPU sampling | Low (~2-5%) | Finding CPU hot paths in production | dotnet-trace `--profile cpu-sampling` |
| Instrumentation | High (10-50%+) | Exact call counts, method entry/exit timing | Rider/VS profiler, PerfView |

CPU sampling is safe for production use due to low overhead. Use it as the default approach. Reserve instrumentation for development environments where exact call counts matter.

### Flame Graph Generation

Trace files (`.nettrace`) must be converted to a flame graph format for visual analysis:

**Using Speedscope (browser-based, recommended):**

```bash
# Convert to Speedscope format
dotnet-trace convert cpu-trace.nettrace --format Speedscope

# Opens cpu-trace.speedscope.json -- load at https://www.speedscope.app/
```

**Using PerfView (Windows, deep .NET integration):**

```bash
# Convert to Chromium trace format (also viewable in chrome://tracing)
dotnet-trace convert cpu-trace.nettrace --format Chromium
```

### Reading Flame Graphs

Flame graphs display call stacks where:

- **Width** of a frame represents the proportion of total sample time spent in that function (wider = more time)
- **Height** represents call stack depth (taller stacks = deeper call chains)
- **Color** is typically arbitrary (not meaningful) unless the tool uses a specific color scheme

**Analysis workflow:**

1. Look for **wide plateaus** -- functions that consume a large proportion of samples
2. Follow the widest frames **upward** to find which callers contribute the mos