dotnet-test-quality

What this skill does

# dotnet-test-quality

Test quality analysis for .NET projects. Covers code coverage collection with coverlet, human-readable coverage reports with ReportGenerator, CRAP (Change Risk Anti-Patterns) score analysis to identify undertested complex code, mutation testing with Stryker.NET to evaluate test suite effectiveness, and strategies for detecting and managing flaky tests.

**Version assumptions:** Coverlet 6.x+, ReportGenerator 5.x+, Stryker.NET 4.x+ (.NET 8.0+ baseline). Coverlet supports both the MSBuild integration (`coverlet.msbuild`) and the `coverlet.collector` data collector; examples use `coverlet.collector` as the recommended approach.

**Out of scope:** Test project scaffolding (creating projects, package references, coverlet setup) is owned by [skill:dotnet-add-testing]. Testing strategy and test type decisions are covered by [skill:dotnet-testing-strategy]. CI test reporting and pipeline integration -- see [skill:dotnet-gha-build-test] and [skill:dotnet-ado-build-test].

**Prerequisites:** Test project already scaffolded via [skill:dotnet-add-testing] with coverlet packages referenced. .NET 8.0+ baseline required.

Cross-references: [skill:dotnet-testing-strategy] for deciding what to test and coverage target guidance, [skill:dotnet-xunit] for xUnit test framework features and configuration.

---

## Code Coverage with Coverlet

Coverlet is the standard open-source code coverage library for .NET. It instruments assemblies at build time or via a data collector and produces coverage reports in multiple formats.

### Packages

```xml
<!-- Data collector approach (recommended) -->
<PackageReference Include="coverlet.collector" Version="8.0.0">
  <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
  <PrivateAssets>all</PrivateAssets>
</PackageReference>
```

### Collecting Coverage

```bash
# Collect coverage with Cobertura output (default for ReportGenerator)
dotnet test --collect:"XPlat Code Coverage"

# Specify output format explicitly
dotnet test --collect:"XPlat Code Coverage" \
  -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=cobertura

# Multiple formats
dotnet test --collect:"XPlat Code Coverage" \
  -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=cobertura,opencover
```

Coverage results are written to `TestResults/<guid>/coverage.cobertura.xml` under each test project's output directory.

### Filtering Coverage

Exclude generated code, test projects, or specific namespaces:

```bash
dotnet test --collect:"XPlat Code Coverage" \
  -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Exclude="[*.Tests]*,[*.IntegrationTests]*" \
  DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.ExcludeByAttribute="GeneratedCodeAttribute,ObsoleteAttribute,ExcludeFromCodeCoverageAttribute"
```

Or configure via a `runsettings` file for repeatability:

```xml
<!-- coverlet.runsettings -->
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
  <DataCollectionRunSettings>
    <DataCollectors>
      <DataCollector friendlyName="XPlat Code Coverage">
        <Configuration>
          <Format>cobertura</Format>
          <Exclude>[*.Tests]*,[*.IntegrationTests]*</Exclude>
          <ExcludeByAttribute>
            GeneratedCodeAttribute,ObsoleteAttribute,ExcludeFromCodeCoverageAttribute
          </ExcludeByAttribute>
          <ExcludeByFile>**/Migrations/**</ExcludeByFile>
          <IncludeTestAssembly>false</IncludeTestAssembly>
        </Configuration>
      </DataCollector>
    </DataCollectors>
  </DataCollectionRunSettings>
</RunSettings>
```

```bash
dotnet test --settings coverlet.runsettings
```

### Merge Coverage from Multiple Test Projects

When a solution has multiple test projects, merge their coverage into a single report:

```bash
# Run all tests, collecting coverage per project
dotnet test --collect:"XPlat Code Coverage"

# Find all coverage files and merge via ReportGenerator (see next section)
```

---

## Coverage Reports with ReportGenerator

ReportGenerator converts raw coverage data (Cobertura, OpenCover) into human-readable HTML reports with line-level highlighting.

### Installation

```bash
# Install as a global tool
dotnet tool install -g dotnet-reportgenerator-globaltool

# Or as a local tool
dotnet tool install dotnet-reportgenerator-globaltool
```

### Generating Reports

```bash
# Single coverage file
reportgenerator \
  -reports:"tests/MyApp.Tests/TestResults/*/coverage.cobertura.xml" \
  -targetdir:"coverage-report" \
  -reporttypes:"Html;TextSummary"

# Multiple test projects (glob pattern merges automatically)
reportgenerator \
  -reports:"**/TestResults/*/coverage.cobertura.xml" \
  -targetdir:"coverage-report" \
  -reporttypes:"Html;Cobertura;TextSummary"
```

### Report Types

| Type | Description | Use Case |
|------|-------------|----------|
| `Html` | Interactive HTML with line highlighting | Local developer review |
| `HtmlInline_AzurePipelines` | HTML optimized for Azure DevOps | CI artifact |
| `Cobertura` | Merged Cobertura XML | Input for other tools |
| `TextSummary` | Plain text summary | CLI/CI output |
| `Badges` | SVG coverage badges | README badges |
| `MarkdownSummaryGithub` | GitHub-flavored markdown | PR comments |

### Example: Full Coverage Pipeline

```bash
#!/bin/bash
# clean previous results
rm -rf coverage-report TestResults

# run tests with coverage
dotnet test --collect:"XPlat Code Coverage" --results-directory TestResults

# generate merged HTML report
reportgenerator \
  -reports:"**/TestResults/*/coverage.cobertura.xml" \
  -targetdir:"coverage-report" \
  -reporttypes:"Html;TextSummary;Badges"

# display summary
cat coverage-report/Summary.txt
```

### Setting Coverage Thresholds

Enforce minimum coverage in CI by parsing the text summary or using a threshold parameter:

```bash
# ReportGenerator does not enforce thresholds directly.
# Parse the summary or use dotnet-coverage (Microsoft) for threshold enforcement.

# Alternative: use coverlet's built-in threshold via MSBuild
dotnet test /p:CollectCoverage=true \
  /p:Threshold=80 \
  /p:ThresholdType=line \
  /p:ThresholdStat=total
```

**Note:** The `/p:Threshold` parameter requires the `coverlet.msbuild` package (not `coverlet.collector`). For `coverlet.collector` workflows, enforce thresholds by parsing the ReportGenerator text summary in your CI script.

---

## CRAP Analysis

CRAP (Change Risk Anti-Patterns) scores identify methods that are both complex and poorly tested. A high CRAP score means the method has high cyclomatic complexity and low code coverage -- a risky combination.

### Formula

```
CRAP(m) = complexity(m)^2 * (1 - coverage(m)/100)^3 + complexity(m)
```

Where:
- `complexity(m)` = cyclomatic complexity of method m
- `coverage(m)` = code coverage percentage of method m (0-100)

### Interpreting CRAP Scores

| CRAP Score | Risk Level | Action |
|------------|------------|--------|
| < 5 | Low | Method is simple or well-tested |
| 5-15 | Moderate | Review -- may need additional tests |
| 15-30 | High | Prioritize: add tests or reduce complexity |
| > 30 | Critical | Refactor and add tests immediately |

### Generating CRAP Reports

ReportGenerator includes CRAP analysis when using OpenCover format as input:

```bash
# Step 1: Collect coverage in OpenCover format
dotnet test --collect:"XPlat Code Coverage" \
  -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=opencover

# Step 2: Generate report with risk hotspot analysis
reportgenerator \
  -reports:"**/TestResults/*/coverage.opencover.xml" \
  -targetdir:"coverage-report" \
  -reporttypes:"Html;RiskHotspots"
```

The Risk Hotspots report highlights methods sorted by CRAP score, showing:
- Method name and containing class
- Cyclomatic complexity
- Code coverage percentage
- Computed CRAP score

### Using CRAP Scores Effectively

```csharp
// Example: a method with high complexity and low coverag