validation-testing

What this skill does

# Power BI Validation and Self-Testing

## Overview

Validation skill for any TMDL, PBIR, DAX, or M artifact a developer (or Claude) generates. The goal: catch syntax, schema, and best-practice errors **locally** before a Fabric REST deploy fails. This skill is essential for the powerbi-expert agent's **Self-Validation Protocol** -- whenever the agent writes TMDL or PBIR, it should describe (or run) the matching validation step from this skill.

As of 2026, Power BI validation has four distinct layers, each catching a different class of error:

| Layer | TMDL Tool | PBIR Tool | What it catches |
|-------|-----------|-----------|-----------------|
| **1. Syntax / parser** | `TmdlSerializer.DeserializeDatabaseFromFolder` (.NET) | JSON schema validation (`$schema` URLs) | Indentation errors, invalid keywords, malformed JSON |
| **2. Object / schema** | `TmdlSerializer` -> `TmdlSerializationException` (valid syntax, invalid TOM metadata) | PBIR JSON schemas in `microsoft/json-schemas` repo | Invalid property combinations, type mismatches, missing required properties |
| **3. Best practice (BPA)** | Tabular Editor BPA rules (`BPARules.json`) or `semantic-link-labs.run_model_bpa` | PBI-InspectorV2 rules (`Base-rules.json`) | Anti-patterns, missing display folders, ambiguous relationships, naming conventions |
| **4. Lineage / cross-reference** | DAX measure references resolve, sortByColumn exists, calculation group precedence | Bookmarks reference real pages, drillthrough targets exist, theme files present | Dangling references, broken bookmarks, missing visuals |

**The cardinal rule:** never deploy without passing layers 1 and 2; never merge to main without passing layer 3.

## 2026 Validation Tooling Snapshot

| Tool | Validates | Runtime | Status |
|------|-----------|---------|--------|
| `TmdlSerializer` (Microsoft.AnalysisServices.Tabular) | TMDL syntax + TOM schema | .NET / pythonnet | GA |
| `Tabular Editor 2 CLI` (free) | TMDL load + BPA + custom C# scripts | .NET CLI | GA, free |
| `Tabular Editor 3 CLI` (paid) | Same + advanced rules + DAX debugger | .NET CLI | GA, commercial |
| `semantic-link-labs.run_model_bpa` | TMDL/TOM model BPA from Python | Fabric notebook (Python) | GA, ~60 rules built in |
| `semantic-link-labs.run_model_bpa_bulk` | BPA across all models in workspace | Fabric notebook | GA |
| `PBI-InspectorV2` ("Fab Inspector") | PBIR / PBIP / Fabric item rules | .NET CLI / Docker | v2.3+, GA |
| `pbi-tools` | PBIX extract/compile + basic TMDL | .NET CLI | Stable for TMDL, evolving for PBIR |
| `fabric-cicd` (built-in) | `parameter.yml` + repo structure pre-deployment | Python | GA |
| `DaxFormatter` API | DAX syntax | HTTP | GA |
| Microsoft TMDL VS Code extension | TMDL syntax in editor | VS Code | GA |
| Community `CPIM.TMDL-language-support` | TMDL + DAX + M semantic highlighting | VS Code | GA |
| INFO DAX functions | Live model introspection (replaces DMVs) | XMLA / Desktop | GA |

## Self-Validation Protocol (For Generated Artifacts)

When generating TMDL or PBIR artifacts inside an agent loop, follow this minimum protocol:

1. **Before writing files** -- mentally validate the structure: every object reference must resolve, every required property must be set.
2. **After writing files** -- run a syntax-level parse (TmdlSerializer for TMDL; JSON schema validation for PBIR).
3. **Before suggesting deployment** -- run a BPA pass (Tabular Editor CLI or semantic-link-labs).
4. **Report results inline** -- never silently swallow validation errors. Surface line numbers, file paths, and the specific rule that failed.

A valid agent response that generates a 50-line TMDL measure block should always be followed by either:
- (a) A validation script the user can paste, OR
- (b) An inline Bash/PowerShell/Python validation invocation if the environment supports it.

## TMDL Validation -- Layer 1 (Syntax Parser)

The fastest, lowest-dependency TMDL syntax check is `TmdlSerializer.DeserializeDatabaseFromFolder`. It throws:

- **`TmdlFormatException`** -- the TMDL text has invalid syntax (bad keyword, wrong indentation, malformed expression). Includes `Document`, `Line`, and `LineText` properties pointing to the exact location.
- **`TmdlSerializationException`** -- the TMDL text parses but produces invalid TOM metadata (e.g., a `column` references a `dataType` that doesn't exist, or a `partition` references an unknown data source).

**Minimal C# validator (.NET 8):**

```csharp
using Microsoft.AnalysisServices.Tabular;
using Microsoft.AnalysisServices.Tabular.Tmdl;

string folder = args[0];
try
{
    var db = TmdlSerializer.DeserializeDatabaseFromFolder(folder);
    Console.WriteLine($"OK: TMDL parsed. CompatLevel={db.CompatibilityLevel}, Tables={db.Model.Tables.Count}");
    return 0;
}
catch (TmdlFormatException fx)
{
    Console.Error.WriteLine($"SYNTAX ERROR  {fx.Document}:{fx.Line}");
    Console.Error.WriteLine($"  {fx.LineText}");
    Console.Error.WriteLine($"  -> {fx.Message}");
    return 1;
}
catch (TmdlSerializationException sx)
{
    Console.Error.WriteLine($"METADATA ERROR  {sx.Document}:{sx.Line}");
    Console.Error.WriteLine($"  {sx.Message}");
    return 2;
}
```

**One-liner via Tabular Editor 2 CLI** (no C# project required):

```bash
# Loads TMDL folder; non-zero exit on parse failure
TabularEditor.exe "MyProject.SemanticModel/definition" -B "MyProject.bim"
```

The `-B` (bim output) switch forces a deserialize + reserialize round-trip. Any parse failure exits non-zero with the error written to stderr.

For full scripted patterns and Python equivalents, see `references/tmdl-validation-recipes.md`.

## TMDL Validation -- Layer 3 (Best Practice Analyzer)

The Best Practice Analyzer (BPA) is the canonical anti-pattern checker for tabular models. It is the same engine in Tabular Editor 2, Tabular Editor 3, semantic-link-labs, and `Fabric > Workspace settings > Best Practice Analyzer`.

**Tabular Editor 2 CLI (free, recommended for CI):**

```bash
# Run BPA against a TMDL folder using the official Microsoft rule set
TabularEditor.exe "MyProject.SemanticModel/definition" \
  -A "https://raw.githubusercontent.com/TabularEditor/BestPracticeRules/master/BPARules.json" \
  -V \
  -G

# Exit codes:
#   0 = no violations
#   1 = warnings only
#   2 = errors found (any rule with Severity >= 3) -- pipeline should FAIL
```

**Switches that matter for CI/CD:**

| Switch | Purpose |
|--------|---------|
| `-A <rules.json>` | Run BPA with the specified rules file (URL or local path) |
| `-V` | Verbose output (lists each violation) |
| `-G` | GitHub Actions / Azure Pipelines log format (group sections, file paths) |
| `-D <conn>` | Deploy after passing BPA |
| `-S <script>` | Run a C# script before BPA (custom validation) |

**Severity-driven failure:** when a BPA rule is set to `Error` (level 3), the CLI **immediately stops and exits non-zero**. Set BPA rules to Error severity for any anti-pattern that should block a PR; set to Warning for advisory-only rules.

**Standard Microsoft rule set:** [TabularEditor/BestPracticeRules](https://github.com/TabularEditor/BestPracticeRules) -- ~60 rules covering performance, error prevention, DAX, maintenance, and naming. Always pin to a specific commit in CI.

For a complete BPA rule reference (every Microsoft rule explained, plus how to author custom rules), see `references/bpa-rules-reference.md`.

## TMDL Validation from Python (semantic-link-labs)

```python
%pip install semantic-link-labs -q
import sempy_labs as labs

# Run the default BPA against a deployed model
results = labs.run_model_bpa(
    dataset="SalesModel",
    workspace="Sales-Dev",
    extended=True,        # adds VertiPaq Analyzer stats for performance rules
)
results.head(20)

# Run BPA against every model in a workspace and store to delta
labs.run_model_bpa_bulk(
    workspace="Sales-Dev",
    extended=True,
)

# Custom rule set from a JSON file in the lakehouse
my_rules = labs.model_bpa_rules()  # built-in rule definitions
my_