policyengine-microsimulation

What this skill does

# PolicyEngine Microsimulation

## Documentation References

- **Microsimulation API**: https://policyengine.github.io/policyengine-us/usage/microsimulation.html
- **Parameter Discovery**: https://policyengine.github.io/policyengine-us/usage/parameter-discovery.html
- **Reform.from_dict()**: https://policyengine.github.io/policyengine-core/usage/reforms.html

## Latest rules and microdata provenance

When the user asks to use the latest PolicyEngine rules, data, microdata,
variable weights, sampled households, or default datasets, verify the package
release from authoritative live metadata before running the microsimulation.
Do not infer "latest" from search snippets, local installed versions, lockfiles,
or repo constraints.

For `policyengine.py` bundled rules plus default microdata:

```bash
# Verify the latest policyengine.py release.
python - <<'PY'
import json
import urllib.request

with urllib.request.urlopen(
    "https://pypi.org/pypi/policyengine/json",
    timeout=20,
) as response:
    print(json.load(response)["info"]["version"])
PY
python -m pip index versions policyengine

# Install the exact verified version with the country extra.
uv pip install "policyengine[us]==X.Y.Z"
```

Then confirm the resolved model/data bundle without importing top-level
`policyengine`, which may initialize countries and private data you are not
using:

```bash
python - <<'PY'
import json
from importlib import metadata
from pathlib import Path

for package in ["policyengine", "policyengine-us"]:
    print(f"{package}=={metadata.version(package)}")
    direct_url = metadata.distribution(package).read_text("direct_url.json")
    if direct_url:
        print(f"{package} direct_url={direct_url}")

manifest_path = Path(
    metadata.distribution("policyengine").locate_file(
        "policyengine/data/release_manifests/us.json"
    )
)
manifest = json.loads(manifest_path.read_text())
print(json.dumps({
    "bundle_id": manifest.get("bundle_id"),
    "model_package": manifest.get("model_package"),
    "data_package": manifest.get("data_package"),
    "default_dataset": manifest.get("default_dataset"),
    "default_dataset_uri": (
        manifest.get("certified_data_artifact") or {}
    ).get("uri"),
    "certification": manifest.get("certification"),
}, indent=2, sort_keys=True))
PY
```

Only call a run "latest" after this verification. If PyPI, npm, GitHub, or the
official provider source cannot be reached, report that latest could not be
verified instead of guessing.

## CRITICAL: Use calc() with MicroSeries — never strip weights or fetch them manually

**MicroSeries handles all weighting automatically. Never convert to numpy, strip types, or do manual weight math.**

### NEVER strip MicroSeries weights

`calc()` and `calculate()` return MicroSeries with embedded weights AND entity context. Any of these operations strip both, producing silently wrong results:

| Anti-pattern | Why it's wrong |
|---|---|
| `np.array(series)` | Converts to unweighted numpy array |
| `series.values` / `series.to_numpy()` | Same — strips weights and entity context |
| `series.astype(float)` / `.astype(int)` | Converts MicroSeries to plain pandas Series, losing weight metadata |
| `float(series.sum())` | Premature scalar extraction — usually a sign of manual weight math nearby |
| `np.average(x, weights=w)` | Manual weighting — `.mean()` already does this correctly |

### NEVER fetch weight variables manually

`calc()` returns MicroSeries that already knows its weights. There is no reason to fetch `household_weight`, `spm_unit_weight`, `person_weight`, or `tax_unit_weight` yourself. If you're writing `sim.calc("spm_unit_weight", ...)`, something is wrong — `calc()` handles weight mapping internally via the `map_to` parameter.

```python
# ❌ WRONG — fetching weights and doing manual math
liheap = sim.calc("dc_liheap_payment", period=2026).astype(float)  # strips weights
weights = sim.calc("spm_unit_weight", period=2026).astype(float)    # unnecessary
total = float((liheap * weights).sum())                              # manual weighting
avg = float(np.average(liheap[liheap > 0], weights=weights[liheap > 0]))  # numpy

# ✅ CORRECT — MicroSeries does everything
liheap = sim.calc("dc_liheap_payment", period=2026)
total = liheap.sum()                  # Weighted total
avg = liheap[liheap > 0].mean()       # Weighted mean of recipients

# ❌ WRONG — np.array() strips weights AND entity context
change_arr = np.array(sim.calc("income_tax", period=2026))
weights = np.array(sim.calc("household_weight", period=2026))
# These may be DIFFERENT LENGTHS (tax units vs households)!
losers = weights[change_arr < -1].sum()  # SILENTLY WRONG

# ✅ CORRECT — keep as MicroSeries, all operations are weighted
income_tax_b = baseline.calc("income_tax", period=2026)
income_tax_r = reformed.calc("income_tax", period=2026)
tax_change = income_tax_r - income_tax_b
loser_count = (tax_change > 1).sum()    # Weighted count of losers
loser_share = (tax_change > 1).mean()   # Weighted share of losers
avg_change = tax_change.mean()          # Weighted mean change
total_change = tax_change.sum()         # Weighted total
```

### Entity-level matching

When comparing variables across entities, use `map_to` to align them — never mix raw arrays from different entities:

```python
# ❌ WRONG - income_tax is tax_unit level, household_weight is household level
tax = np.array(sim.calc("income_tax", period=2026))        # 23K tax units
wt = np.array(sim.calc("household_weight", period=2026))   # 15K households
# tax and wt have DIFFERENT lengths — any indexing is wrong

# ✅ CORRECT - map income_tax to household level, or just use MicroSeries
tax = sim.calc("income_tax", period=2026)  # tax_unit level MicroSeries
losers = (tax > 0).sum()  # Weighted count, correct entity
```

## Quick start

```python
from policyengine_us import Microsimulation
from policyengine_core.reforms import Reform

baseline = Microsimulation()
reform = Reform.from_dict({
    'gov.irs.credits.ctc.amount.base[0].amount': {'2026-01-01.2100-12-31': 3000}
}, 'policyengine_us')
reformed = Microsimulation(reform=reform)

# calc() returns MicroSeries - all operations are weighted automatically
baseline_income = baseline.calc('household_net_income', period=2026, map_to='person')
reformed_income = reformed.calc('household_net_income', period=2026, map_to='person')
change = reformed_income - baseline_income

# Weighted stats - no manual weight handling needed!
print(f"Average impact: ${change.mean():,.0f}")
print(f"Total cost: ${change.sum()/1e9:,.1f}B")
print(f"Share losing: {(change < 0).mean():.1%}")
```

## API methods

- **US: `sim.calc()`** — `policyengine_us.Microsimulation` uses `.calc()`. Do NOT use `.calculate()` for US.
- **UK: `sim.calculate()`** — `policyengine_uk.Microsimulation` uses `.calculate()`. It does NOT have `.calc()`.
- Both return MicroSeries with automatic weighting. Use `.sum()`, `.mean()`, arithmetic operators.
- Use the `period=` keyword: `sim.calc("variable", period=2026)`, not `sim.calc("variable", 2026)`.

## Creating reforms

### US (policyengine_us.Microsimulation)
```python
from policyengine_us import Microsimulation
from policyengine_core.reforms import Reform

reform = Reform.from_dict({
    'gov.irs.credits.ctc.amount.base[0].amount': {'2026-01-01.2100-12-31': 3600},
}, 'policyengine_us')

baseline = Microsimulation()
reformed = Microsimulation(reform=reform)
```

### UK (policyengine_uk.Microsimulation)
```python
from policyengine_uk import Microsimulation

# UK: pass reform dict directly — do NOT use Reform.from_dict()
reform = {
    'gov.hmrc.income_tax.allowances.personal_allowance.amount': {'2026-01-01.2100-12-31': 15000}
}

baseline = Microsimulation()
reformed = Microsimulation(reform=reform)
```

**IMPORTANT**: For UK, pass the dict directly to `Microsimulation(reform=dict)`. `Reform.from_dict()` works for US but causes errors in UK.

### Dataset argument
- **National**: Omit dataset argument entirely (default is correc