policyengine-core

What this skill does

# PolicyEngine Core

PolicyEngine Core is the microsimulation engine that powers all PolicyEngine calculations. It's a fork of OpenFisca-Core adapted for PolicyEngine's needs.

## For Users

### What is Core?

When you use policyengine.org to calculate taxes or benefits, PolicyEngine Core is the "calculator" running behind the scenes.

**Core provides:**
- The simulation engine that processes tax rules
- Variable and parameter management
- Entity relationships (person → family → household)
- Period handling (2024, 2025, etc.)

You don't interact with Core directly - you use it through:
- **Web app:** policyengine.org
- **Python packages:** policyengine-us, policyengine-uk
- **API:** api.policyengine.org

### Why Core Matters

Core ensures:
- ✅ **Accuracy** - Calculations follow official rules exactly
- ✅ **Consistency** - Same rules applied everywhere
- ✅ **Transparency** - All rules traceable to legislation
- ✅ **Performance** - Vectorized calculations for speed

## For Analysts

### Understanding Core Concepts

When writing PolicyEngine code, you'll encounter Core concepts:

**Variables:**
- Represent quantities (income_tax, ctc, snap, etc.)
- Defined for specific entities (person, household, tax_unit)
- Calculated from formulas or set directly

**Parameters:**
- Policy rules that change over time (tax rates, benefit amounts)
- Organized hierarchically (gov.irs.credits.ctc.amount.base_amount)
- Stored in YAML files

**Entities:**
- Person: Individual
- Family: Family unit
- Tax unit: Tax filing unit
- Household: Physical household
- Marital unit: Marital status grouping
- SPM unit: Supplemental Poverty Measure unit

**Periods:**
- Year: 2024, 2025, etc.
- Month: 2024-01, 2024-02, etc.
- Specific dates: 2024-06-15

### Core in Action

```python
from policyengine_us import Simulation

# When you create a simulation
sim = Simulation(situation=household)

# Core manages:
# - Entity relationships
# - Variable dependencies
# - Parameter lookups
# - Period conversions

# When you calculate
result = sim.calculate("income_tax", 2026)

# Core:
# 1. Checks if already calculated
# 2. Identifies dependencies (income → AGI → taxable income → tax)
# 3. Calculates dependencies first
# 4. Applies formulas
# 5. Returns result
```

### Core vs Country Packages

**Core (policyengine-core):**
- Generic simulation engine
- No specific tax/benefit rules
- Variable and parameter infrastructure

**Country packages (policyengine-us, etc.):**
- Built on Core
- Contain specific tax/benefit rules
- Define variables and parameters for that country

**Relationship:**
```
policyengine-core (engine)
    ↓ powers
policyengine-us (US rules)
    ↓ used by
policyengine-api (REST API)
    ↓ serves
policyengine-app (web interface)
```

## For Contributors

### Repository

**Location:** PolicyEngine/policyengine-core
**Origin:** Fork of OpenFisca-Core

**Clone:**
```bash
git clone https://github.com/PolicyEngine/policyengine-core
```

### Current Architecture

**To see current structure:**
```bash
tree policyengine_core/

# Key directories:
# - variables/ - Variable class and infrastructure
# - parameters/ - Parameter class and infrastructure
# - entities/ - Entity definitions
# - simulations/ - Simulation class
# - periods/ - Period handling
# - reforms/ - Reform application
```

**To understand a specific component:**
```bash
# Variable system
cat policyengine_core/variables/variable.py

# Parameter system
cat policyengine_core/parameters/parameter.py

# Simulation engine
cat policyengine_core/simulations/simulation.py

# Entity system
cat policyengine_core/entities/entity.py
```

### Key Classes

**Variable:**
```python
# To see Variable class implementation
cat policyengine_core/variables/variable.py

# Variables in country packages inherit from this:
from policyengine_core.variables import Variable

class income_tax(Variable):
    value_type = float
    entity = Person
    label = "Income tax"
    definition_period = YEAR

    def formula(person, period, parameters):
        # Vectorized formula
        return calculate_tax(...)
```

**Simulation:**
```python
# To see Simulation class implementation
cat policyengine_core/simulations/simulation.py

# Manages calculation graph and caching
sim = Simulation(situation=situation)
sim.calculate("variable", period)
```

**Parameters:**
```python
# To see Parameter handling
cat policyengine_core/parameters/parameter_node.py

# Access in formulas:
parameters(period).gov.irs.credits.ctc.amount.base_amount
```

### Vectorization (Critical!)

Core requires vectorized operations - no if-elif-else with arrays:

**❌ Wrong (scalar logic):**
```python
if age < 18:
    eligible = True
else:
    eligible = False
```

**✅ Correct (vectorized):**
```python
eligible = age < 18  # NumPy boolean array
```

**Why:** Core processes many households simultaneously for performance.

**To see vectorization examples:**
```bash
# Search for where() usage (vectorized if-then-else)
grep -r "np.where" policyengine_core/

# Find select() usage (vectorized case statements)
grep -r "select" policyengine_core/
```

### Formula Dependencies

Core automatically resolves variable dependencies:

```python
class taxable_income(Variable):
    def formula(person, period, parameters):
        # Core automatically calculates these first:
        agi = person("adjusted_gross_income", period)
        deduction = person("standard_deduction", period)
        return agi - deduction

class income_tax(Variable):
    def formula(person, period, parameters):
        # Core knows to calculate taxable_income first
        taxable = person("taxable_income", period)
        return apply_brackets(taxable, ...)
```

**To see dependency resolution:**
```bash
# Find trace functionality
grep -r "trace" policyengine_core/simulations/

# Enable in your code:
simulation.trace = True
simulation.calculate("income_tax", 2026)
```

### Period Handling

**To see period implementation:**
```bash
cat policyengine_core/periods/period.py

# Period types:
# - YEAR: 2024
# - MONTH: 2024-01
# - ETERNITY: permanent values
```

**Usage in variables:**
```python
# Annual variable
definition_period = YEAR  # Called with 2024

# Monthly variable
definition_period = MONTH  # Called with "2024-01"

# Convert periods
yearly_value = person("monthly_income", period.this_year) * 12
```

### Testing Core Changes

**To run Core tests:**
```bash
cd policyengine-core
make test

# Specific test
pytest tests/core/test_variables.py -v
```

**To test in country package:**
```bash
# Changes to Core affect all country packages
cd policyengine-us
uv pip install -e ../policyengine-core  # Local development install
make test
```

### Key Differences from OpenFisca

PolicyEngine Core differs from OpenFisca-Core:

**To see PolicyEngine changes:**
```bash
# Compare to OpenFisca
# Core fork diverged to add:
# - Enhanced performance
# - Better error messages
# - PolicyEngine-specific features

# See commit history for PolicyEngine changes
git log --oneline
```

## Core Development Workflow

### Making Changes to Core

1. **Clone repo:**
   ```bash
   git clone https://github.com/PolicyEngine/policyengine-core
   ```

2. **Install for development:**
   ```bash
   make install
   ```

3. **Make changes** to variable.py, simulation.py, etc.

4. **Test locally:**
   ```bash
   make test
   ```

5. **Test in country package:**
   ```bash
   cd ../policyengine-us
   uv pip install -e ../policyengine-core
   make test
   ```

6. **Format and commit:**
   ```bash
   make format
   git commit -m "Description"
   ```

### Understanding Impact

Changes to Core affect:
- ✅ All country packages (US, UK, Canada, IL, NG)
- ✅ The API
- ✅ The web app
- ✅ All analysis tools

**Critical:** Always test in multiple country packages before merging.

## Common Core Patterns

### Pattern 1: Adding a New Variable Type

**Current variable types:**
```bash
# See supported types
grep "value_type" policyengine_core/variables/variable.py
```

**Types:** int, float, bo