Python

What this skill does

<!-- PYTHON:START -->
# Python Project Rules

## Agent Automation Commands

**CRITICAL**: Execute these commands after EVERY implementation (see AGENT_AUTOMATION module for full workflow).

```bash
# Complete quality check sequence:
ruff format --check .      # Format check
ruff check .               # Linting
mypy .                     # Type checking
pytest                     # All tests (100% pass required)
pytest --cov               # Coverage check (95%+ required)

# Security audit:
pip-audit                  # Vulnerability scan
pip list --outdated        # Check outdated deps
```

## Python Version

**CRITICAL**: Use Python 3.11+ for modern features and performance.

- **Minimum Version**: Python 3.11+
- **Recommended**: Python 3.12+
- **Type Hints**: Required for all public APIs

### Formatting

- Use `ruff format` (fast, modern) or `black` (traditional)
- Line length: 100 characters (configurable)
- Consistent formatting across entire project
- Format before committing

Configuration in `pyproject.toml`:
```toml
[tool.ruff]
line-length = 100
target-version = "py311"

[tool.ruff.format]
quote-style = "double"
indent-style = "space"
```

### Linting

- Use `ruff check` (fast, comprehensive) or `ruff` + `flake8`
- Fix all linting errors before committing
- Document any disabled rules with justification

Configuration in `pyproject.toml`:
```toml
[tool.ruff.lint]
select = ["E", "F", "I", "N", "W", "UP", "B", "A", "C4", "SIM"]
ignore = ["E501"]  # Line too long (handled by formatter)

[tool.ruff.lint.per-file-ignores]
"tests/*" = ["S101"]  # Allow assert in tests
```

### Type Checking

- Use `mypy` for static type checking
- All public APIs must have type hints
- Use `typing` module for complex types
- Gradual typing allowed for legacy code

Configuration in `pyproject.toml`:
```toml
[tool.mypy]
python_version = "3.11"
strict = true
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
```

Example:
```python
from typing import Optional, List, Dict, Any

def process_data(
    input_data: str,
    options: Optional[Dict[str, Any]] = None
) -> List[str]:
    """Process input data and return results."""
    # Implementation
    return []
```

### Testing

- **Framework**: pytest
- **Location**: `/tests` directory
- **Coverage**: Must meet project threshold (default 95%)
- **Fixtures**: Use pytest fixtures for setup/teardown
- **Parametrize**: Use `@pytest.mark.parametrize` for multiple test cases

Example test structure:
```python
import pytest
from mymodule import process_data

@pytest.fixture
def sample_data():
    """Provide sample data for tests."""
    return "test input"

def test_process_data_valid_input(sample_data):
    """Test process_data with valid input."""
    result = process_data(sample_data)
    assert result == ["expected"]

@pytest.mark.parametrize("input_val,expected", [
    ("hello", ["HELLO"]),
    ("world", ["WORLD"]),
])
def test_process_data_parametrized(input_val, expected):
    """Test multiple input cases."""
    result = process_data(input_val)
    assert result == expected
```

### Test Categories: S2S and Slow Tests

**CRITICAL**: Tests must be categorized based on execution time and dependencies.

#### Test Time Limits

- **Fast Tests**: Must complete in ≤ 10-20 seconds
- **Slow Tests**: Any test taking > 10-20 seconds must be marked as slow
- **S2S Tests**: Tests requiring active server/database must be isolated and run on-demand

#### S2S (Server-to-Server) Tests

**Tests that require active servers, databases, or external services must be isolated using pytest markers.**

**Implementation**:

1. **Mark S2S tests with pytest markers**:
```python
import pytest
import os

# Regular fast test (always runs)
def test_local_computation():
    """Fast test, no external dependencies."""
    result = compute_locally("input")
    assert result == "expected"

# S2S test (only runs with -m s2s)
@pytest.mark.s2s
def test_database_connection():
    """Requires active database server."""
    db = connect_to_database()
    # ... test implementation

@pytest.mark.s2s
def test_api_integration():
    """Requires active API server."""
    client = create_api_client()
    # ... test implementation
```

2. **Configure `pytest.ini` or `pyproject.toml`**:
```ini
# pytest.ini
[pytest]
markers =
    s2s: Server-to-server tests requiring active services
    slow: Slow tests taking > 20 seconds
```

Or in `pyproject.toml`:
```toml
[tool.pytest.ini_options]
markers = [
    "s2s: Server-to-server tests requiring active services",
    "slow: Slow tests taking > 20 seconds",
]
```

3. **Run tests**:
```bash
# Regular tests (excludes S2S)
pytest

# Include S2S tests (requires active servers)
pytest -m s2s

# Run all tests including S2S
pytest -m "not slow"  # Fast + S2S, excludes slow
```

#### Slow Tests

**Tests that take > 10-20 seconds must be marked and run separately.**

**Implementation**:

1. **Mark slow tests with pytest markers**:
```python
import pytest

# Fast test (always runs)
def test_quick_operation():
    """Completes in < 1 second."""
    result = quick_compute("input")
    assert result == "expected"

# Slow test (only runs with -m slow)
@pytest.mark.slow
def test_heavy_computation():
    """Takes 30+ seconds."""
    # Heavy processing, large dataset, etc.
    result = process_large_dataset()
    assert result is not None

@pytest.mark.slow
def test_large_file_processing():
    """Processes large files, takes > 20 seconds."""
    result = process_file("large_file.dat")
    assert result.success
```

2. **Run tests**:
```bash
# Regular tests (excludes slow and S2S)
pytest -m "not slow and not s2s"

# Include slow tests
pytest -m slow

# Run all tests
pytest -m ""  # Empty marker means all tests
```

3. **Add pytest configuration for timeouts**:
```python
# conftest.py
import pytest

@pytest.fixture(autouse=True)
def configure_timeouts(request):
    """Configure timeouts based on test markers."""
    if 'slow' in request.keywords:
        request.node.add_marker(pytest.mark.timeout(300))  # 5 minutes
    elif 's2s' in request.keywords:
        request.node.add_marker(pytest.mark.timeout(60))  # 1 minute
    else:
        request.node.add_marker(pytest.mark.timeout(20))  # 20 seconds
```

4. **Add scripts in `pyproject.toml` or `setup.py`**:
```toml
[tool.poetry.scripts]
test = "pytest -m 'not slow and not s2s'"
test-s2s = "pytest -m s2s"
test-slow = "pytest -m slow"
test-all = "pytest"
```

#### Best Practices

- ✅ **Always run fast tests** in CI/CD by default
- ✅ **Isolate S2S tests** - never run them in standard test suite
- ✅ **Mark slow tests** - prevent CI/CD timeouts
- ✅ **Document requirements** - specify which servers/services are needed for S2S tests
- ✅ **Use timeouts** - Set appropriate timeouts: `@pytest.mark.timeout(60)`
- ✅ **Use pytest markers** - `@pytest.mark.s2s` and `@pytest.mark.slow`
- ✅ **Skip conditionally** - `@pytest.mark.skipif(not os.getenv('RUN_S2S_TESTS'), reason='S2S tests disabled')`
- ❌ **Never mix** fast and slow/S2S tests in same test run
- ❌ **Never require** external services for standard test suite
- ❌ **Never exceed** 10-20 seconds for regular tests

## Dependency Management

**CRITICAL**: Use modern dependency management tools.

### Recommended: Poetry

```toml
[tool.poetry]
name = "myproject"
version = "0.1.0"
description = ""
authors = ["Your Name <[email protected]>"]

[tool.poetry.dependencies]
python = "^3.11"
requests = "^2.31.0"

[tool.poetry.group.dev.dependencies]
pytest = "^7.4.0"
mypy = "^1.5.0"
ruff = "^0.1.0"
```

Commands:
```bash
poetry install              # Install dependencies
poetry add requests         # Add dependency
poetry add --group dev pytest  # Add dev dependency
poetry update               # Update dependencies
```

### Alternative: pip-tools

```
# requirements.in
requests>=2.31.0
pydantic>=2.0.0

# requirements-dev.in
-r requirements.in
pytest>=7.4.0
mypy>=1.5.0
```

Commands:
```bash
pip-compile requirements.in
pip-compile requirements-