semgrep

What this skill does

# Semgrep Static Analysis

## When to Use Semgrep

**Ideal scenarios:**

- Quick security scans (minutes, not hours)
- Pattern-based vulnerability detection
- Enforcing coding standards and best practices
- Finding known vulnerability patterns (OWASP Top 10, CWE Top 25)
- Intra-file taint analysis and data flow tracking
- Custom rule development for specific code patterns
- First-pass security analysis before deeper tools
- CI/CD security gates for fast feedback
- Multi-language security scanning

**Complements other tools:**

- Use before manual code review to catch common patterns
- Combine with SARIF Issue Reporter for detailed findings
- Use alongside CodeQL for comprehensive coverage
- Pair with dependency scanners (OSV-Scanner, Depscan)

**Consider CodeQL instead when:**

- Need interprocedural taint tracking across files
- Complex data flow analysis across modules required
- Analyzing custom proprietary frameworks with deep integration

## When NOT to Use

Do NOT use this skill for:

- Complex interprocedural data flow analysis (use CodeQL instead)
- Binary analysis or compiled code without source
- Custom deep semantic analysis requiring AST/CFG traversal
- Tracking taint across many function boundaries and files
- Secrets detection (use Gitleaks)
- Dependency vulnerability scanning (use OSV-Scanner or Depscan)
- IaC security analysis (use KICS)
- API endpoint discovery (use Noir)

## Installation

```bash
# pip
python3 -m pip install semgrep

# pipx (recommended)
pipx install semgrep

# Homebrew
brew install semgrep

# Docker
docker pull returntocorp/semgrep:latest
docker run --rm -v "${PWD}:/src" returntocorp/semgrep semgrep --config auto /src

# Update
pip install --upgrade semgrep

# Verify
semgrep --version
```

## Core Workflow

### 1. Quick Scan

```bash
semgrep --config auto .                    # Auto-detect rules
semgrep --config auto --metrics=off .      # Disable telemetry for proprietary code
```

### 2. Use Rulesets

```bash
semgrep --config p/<RULESET> .             # Single ruleset
semgrep --config p/security-audit --config p/trailofbits .  # Multiple
```

| Ruleset | Description |
|---------|-------------|
| `p/default` | General security and code quality |
| `p/security-audit` | Comprehensive security rules |
| `p/owasp-top-ten` | OWASP Top 10 vulnerabilities |
| `p/cwe-top-25` | CWE Top 25 vulnerabilities |
| `p/r2c-security-audit` | r2c security audit rules |
| `p/trailofbits` | Trail of Bits security rules |
| `p/python` | Python-specific |
| `p/javascript` | JavaScript-specific |
| `p/golang` | Go-specific |

### 3. Output Formats

```bash
# SARIF output (for CI/CD)
semgrep --config p/security-audit --sarif -o results.sarif .

# JSON output
semgrep --config p/security-audit --json -o results.json .

# Text output with dataflow traces
semgrep --config p/security-audit --dataflow-traces .

# JUnit XML
semgrep --config p/security-audit --junit-xml -o results.xml .

# GitLab SAST format
semgrep --config p/security-audit --gitlab-sast -o gl-sast-report.json .

# Vim quickfix
semgrep --config p/security-audit --vim .
```

### 4. Scan Specific Paths

```bash
# Single file
semgrep --config p/python app.py

# Specific directory
semgrep --config p/javascript src/

# Include tests (excluded by default)
semgrep --config auto --include='**/test/**' .

# Exclude paths
semgrep --config auto --exclude='vendor' --exclude='node_modules' .

# Multiple languages
semgrep --config p/python --config p/javascript .
```

### 5. Advanced Features

```bash
# Enable Pro Engine features (requires license)
semgrep --config p/security-audit --pro .

# Pro Engine interfile analysis
semgrep --config p/security-audit --pro --pro-intrafile .

# Disable telemetry
semgrep --config auto --metrics=off .

# Verbose output
semgrep --config p/security-audit --verbose .

# Quiet mode (only show findings)
semgrep --config p/security-audit --quiet .
```

## Writing Custom Rules

### Basic Structure

```yaml
rules:
  - id: hardcoded-password
    languages: [python]
    message: "Hardcoded password detected: $PASSWORD"
    severity: ERROR
    pattern: password = "$PASSWORD"
```

### Pattern Syntax

| Syntax | Description | Example |
|--------|-------------|---------|
| `...` | Match anything | `func(...)` |
| `$VAR` | Capture metavariable | `$FUNC($INPUT)` |
| `<... ...>` | Deep expression match | `<... user_input ...>` |

### Pattern Operators

| Operator | Description |
|----------|-------------|
| `pattern` | Match exact pattern |
| `patterns` | All must match (AND) |
| `pattern-either` | Any matches (OR) |
| `pattern-not` | Exclude matches |
| `pattern-inside` | Match only inside context |
| `pattern-not-inside` | Match only outside context |
| `pattern-regex` | Regex matching |
| `metavariable-regex` | Regex on captured value |
| `metavariable-comparison` | Compare values |

### Combining Patterns

```yaml
rules:
  - id: sql-injection
    languages: [python]
    message: "Potential SQL injection"
    severity: ERROR
    patterns:
      - pattern-either:
          - pattern: cursor.execute($QUERY)
          - pattern: db.execute($QUERY)
      - pattern-not:
          - pattern: cursor.execute("...", (...))
      - metavariable-regex:
          metavariable: $QUERY
          regex: .*\+.*|.*\.format\(.*|.*%.*
```

### Taint Mode (Data Flow)

Simple pattern matching finds obvious cases:

```python
# Pattern `os.system($CMD)` catches this:
os.system(user_input)  # Found
```

But misses indirect flows:

```python
# Same pattern misses this:
cmd = user_input
processed = cmd.strip()
os.system(processed)  # Missed - no direct match
```

Taint mode tracks data through assignments and transformations:

- **Source**: Where untrusted data enters (`user_input`)
- **Propagators**: How it flows (`cmd = ...`, `processed = ...`)
- **Sanitizers**: What makes it safe (`shlex.quote()`)
- **Sink**: Where it becomes dangerous (`os.system()`)

```yaml
rules:
  - id: command-injection
    languages: [python]
    message: "User input flows to command execution"
    severity: ERROR
    mode: taint
    pattern-sources:
      - pattern: request.args.get(...)
      - pattern: request.form[...]
      - pattern: request.json
    pattern-sinks:
      - pattern: os.system($SINK)
      - pattern: subprocess.call($SINK, shell=True)
      - pattern: subprocess.run($SINK, shell=True, ...)
    pattern-sanitizers:
      - pattern: shlex.quote(...)
      - pattern: int(...)
```

### Full Rule with Metadata

```yaml
rules:
  - id: flask-sql-injection
    languages: [python]
    message: "SQL injection: user input flows to query without parameterization"
    severity: ERROR
    metadata:
      cwe: "CWE-89: SQL Injection"
      owasp: "A03:2021 - Injection"
      confidence: HIGH
    mode: taint
    pattern-sources:
      - pattern: request.args.get(...)
      - pattern: request.form[...]
      - pattern: request.json
    pattern-sinks:
      - pattern: cursor.execute($QUERY)
      - pattern: db.execute($QUERY)
    pattern-sanitizers:
      - pattern: int(...)
    fix: cursor.execute($QUERY, (params,))
```

## Testing Rules

### Test File Format

```python
# test_rule.py
def test_vulnerable():
    user_input = request.args.get("id")
    # ruleid: flask-sql-injection
    cursor.execute("SELECT * FROM users WHERE id = " + user_input)

def test_safe():
    user_input = request.args.get("id")
    # ok: flask-sql-injection
    cursor.execute("SELECT * FROM users WHERE id = ?", (user_input,))
```

```bash
semgrep --test rules/
```

## CI/CD Integration (GitHub Actions)

```yaml
name: Semgrep

on:
  push:
    branches: [main]
  pull_request:
  schedule:
    - cron: '0 0 1 * *'  # Monthly

jobs:
  semgrep:
    runs-on: ubuntu-latest
    container:
      image: returntocorp/semgrep

    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0  # Required for diff-aware scanning

      - name: Run Semgrep
        run: |
          if [ "${{ github.event_name }}" = "pull_request" ]; then