bug-clustering

What this skill does

# Bug Clustering Process

Step-by-step procedure for transforming raw XPost objects into structured, clustered bug candidates with PII redaction and reliability scoring.

## Overview

Loaded by the `bug-clusterer` agent inside the `x-bug-triage` plugin. Transforms raw `XPost` records ingested from X/Twitter into structured `BugCandidate` rows, deduplicates near-identical candidates, classifies each candidate into one of 12 bug families, redacts six categories of PII, scores reporter reliability across four dimensions, and groups results into bug clusters by deterministic signature. The output feeds the downstream repo-scanning, owner-routing, and triage-display stages.

## Prerequisites

- Database initialized via `lib/db.ts` schema migrations
- `config/cluster-matching-thresholds.json` present (controls dedup + cluster overlap thresholds)
- `config/approved-accounts.json` present (drives reporter category tagging)
- Raw `XPost[]` array passed in by the orchestrator (not fetched here)

## Instructions

### Step 1: Parse

For each XPost, produce a BugCandidate with all 33 fields using `lib/parser.ts`:
- Extract product_surface, feature_area, symptoms, error_strings, repro_hints
- Extract urls, media_keys, language, conversation references
- Determine source_type (mention, reply, quote_post, search_hit)

### Step 1.5: Deduplicate

Before classification, run content-similarity deduplication using `lib/dedupe.ts`:
- Call `deduplicateCandidates()` with parsed candidates and the `candidate_dedup.hybrid_similarity_threshold` from `config/cluster-matching-thresholds.json` (default 0.70)
- Uses char-trigram + token-Jaccard hybrid similarity
- Does NOT remove posts — tags them as duplicate groups with a canonical post (highest engagement)
- Only canonical posts and non-duplicates (`forward_ids`) proceed to classification
- Log dedup stats in the form: "N posts (M unique, K duplicate groups)" with N/M/K replaced by integer counts

### Step 2: Classify

Run `lib/classifier.ts` on each candidate:
- Assign one of 12 classifications with confidence score (0.0-1.0) and rationale
- Sarcastic bug reports get classified separately — still treated as signal

### Step 3: Redact PII

Run `lib/redactor.ts` on each candidate:
- Detect 6 PII types: email, API key, phone, account ID, media flag, URL token
- Replace with [REDACTED:type] tags
- Set pii_flags array and raw_text_storage_policy

### Step 4: Score Reliability

Run `lib/reporter-scorer.ts` on each candidate:
- 4 dimensions: report quality, independence, account authenticity, historical accuracy
- Composite reporter_reliability_score (0.0-1.0)

### Step 5: Tag Reporter Category

Match author against approved_accounts config:
- Categories: public, internal, partner, tester

### Step 6: Cluster

Using `lib/clusterer.ts` and `lib/signatures.ts`:
- Generate deterministic bug signature from error_strings + symptoms + feature_area
- Match against active_clusters at >=70% signature overlap
- Family-first guard: different ClusterFamilies NEVER cluster together
- New match: create cluster (initial severity "low")
- Existing match: update report_count, last_seen, sub_status
- Resolved match: reopen with sub_status "regression_reopened"
- Suppressed match: skip, log to audit

### Step 7: Persist

- Insert candidates to DB via `lib/db.ts`
- Insert/update clusters and cluster_posts junction
- Write audit events for each classification, redaction, and cluster action

## Output

- `bug_candidates` rows with classification, PII flags, and reporter reliability scores
- `bug_clusters` rows (new or updated) with severity, sub_status, and report_count
- `cluster_posts` junction rows linking candidates to clusters
- Audit-event rows recording every classification, redaction, and cluster action

## Error Handling

- Parser failure on a single XPost: log + skip, continue with remaining posts (degraded mode)
- Classification confidence below threshold: still recorded, flagged for human review
- Cluster signature collision across families: hard-blocked by the family-first guard — never cross-clusters
- Persist failure mid-batch: rollback uncommitted writes for that batch only; preserve already-committed work

## Examples

The bug-clusterer agent invokes this skill after the X/Twitter ingest phase completes. A typical batch processes 50–500 candidates per run, producing 5–30 clusters depending on overlap. Sample audit log line: "127 posts (89 unique, 14 duplicate groups, 6 new clusters, 8 cluster updates)".

## Resources

Load evidence tier definitions for proper cluster evidence assessment:

```
!cat skills/x-bug-triage/references/evidence-policy.md
```

Load data model reference for BugCandidate fields and cluster schemas:

```
!cat skills/x-bug-triage/references/schemas.md
```