fiftyone-dataset-curation

What this skill does

# FiftyOne Dataset Curation

End-to-end curation pipeline for any FiftyOne dataset: images, video, point clouds, grouped multimodal. Run all phases sequentially or jump directly to any single phase.

## Key Directives

**ALWAYS follow these rules — no exceptions:**

### 1. Check for a loaded dataset first
```
list_datasets()
```
If none exists, offer to delegate to `fiftyone-dataset-import` skill.

### 2. Set context before any operation
```
set_context(dataset_name="<name>")
```

### 3. Launch App before any brain operator
```
launch_app()
```
Wait 5–10 seconds for initialization before executing delegated operators.

### 4. Discover schema before referencing any field
```
dataset_summary()
get_field_schema(flat=True)
```
Never hardcode field names. Adapt all field references from schema discovery results.

### 5. Discover operators dynamically
```
list_operators(builtin_only=False)
get_operator_schema(operator_uri="<uri>")
```
Always call `list_operators()` before any `execute_operator()`. Confirm the operator exists.

### 6. Confirm before mutating
Before tagging, adding fields, or deleting — present findings to the user and ask for confirmation.

### 7. Keep App open for interactive review
Do NOT call `close_app()` automatically. Leave the App running so the user can explore results.

### 8. Check delegated service before brain operations
Before running any brain operator with `delegate=True`, verify the delegated service is running:
```bash
fiftyone delegated list
```
If no services are running:
```bash
fiftyone delegated launch &
```
Wait ~5 seconds for initialization. Without this service, delegated operators will queue but never execute.

### 9. Always discover plugin names dynamically
Never assume a plugin's exact name or operator URI from documentation. Always verify:
```
list_plugins()
```
Use the exact plugin name from the output. Then:
```
list_operators(builtin_only=False)
```
Use the exact operator URI from the output. Documentation names may differ from installed names.

### 10. Generate interactive insights after each phase
After computing any metric (uniqueness, quality, similarity, etc.):
1. Calculate counts of flagged/notable samples
2. Create named saved views for the findings
3. Load the view in the App automatically
4. Present a narrative insight to the user
5. Offer to continue to the next phase

See **Insight Generation Pattern** at the bottom of this file.

---

## Phase -1 — Curation Plan

**Before doing anything else**, present a complete curation plan to the user. This sets expectations and allows them to choose which phases to run.

```
list_datasets()
dataset_summary()
get_field_schema(flat=True)
```

Present the plan:

```
## Curation Plan for <dataset_name>

Dataset: <name> | <N> samples | <media_type>
Label fields: <list from schema>
Existing brain runs: <list or "none">

### Phases I will run:
  Phase 1 — Dataset Inspection        (schema, tags, class distribution)
  Phase 2 — Data Quality Audit        (metadata, resolution, file size, image quality via jacobmarks/image_issues plugin)
  Phase 3 — Near-Duplicate Detection  (via fiftyone-find-duplicates skill)
  Phase 4 — Class Distribution        (imbalance, coverage gaps)
  Phase 5 — Embedding Exploration     (CLIP UMAP, gap detection)
  Phase 6 — Annotation Audit          (mistakenness — requires predictions field)
  Phase 7 — Curated Subset & Splits   (clean view, train/val/test)
  Phase 8 — Data Q&A                  (on-demand questions)

### Phases I will SKIP (and why):
  Phase 6 — No predictions field found. Run fiftyone-dataset-inference first.

### What you'll get:
  - Named saved views for each key finding (loadable in App)
  - Interactive insights at each step
  - A clean_<dataset_name> view with quality filters applied

Ready to start? (yes / skip to phase X / select specific phases)
```

Wait for user confirmation before proceeding. If they want to skip phases or run only specific ones, adapt accordingly.

---

## Phase 0 — Dataset Loading (Optional)

Run this phase only if no dataset is loaded yet.

```
list_datasets()
```

- If a dataset exists → `set_context(dataset_name="<name>")` and proceed to Phase 1.
- If no dataset exists → delegate to `fiftyone-dataset-import` skill.

After loading:
```
dataset_summary()
```

Confirm with user:
- Media type (image, video, point-cloud, group)
- Sample count
- Whether `persistent=True` is set

---

## Phase 1 — Dataset Inspection

**MCP tools:** `dataset_summary`, `get_field_schema`, `count_values`, `distinct`

### Steps

```
dataset_summary()
get_field_schema(flat=True)
count_values("tags")
```

From the schema, identify label fields (Detections, Classifications, Keypoints, Segmentation, Polylines, etc.) and run:
```
distinct("<label_field>.label")
```
Adapt the field path from the schema — never assume a field name.

### Report to user

Present a summary with:
- Media type and sample count
- All field names and their types
- Existing tags and counts
- Unique class names per label field
- Any existing brain runs (embeddings, similarity indexes, mistakenness scores)
- Missing or empty fields (flag these)
- **Label type awareness**: Note whether label fields are Classifications vs. Detections — this affects which export formats are valid later

### Export format note

After identifying label types, inform the user:
```
Label type: Classification → supports Image Classification Directory Tree export
Label type: Detection → supports COCO, YOLO, VOC export
For COCO export with Classification labels → must run inference first (fiftyone-dataset-inference) to generate Detection labels
```

### Grouped dataset detection

If `dataset_summary()` reports `media_type == "group"`:
1. List available group slices from the schema
2. Recommend operating on individual sensor slices for all curation phases:
   ```python
   # Python SDK equivalent (use for user guidance)
   rgb_view = dataset.select_group_slices(["rgb_center"])
   dataset.save_view("rgb_view", rgb_view)
   ```
3. Apply all subsequent brain operations to the slice view, not the full grouped dataset
4. Use sensor-prefixed brain keys (e.g., `rgb_clip_umap`, `ir_clip_umap`)

### Existing brain runs

If brain runs are reported in `dataset_summary()`:
- List them and their types (similarity, visualization, mistakenness, uniqueness, etc.)
- Offer to reuse existing runs instead of recomputing
- Ask user which phases to run and whether to skip phases with existing results

---

## Phase 2 — Data Quality Audit

See `QUALITY-CHECKS.md` for the full deep-dive workflow.

**MCP tools:** `execute_operator`, `bounds`, `histogram_values`, `mean`, `std`, `count_values`, `set_view`, `list_plugins`, `download_plugin`, `enable_plugin`

### Minimum steps

**Step 1: Compute metadata**
```
list_operators(builtin_only=False)
get_operator_schema(operator_uri="@voxel51/utils/compute_metadata")
execute_operator(operator_uri="@voxel51/utils/compute_metadata", params={})
```

**Step 2: Check for corruption (null metadata)**
```
set_view(filters={"metadata": null})
```
If samples appear → report count; offer to tag them as `"corrupted"` after confirmation.

**Step 3: Resolution analysis**
```
bounds("metadata.width")
bounds("metadata.height")
histogram_values("metadata.width", bins=20)
histogram_values("metadata.height", bins=20)
```

**Step 4: File size analysis**
```
bounds("metadata.size_bytes")
mean("metadata.size_bytes")
std("metadata.size_bytes")
```

**Step 5: Aspect ratio**
Report wide/tall outliers using ViewField pattern (see QUALITY-CHECKS.md).

**Step 6: Image quality via jacobmarks plugin (REQUIRED for image datasets)**

This is the primary method for detecting blur, brightness issues, contrast, saturation, and aspect ratio anomalies. Do NOT use custom cv2 code. Follow the discovery workflow:

```
list_plugins()
```

Find the entry matching "image_issues" or "image-quality". Note the exact plugin name from the output — **do not assume it matches the documentation name**.

If not i