datajoint-biomechanics-schema

What this skill does

# DataJoint Biomechanics Schema Reference

## CRITICAL: NEVER Modify Database Entries

**DO NOT update, delete, or alter any DataJoint database entries.** This includes:
- `update1()`, `delete()`, `drop()` on any table
- Modifying settings lookup tables (KinematicReconstructionSettingsLookup, ProbabilisticReconstructionSettingsLookup, KineticReconstructionSettingsLookup, KeypointSet, etc.)
- Altering any computed table entries

Database entries are shared state used by the entire lab. Changing a settings entry changes it for everyone and invalidates prior results computed with those settings.

## Overview

The biomechanics pipeline uses DataJoint across four repositories with **two complementary data collection approaches**:

1. **Multi-Camera (MMC)** - Lab-based with calibrated camera rigs → `MultiCameraTracking` + `BodyModels/kinematic_dj.py`
2. **Portable/Monocular (PBL)** - Smartphone videos with optional IMU → `PortableBiomechanicsSessions` + `BodyModels/monocular_dj.py`

Both produce the same output format: **qpos, qvel, joints, sites** from MuJoCo body models.

---

## DataJoint Query Basics

### Operators
| Operator | Meaning | Example |
|----------|---------|---------|
| `&` | Restrict (filter) | `(Table & {'field': value})` |
| `*` | Join tables | `(Table1 * Table2)` |
| `-` | Set difference | `(Table1 - Table2)` finds rows NOT in Table2 |
| `.proj()` | Select attributes | `Table.proj('field1', 'field2')` |

### Fetch Patterns
```python
# fetch1() - Exactly one row, returns tuple or dict
timestamps, qpos = (KinematicReconstruction.Trial & key).fetch1('timestamps', 'qpos')

# fetch() - Multiple rows, returns arrays
all_keys = (Session & filter).fetch('KEY')
keypoints = (TopDownPerson & key).fetch('keypoints')

# With ordering (critical for multi-camera consistency)
kp, names = (Table & key).fetch('keypoints', 'camera_name', order_by='camera_name')
```

---

## Filtering by Project and Participant

### video_project Field

`video_project` is a categorical field (varchar(50)) that identifies the study/cohort. It lives in `MultiCameraRecording` and `Video` tables.

**Common projects:**
| Project | Description |
|---------|-------------|
| `GAIT_CONTROLS` | Healthy control gait data |
| `PROSTHETIC_GAIT` | Prosthetic gait subjects |
| `CM_GAIT` | Cervical myelopathy gait |
| `CLINIC_GAIT` | Clinical gait assessment |
| `PEDIATRIC_GAIT` | Pediatric subjects |
| `ASB2024` | ASB conference dataset |

### Query Patterns by Project

```python
# List all unique projects
projects = np.unique((MultiCameraRecording).fetch("video_project"))

# Find all participants in a project
participants = (Session & (Recording & (MultiCameraRecording & 'video_project="GAIT_CONTROLS"'))).fetch("participant_id")

# Filter trials by project
trials = KinematicReconstruction.Trial & (Recording & (MultiCameraRecording & f"video_project='{project}'"))

# Multiple projects with IN clause
filt = 'video_project in ("CLINIC_GAIT", "GAIT_CONTROLS", "PROSTHETIC_GAIT")'
sessions = Session & (Recording * MultiCameraRecording & filt)
```

### Participant Naming Conventions

| Project Type | Format | Examples |
|--------------|--------|----------|
| ASB studies | `ASB_###` | ASB_001, ASB_022 |
| Numeric IDs | integers | 72, 504, 127 |
| Special codes | mapped | TF01, TF02, TF47 |

### Session → Video Path (via video_project)
```
Session (participant_id, session_date)
    → Recording → MultiCameraRecording (video_project)
        → SingleCameraVideo → Video (video_project, filename)
```

---

## Key Output Fields Reference

| Field | Table | Shape | Description |
|-------|-------|-------|-------------|
| **qpos** | KinematicReconstruction.Trial | (T, 41) | Joint angles in generalized coordinates (radians) |
| **qvel** | KinematicReconstruction.Trial | (T, 41) | Joint velocities (rad/s) |
| **joints** | KinematicReconstruction.Trial | (T, N_joints, 3) | 3D joint positions (mm) |
| **sites** | KinematicReconstruction.Trial | (T, N_sites, 3) | 3D anatomical markers (mm) |
| **keypoints** | TopDownPerson | (T, N_kpts, 3) | 2D keypoints [x, y, conf] |
| **keypoints3d** | PersonKeypointReconstruction | (T, N_kpts, 4) | 3D keypoints [x, y, z, conf] (mm) |
| **keypoints_3d** | LiftingPerson | (T, N_kpts, 4) | Lifted 3D from monocular [x, y, z, conf] |
| **timestamps** | VideoInfo | list[datetime] | Absolute frame times |
| **delta_time** | VideoInfo | (T,) | Seconds from first frame |
| **fps** | VideoInfo | float | Frames per second |

### Common Access Patterns
```python
# Get kinematic reconstruction (ALWAYS specify method!)
qpos, sites = (KinematicReconstruction.Trial & key &
               {'kinematic_reconstruction_settings_num': 137}).fetch1('qpos', 'sites')

# Get 3D triangulated keypoints
kp3d = (PersonKeypointReconstruction & key).fetch1('keypoints3d')
```

**For 2D keypoints synchronized with KR qpos**, see the Temporal Synchronization section below. Do NOT use `(TopDownPerson * VideoInfo & key).fetch1('timestamps', 'keypoints')` — this produces misaligned frames.

---

## Temporal Synchronization (CRITICAL)

KR qpos and 2D keypoints have **independently managed timestamps**. Naive matching silently corrupts frame alignment.

**Correct approach:** Use `fetch_keypoints(only_detected=True)` from BodyModels:

```python
from body_models.datajoint.dataset import fetch_keypoints as bm_fetch_keypoints

# Returns keypoints in 1:1 frame correspondence with KR qpos
timestamps, keypoints_raw = bm_fetch_keypoints(trial_key, only_detected=True)
# keypoints_raw: (C, T, 87, 3) — (x, y, confidence)

# ALWAYS verify correspondence
assert qpos.shape[0] == keypoints_raw.shape[1]
```

**Why direct TopDownPerson/VideoInfo fetch is broken:** KR timestamps are zeroed relative to first detection; VideoInfo timestamps are zeroed relative to video start. When first detection occurs at frame N>0, matching on timestamp magnitude pairs the wrong frames (N-frame offset).

**Camera parameter reordering:** `KinematicReconstruction.updated_calibration` uses Calibration table order; keypoints use alphabetical SingleCameraVideo order. These differ — reorder explicitly before projection.

See `rae:fetching-synchronized-data` for the complete pattern with camera reordering and contiguous segment selection.

---

## Two Data Collection Approaches

### 1. Multi-Camera (MMC) → KinematicReconstruction

**Key space:** `(participant_id, session_date)` bridged to `(recording_timestamps, camera_config_hash)`

```
Session → Recording → MultiCameraRecording → SingleCameraVideo → Video
                   ↘ SessionCalibration.Grouping → Calibration
                                     ↓
Video → TopDownPerson → PersonKeypointReconstruction (3D triangulation)
                                     ↓
                   KinematicReconstruction.Trial (qpos, qvel, joints, sites)
```

**Bridge Tables:**
- `Recording` - Links Session → MultiCameraRecording
- `SingleCameraVideo` - Links MultiCameraRecording → Video
- `SessionCalibration.Grouping` - Links Session → Calibration (required for KinematicReconstruction)

**Use when:** Lab-based data, multiple synchronized cameras, highest accuracy needed.

### 2. Portable/Monocular (PBL) → MonocularReconstruction

**Key space:** `(subject_id, session_start_time)` via Firebase

```
Subject → Session → Session.FirebaseSessionInfo
                              ↓
                    FirebaseSession (Computed)
                              ↓
            ┌─────────────────┼─────────────────┐
       AppVideo           Attitude/Gyro      PhoneAttitude
     (→ Video)            (IMU sensors)      (phone quaternion)
            ↓
    TopDownPerson → LiftingPerson (3D lifting)
                              ↓
            MonocularReconstruction.Trial (qpos, qvel, joints, sites, rnc)
```

**Additional output:** `rnc` (T, 3) - camera rotation vector from phone attitude

**Use when:** Smartphone videos, remote/home monitoring, wearable IMU/EMG data.

### Comparison

| Aspect | Multi-Camera | Monocular |
|----