gait-lab-dataset

What this skill does

# GaitLab Dataset Reference

## Overview

The `gait_lab` schema stores traditional clinical gait lab data from Vicon c3d files. This is a **separate system** from the markerless multicamera (MMC) pipeline. The same participant may have data in both systems, recorded during the same session (MAC_LAB participants).

**Source:** `GaitLabDataset/gait_lab_dataset/`

**Schema name:** `gait_lab`

---

## Table Hierarchy

```
Subject (subject_hash)
    -> Session (subject_hash, session_name)
        -> Trial (subject_hash, session_name, trial_name)
        |   -> Mocap (point_names, points in mm, point_rate, timestamps)
        |   |   -> GaitEvents (event, side, time)
        |   |   -> GaitParameters (parameter, side, value)
        |   |   -> MocapAnalysis (property, value)
        |   |   -> MocapProjection (toffset, camera params, error, proj, frames)
        |   -> GaitTrialVideo -> pose_pipeline.Video
        |   -> Analog (analog_names, channels, timestamps, units)
        -> PTEval (eval_html) -> PTDemographics (age, diagnosis)
        -> Assessment (assessment)
        -> Height (computed, meters)
        -> Weight (computed, kg)
        -> SessionConditions -> SessionCondition + ConditionTrial
        -> MPParam (property, value, all_values)
    -> SubjectTrainTest (train boolean)
```

---

## Table Definitions

### Core Tables

**Subject** - De-identified subject identifier
```python
subject_hash: varchar(100)  # Primary key
```

**Session** - Clinical session grouping
```python
-> Subject
session_name: varchar(100)
```

**Trial** - Individual gait trial
```python
-> Session
trial_name: varchar(100)
---
timestamp: timestamp          # When the trial was recorded
description: varchar(1000)    # Free-text (e.g., "Barefoot with cane", "Right AFO")
vid_present: boolean          # Whether video was recorded
mocap_present: boolean        # Whether c3d mocap data exists
```

### Motion Capture Data

**Mocap** - 3D marker data from c3d files
```python
-> Trial
---
point_names: longblob    # List of marker names (e.g., ["SACR", "LASI", "RASI", ...])
points: longblob         # Shape: [n_frames, n_markers, 5]
                         #   Channels: [x, y, z, residual, visibility]
                         #   Units: MILLIMETERS
                         #   Visibility: 0 = valid, non-zero = occluded
point_rate: float        # Sampling rate, typically 100 Hz
frames: longblob         # Frame indices
timestamps: longblob     # Computed as frames / point_rate (seconds)
```

**Analog** - Force plate and EMG data
```python
-> Trial
---
analog_names: longblob   # Channel labels (e.g., forceplate channels, EMG channels)
channels: longblob       # Shape: [n_channels, n_samples] (transposed)
timestamps: longblob     # Synced to mocap: mocap_start + arange(n_samples) / analog_rate
units: longblob          # Physical units per channel (e.g., "N", "V")
```

Analog data is synchronized to mocap via a shared trigger pulse. Timestamps are computed relative to the mocap frame start time.

**GaitEvents** - Annotated gait events from Vicon processing
```python
-> Mocap
event_num: smallint
---
event: varchar(100)          # e.g., "The instant the heel strikes the ground"
side: enum('Left', 'Right')
time: float                  # Event time in seconds relative to mocap timestamps
```

**GaitParameters** - Pre-computed clinical gait parameters
```python
-> Mocap
parameter: varchar(50)       # e.g., stride length, cadence
side: enum('Left', 'Right')
---
value: float
```

### Video Linkage

**GaitTrialVideo** - Links gait trials to the pose pipeline's Video table
```python
-> Trial
-> Video                     # from pose_pipeline (video_project, filename)
```

### Session Metadata

**SessionConditions** (Computed) - Parsed trial conditions from descriptions
```python
-> Session
---
num_conditions: smallint     # Number of distinct gait conditions tested
```

Part table **SessionCondition**:
```python
-> SessionConditions
support: varchar(50)         # "none", "cane", "walker", "gait_trainer", "handheld", "crutch"
left_foot: varchar(50)       # "barefoot", "afo", "smo", "kafos", "brace", "ucb", "insert"
right_foot: varchar(50)      # Same options as left_foot
shoe: varchar(50)            # Whether shoes were worn
---
num_mocap: smallint           # Trials with synchronized mocap
num_frontal: smallint         # Frontal-view-only video trials
num_sagital: smallint         # Sagittal-view-only video trials
```

Part table **ConditionTrial**:
```python
-> GaitTrialVideo
-> SessionConditions.SessionCondition
---
view: smallint               # 0=with mocap sync, 1=frontal only, 2=sagittal only
```

**Height** (Computed) - Session-level height
```python
-> Session
---
height: float                # Meters, from MocapAnalysis("HEIGHT") or MPParam("Height")
```

**Weight** (Computed) - Session-level weight
```python
-> Session
---
weight: float                # kg, from MocapAnalysis("BODYMASS") or MPParam("Bodymass")
```

---

## MocapProjection: Video-Mocap Synchronization

The `MocapProjection` table synchronizes 3D mocap markers with 2D video keypoints, solving for both a temporal offset and camera projection parameters.

**Source:** `gait_lab_dataset/mocap_sync.py`

### Table Definition

```python
-> Mocap
-> GaitTrialVideo
-> TopDownPerson              # 2D keypoints from pose detection
-> VideoInfo
---
toffset: float               # Temporal offset (seconds) between mocap and video
fx, fy: float                # Camera focal lengths (pixels)
cx, cy: float                # Principal point (image center)
tx, ty, tz: float            # 3D translation (mocap -> camera frame)
rx, ry, rz: float            # Rotation (Rodrigues / exponential map, stored at 1/1000 scale)
error: float                 # Huber loss of final projection fit (lower = better)
proj: longblob               # Projected mocap positions in 2D [n_frames, n_joints, 2]
frames: longblob             # Video frame indices where projection was valid
```

### Synchronization Algorithm

The algorithm jointly optimizes temporal offset and camera parameters to align 3D mocap markers to 2D pose detections:

1. **Data extraction** (`get_dual_data()`):
   - Fetches mocap 3D markers (hip, knee, ankle via Plug-In Gait bone model)
   - Fetches 2D keypoints from TopDownPerson (COCO format)
   - Filters to frames with high confidence 2D detections (>0.2) and valid mocap markers (visibility==0)
   - Swaps Y/Z axes to align mocap vertical with image vertical
   - Clips to temporal overlap with 200ms buffer at each end

2. **OpenCV initialization** (`cv2_calibrate()`):
   - Uses `cv2.calibrateCamera()` with all points across time as one observation
   - Fixes principal point at image center, fixes all distortion coefficients
   - Initial focal length: ~1200 pixels
   - Must achieve Huber loss < 12 (quality gate)

3. **JAX/BFGS optimization** (`optimize_projection()`):
   - 9 parameters: `[tx, ty, tz, rx, ry, rz, fx, fy, toffset_raw]`
   - Time offset bounded: `toffset = tanh(toffset_raw) * 0.2` (max +/- 200ms)
   - Mocap is interpolated at `timestamps + toffset` for each candidate offset
   - Loss: Huber loss between detected 2D keypoints and projected 3D markers
   - Uses `jax.scipy.optimize.minimize(method="BFGS", maxiter=1000)`
   - Projection uses `SO3.exp(rvec)` for rotation (jaxlie)

4. **Storage**: Optimized parameters stored in MocapProjection table. Note `rx, ry, rz` are stored at **1/1000 scale** (divided by 1000 before insert).

### Marker Mapping (Mocap <-> COCO)

The synchronization maps Plug-In Gait bone markers to COCO keypoints:

| COCO Keypoint | Plug-In Gait Marker | Description |
|---------------|---------------------|-------------|
| Right Hip | RFEP | Right Femur Proximal |
| Right Knee | RFEO | Right Femur Origin |
| Right Ankle | RTIO | Right Tibia Origin |
| Left Hip | LFEP | Left Femur Proximal |
| Left Knee | LFEO | Left Femur Origin |
| Left Ankle | LTIO | Left Tibia Origin |

### Data Statistics

- ~8,568 success