Datajoint Biomechanics Schema

DataJoint Biomechanics Schema Reference

CRITICAL: NEVER Modify Database Entries

DO NOT update, delete, or alter any DataJoint database entries. This includes:

• update1(), delete(), drop() on any table
• Modifying settings lookup tables (KinematicReconstructionSettingsLookup, ProbabilisticReconstructionSettingsLookup, KineticReconstructionSettingsLookup, KeypointSet, etc.)
• Altering any computed table entries

Database entries are shared state used by the entire lab. Changing a settings entry changes it for everyone and invalidates prior results computed with those settings.

Overview

The biomechanics pipeline uses DataJoint across four repositories with two complementary data collection approaches:

1. Multi-Camera (MMC) - Lab-based with calibrated camera rigs → MultiCameraTracking + BodyModels/kinematic_dj.py
2. Portable/Monocular (PBL) - Smartphone videos with optional IMU → PortableBiomechanicsSessions + BodyModels/monocular_dj.py

Both produce the same output format: qpos, qvel, joints, sites from MuJoCo body models.

DataJoint Query Basics

Operators

Fetch Patterns

# fetch1() - Exactly one row, returns tuple or dict
timestamps, qpos = (KinematicReconstruction.Trial & key).fetch1('timestamps', 'qpos')

# fetch() - Multiple rows, returns arrays
all_keys = (Session & filter).fetch('KEY')
keypoints = (TopDownPerson & key).fetch('keypoints')

# With ordering (critical for multi-camera consistency)
kp, names = (Table & key).fetch('keypoints', 'camera_name', order_by='camera_name')

Filtering by Project and Participant

video_project Field

video_project is a categorical field (varchar(50)) that identifies the study/cohort. It lives in MultiCameraRecording and Video tables.

Common projects:

Query Patterns by Project

# List all unique projects
projects = np.unique((MultiCameraRecording).fetch("video_project"))

# Find all participants in a project
participants = (Session & (Recording & (MultiCameraRecording & 'video_project="GAIT_CONTROLS"'))).fetch("participant_id")

# Filter trials by project
trials = KinematicReconstruction.Trial & (Recording & (MultiCameraRecording & f"video_project='{project}'"))

# Multiple projects with IN clause
filt = 'video_project in ("CLINIC_GAIT", "GAIT_CONTROLS", "PROSTHETIC_GAIT")'
sessions = Session & (Recording * MultiCameraRecording & filt)

Participant Naming Conventions

Session → Video Path (via video_project)

Session (participant_id, session_date)
    → Recording → MultiCameraRecording (video_project)
        → SingleCameraVideo → Video (video_project, filename)

Key Output Fields Reference

Common Access Patterns

# Get kinematic reconstruction (ALWAYS specify method!)
qpos, sites = (KinematicReconstruction.Trial & key &
               {'kinematic_reconstruction_settings_num': 137}).fetch1('qpos', 'sites')

# Get 3D triangulated keypoints
kp3d = (PersonKeypointReconstruction & key).fetch1('keypoints3d')

For 2D keypoints synchronized with KR qpos, see the Temporal Synchronization section below. Do NOT use (TopDownPerson * VideoInfo & key).fetch1('timestamps', 'keypoints') — this produces misaligned frames.

Temporal Synchronization (CRITICAL)

KR qpos and 2D keypoints have independently managed timestamps. Naive matching silently corrupts frame alignment.

Correct approach: Use fetch_keypoints(only_detected=True) from BodyModels:

from body_models.datajoint.dataset import fetch_keypoints as bm_fetch_keypoints

# Returns keypoints in 1:1 frame correspondence with KR qpos
timestamps, keypoints_raw = bm_fetch_keypoints(trial_key,
# keypoints_raw: (C, T, 87, 3) — (x, y, confidence)

# ALWAYS verify correspondence
assert qpos.shape[0] == keypoints_raw.shape[1]

Why direct TopDownPerson/VideoInfo fetch is broken: KR timestamps are zeroed relative to first detection; VideoInfo timestamps are zeroed relative to video start. When first detection occurs at frame N>0, matching on timestamp magnitude pairs the wrong frames (N-frame offset).

Camera parameter reordering: KinematicReconstruction.updated_calibration uses Calibration table order; keypoints use alphabetical SingleCameraVideo order. These differ — reorder explicitly before projection.

See rae:fetching-synchronized-data for the complete pattern with camera reordering and contiguous segment selection.

Two Data Collection Approaches

1. Multi-Camera (MMC) → KinematicReconstruction

Key space: (participant_id, session_date) bridged to (recording_timestamps, camera_config_hash)

Session → Recording → MultiCameraRecording → SingleCameraVideo → Video
                   ↘ SessionCalibration.Grouping → Calibration
                                     ↓
Video → TopDownPerson → PersonKeypointReconstruction (3D triangulation)
                                     ↓
                   KinematicReconstruction.Trial (qpos, qvel, joints, sites)

Bridge Tables:

• Recording - Links Session → MultiCameraRecording
• SingleCameraVideo - Links MultiCameraRecording → Video
• SessionCalibration.Grouping - Links Session → Calibration (required for KinematicReconstruction)

Use when: Lab-based data, multiple synchronized cameras, highest accuracy needed.

2. Portable/Monocular (PBL) → MonocularReconstruction

Key space: (subject_id, session_start_time) via Firebase

Subject → Session → Session.FirebaseSessionInfo
                              ↓
                    FirebaseSession (Computed)
                              ↓
            ┌─────────────────┼─────────────────┐
       AppVideo           Attitude/Gyro      PhoneAttitude
     (→ Video)            (IMU sensors)      (phone quaternion)
            ↓
    TopDownPerson → LiftingPerson (3D lifting)
                              ↓
            MonocularReconstruction.Trial (qpos, qvel, joints, sites, rnc)

Additional output: rnc (T, 3) - camera rotation vector from phone attitude

Use when: Smartphone videos, remote/home monitoring, wearable IMU/EMG data.

Comparison

Method Numbers by Pipeline

CRITICAL: Always specify the method number in queries to avoid duplicates.

Multi-Camera: Method 137

# CORRECT - Always specify method for KinematicReconstruction
key = {'participant_id': '102', 'session_date': date(2023, 7, 21),
       'kinematic_reconstruction_settings_num': 137}  # CRITICAL!

# WRONG - Returns BOTH 130 and 137 for dual-processed participants
key = {'participant_id': '102', 'session_date': date(2023, 7, 21)}

Dual-processed participants (have both 130 and 137): ASB_022, ASB_007, ASB_008, ASB_020, ASB_041, ASB_071, ASB_081, ASB_083, ASB_085, ASB_095, ASB_125, ASB_130

Method 137 config: humanoid_torque_rl.xml, floor leveling enabled, 40k iterations

Monocular: Method 1

# For MonocularReconstruction queries
key = {'subject_id': 123, 'session_start_time': ...,
       'monocular_reconstruction_settings_num': 1}

Bridging Algorithm

Bottom-up MeTRAbs detection inserted into top-down tables:

MeTRAbs (580 keypoints) → BottomUpBridging → BottomUpBridgingPerson
    ↓ filter_skeleton("bml_movi_87") → indices 264-350
TopDownPerson (method="Bridging_bml_movi_87")
    ↓ re-fetch from BottomUpBridging (not TopDownPerson!)
LiftingPerson (method="Bridging_bml_movi_87")

Why "weird": LiftingPerson normally uses TopDownPerson, but for bridging it re-fetches directly from BottomUpBridging.

Quick Reference: Key Files

Common Mistakes

1. Missing method number - Queries without kinematic_reconstruction_settings_num=137 return duplicates
2. Wrong key space - Using (video_project, filename) when you need (participant_id, session_date)
3. Assuming LiftingPerson uses TopDownPerson - For bridging methods, it re-fetches from BottomUpBridging
4. Missing SessionCalibration.Grouping - KinematicReconstruction requires this bridge
5. Forgetting order_by - Multi-camera queries need order_by='camera_name' for consistent ordering
6. Confusing MonocularReconstruction vs KinematicReconstruction - Monocular uses FirebaseSession.AppVideo, Kinematic uses SessionCalibration.Recordings
7. Not filtering by video_project - Queries across all projects mix different populations; always filter to your target cohort
8. Matching KR timestamps with VideoInfo timestamps - Use fetch_keypoints(only_detected=True) instead; see Temporal Synchronization section
9. Assuming camera_params order matches keypoint order - Calibration order != SingleCameraVideo alphabetical order; reorder explicitly
10. Zeroing timestamps manually (ts - ts[0]) - Destroys physical time correspondence between data sources