Deeplabcut

DeepLabCut — Markerless Animal Pose Estimation

Purpose

DeepLabCut is a Python toolbox for state-of-the-art markerless pose estimation of animals. It uses deep learning to track body parts from videos without physical markers. The library is animal-agnostic, supports both single and multi-animal scenarios, and includes the SuperAnimal family of pretrained models for out-of-the-box inference.

When to Use This Skill

Activate when the user:

• Wants to track animal body parts from video
• Asks about pose estimation, keypoint detection, or behavioral tracking
• Mentions DeepLabCut, DLC, SuperAnimal, or markerless tracking
• Needs to analyze animal movement/kinematics
• Asks about multi-animal tracking or 3D pose reconstruction
• Wants to use pretrained animal pose models

Quick Decision Tree

What does the user need?
├── No labeled data, just want to track animals → SuperAnimal (video_inference_superanimal)
├── Single animal, have labeled data → Standard single-animal pipeline
├── Multiple animals interacting → maDLC (multi-animal pipeline)
├── 3D pose reconstruction → 3D pipeline (calibrate_cameras + triangulate)
└── Just post-process results → filterpredictions, analyzeskeleton

Reference Files (Progressive Disclosure)

Installation

# Minimal (headless, no GUI):
pip install deeplabcut

# With GUI (label_frames, refine_labels, SkeletonBuilder):
pip install "deeplabcut[gui]"

# PyTorch must be installed separately:
pip install torch torchvision
# Or for GPU (check pytorch.org for your CUDA version):
conda install pytorch cudatoolkit=11.3 -c pytorch

Verify: python -c "import deeplabcut; print(deeplabcut.__version__)"

Standard Pipeline Overview

import deeplabcut as dlc

# 1. Create project
config_path = dlc.create_new_project(
    "ProjectName", "ExperimenterName", ["/path/to/video.mp4"],
    working_directory="/path/to/projects"
)

# 2. Extract frames for labeling
dlc.extract_frames(config_path, mode="automatic", algo="kmeans", crop=True)

# 3. USER labels frames manually in GUI
# dlc.label_frames(config_path)  # launches the labeling GUI

# 4. Create training dataset from labeled frames
dlc.create_training_dataset(config_path, net_type="resnet_50")

# 5. Train the network
dlc.train_network(config_path, maxiters=100000, saveiters=5000)

# 6. Evaluate
dlc.evaluate_network(config_path, plotting=True)

# 7. Analyze videos (predict poses)
dlc.analyze_videos(config_path, ["/path/to/video.mp4"], videotype=".mp4")

# 8. Create labeled videos (overlay predictions)
dlc.create_labeled_video(config_path, ["/path/to/video.mp4"])

# 9. Export results to CSV
dlc.analyze_videos_converth5_to_csv("/path/to/videoDLC_resnet50_ProjectNameJul9")

Key API Reference

Project Management

Training & Evaluation

Video Analysis

Post-Processing

SuperAnimal Pretrained Models

Available models for zero-shot inference:

Common Pitfalls

1. Wrong PyTorch version: Always install PyTorch BEFORE deeplabcut. Check pytorch.org for the correct CUDA version.
2. GPU out of memory: Reduce batch size (default 8 → 4 or 2) in pose_cfg.yaml.
3. Missing GUI dependencies: label_frames requires pip install "deeplabcut[gui]". On headless servers, use X11 forwarding or label locally.
4. Video codec issues: If create_labeled_video fails, try converting to .mp4 (H.264) or .avi first.
5. maDLC detector: Multi-animal mode requires a detection model (fasterrcnn_resnet50_fpn_v2 is the default). Single-animal can work with heatmap regression alone.
6. SuperAnimal scale: Adjust scale_list parameter for SuperAnimal — try [200, 300, 400] first; smaller animals may need higher values.