Drone CV Expert
Expert in robotics, drone systems, and computer vision for autonomous aerial platforms.
Decision Tree: When to Use This Skill
User mentions drones or UAVs?
├─ YES → Is it about inspection/detection of specific things (fire, roof damage, thermal)?
│ ├─ YES → Use drone-inspection-specialist
│ └─ NO → Is it about flight control, navigation, or general CV?
│ ├─ YES → Use THIS SKILL (drone-cv-expert)
│ └─ NO → Is it about GPU rendering/shaders?
│ ├─ YES → Use metal-shader-expert
│ └─ NO → Use THIS SKILL as default drone skill
└─ NO → Is it general object detection without drone context?
├─ YES → Use clip-aware-embeddings or other CV skill
└─ NO → Probably not a drone question
Core Competencies
Flight Control & Navigation
- PID Tuning: Position, velocity, attitude control loops
- SLAM: ORB-SLAM, LSD-SLAM, visual-inertial odometry (VIO)
- Path Planning: A*, RRT, RRT*, Dijkstra, potential fields
- Sensor Fusion: EKF, UKF, complementary filters
- GPS-Denied Navigation: AprilTags, visual odometry, LiDAR SLAM
Computer Vision
- Object Detection: YOLO (v5/v8/v10), EfficientDet, SSD
- Tracking: ByteTrack, DeepSORT, SORT, optical flow
- Edge Deployment: TensorRT, ONNX, OpenVINO optimization
- 3D Vision: Stereo depth, point clouds, structure-from-motion
Hardware Integration
- Flight Controllers: Pixhawk, Ardupilot, PX4, DJI
- Protocols: MAVLink, DroneKit, MAVSDK
- Edge Compute: Jetson (Nano/Xavier/Orin), Coral TPU
- Sensors: IMU, GPS, barometer, LiDAR, depth cameras
Anti-Patterns to Avoid
1. "Simulation-Only Syndrome"
Wrong: Testing only in Gazebo/AirSim, then deploying directly to real drone.
Right: Simulation → Bench test → Tethered flight → Controlled environment → Field.
2. "EKF Overkill"
Wrong: Using Extended Kalman Filter when complementary filter suffices.
Right: Match filter complexity to requirements:
- Complementary filter: Basic stabilization, attitude only
- EKF: Multi-sensor fusion, GPS+IMU+baro
- UKF: Highly nonlinear systems, aggressive maneuvers
3. "Max Resolution Assumption"
Wrong: Processing 4K frames at 30fps expecting real-time performance.
Right: Resolution trade-offs by altitude/speed:
| Altitude |
Speed |
Resolution |
FPS |
Rationale |
| <30m |
Slow |
1920x1080 |
30 |
Detail needed |
| 30-100m |
Medium |
1280x720 |
30 |
Balance |
| >100m |
Fast |
640x480 |
60 |
Speed priority |
4. "Single-Thread Processing"
Wrong: Sequential detect → track → control in one loop.
Right: Pipeline parallelism:
Thread 1: Camera capture (async)
Thread 2: Object detection (GPU)
Thread 3: Tracking + state estimation
Thread 4: Control commands
5. "GPS Trust"
Wrong: Assuming GPS is always accurate and available.
Right: Multi-source position estimation:
- GPS: 2-5m accuracy outdoor, unavailable indoor
- Visual odometry: 0.1-1% drift, lighting dependent
- AprilTags: cm-level accuracy where deployed
- IMU: Short-term only, drift accumulates
6. "One Model Fits All"
Wrong: Using same YOLO model for all scenarios.
Right: Model selection by constraint:
| Constraint |
Model |
Notes |
| Latency critical |
YOLOv8n |
6ms inference |
| Balanced |
YOLOv8s |
15ms, better accuracy |
| Accuracy first |
YOLOv8x |
50ms, highest mAP |
| Edge device |
YOLOv8n + TensorRT |
3ms on Jetson |
Problem-Solving Framework
1. Constraint Analysis
- Compute: What hardware? (Jetson Nano = ~5 TOPS, Xavier = 32 TOPS)
- Power: Battery capacity? Flight time impact?
- Latency: Control loop rate? Detection response time?
- Weight: Payload capacity? Center of gravity?
- Environment: Indoor/outdoor? GPS available? Lighting conditions?
2. Algorithm Selection Matrix
| Problem |
Classical Approach |
Deep Learning |
When to Use Each |
| Feature tracking |
KLT optical flow |
FlowNet |
Classical: Real-time, limited compute. DL: Robust, more compute |
| Object detection |
HOG+SVM |
YOLO/SSD |
Classical: Simple objects, no GPU. DL: Complex, GPU available |
| SLAM |
ORB-SLAM |
DROID-SLAM |
Classical: Mature, debuggable. DL: Better in challenging scenes |
| Path planning |
A*, RRT |
RL-based |
Classical: Known environments. DL: Complex, dynamic |
3. Safety Checklist
Quick Reference Tables
MAVLink Message Types
| Message |
Purpose |
Frequency |
| HEARTBEAT |
Connection alive |
1 Hz |
| ATTITUDE |
Roll/pitch/yaw |
10-100 Hz |
| LOCAL_POSITION_NED |
Position |
10-50 Hz |
| GPS_RAW_INT |
Raw GPS |
1-10 Hz |
| SET_POSITION_TARGET |
Commands |
As needed |
Kalman Filter Tuning
| Matrix |
High Values |
Low Values |
| Q (process noise) |
Trust measurements more |
Trust model more |
| R (measurement noise) |
Trust model more |
Trust measurements more |
| P (initial covariance) |
Uncertain initial state |
Confident initial state |
Common Coordinate Frames
| Frame |
Origin |
Axes |
Use |
| NED |
Takeoff point |
North-East-Down |
Navigation |
| ENU |
Takeoff point |
East-North-Up |
ROS standard |
| Body |
Drone CG |
Forward-Right-Down |
Control |
| Camera |
Lens center |
Right-Down-Forward |
Vision |
Reference Files
Detailed implementations in references/:
navigation-algorithms.md - SLAM, path planning, localizationsensor-fusion-ekf.md - Kalman filters, multi-sensor fusionobject-detection-tracking.md - YOLO, ByteTrack, optical flow
Simulation Tools
| Tool |
Strengths |
Weaknesses |
Best For |
| Gazebo |
ROS integration, physics |
Graphics quality |
ROS development |
| AirSim |
Photorealistic, CV-focused |
Windows-centric |
Vision algorithms |
| Webots |
Multi-robot, accessible |
Less drone-specific |
Swarm simulations |
| MATLAB/Simulink |
Control design |
Not real-time |
Controller tuning |
Emerging Technologies (2024-2025)
- Event cameras: 1μs temporal resolution, no motion blur
- Neuromorphic computing: Loihi 2 for ultra-low-power inference
- 4D Radar: Velocity + 3D position, works in all weather
- Swarm autonomy: Decentralized coordination, emergent behavior
- Foundation models: SAM, CLIP for zero-shot detection
Integration Points
- drone-inspection-specialist: Domain-specific detection (fire, damage, thermal)
- metal-shader-expert: GPU-accelerated vision processing, custom shaders
- collage-layout-expert: Report generation, visual composition
Key Principle: In drone systems, reliability trumps performance. A 95% accurate system that never crashes is better than 99% accurate that fails unpredictably. Always have fallbacks.
1---2name: drone-cv-expert-23description: Expert in drone systems, computer vision, and autonomous navigation. Specializes in flight control, SLAM, object detection, sensor fusion, and path planning. Activate on "drone", "UAV", "SLAM", "visual odometry", "PID control", "MAVLink", "Pixhawk", "path planning", "A*", "RRT", "EKF", "sensor fusion", "optical flow", "ByteTrack". NOT for domain-specific inspection tasks like fire detection, roof damage assessment, or thermal analysis (use drone-inspection-specialist), GPU shader optimization (use metal-shader-expert), or general image classification without drone context (use clip-aware-embeddings).4---56# Drone CV Expert78Expert in robotics, drone systems, and computer vision for autonomous aerial platforms.910## Decision Tree: When to Use This Skill1112```13User mentions drones or UAVs?14├─ YES → Is it about inspection/detection of specific things (fire, roof damage, thermal)?15│ ├─ YES → Use drone-inspection-specialist16│ └─ NO → Is it about flight control, navigation, or general CV?17│ ├─ YES → Use THIS SKILL (drone-cv-expert)18│ └─ NO → Is it about GPU rendering/shaders?19│ ├─ YES → Use metal-shader-expert20│ └─ NO → Use THIS SKILL as default drone skill21└─ NO → Is it general object detection without drone context?22 ├─ YES → Use clip-aware-embeddings or other CV skill23 └─ NO → Probably not a drone question24```2526## Core Competencies2728### Flight Control & Navigation29- **PID Tuning**: Position, velocity, attitude control loops30- **SLAM**: ORB-SLAM, LSD-SLAM, visual-inertial odometry (VIO)31- **Path Planning**: A*, RRT, RRT*, Dijkstra, potential fields32- **Sensor Fusion**: EKF, UKF, complementary filters33- **GPS-Denied Navigation**: AprilTags, visual odometry, LiDAR SLAM3435### Computer Vision36- **Object Detection**: YOLO (v5/v8/v10), EfficientDet, SSD37- **Tracking**: ByteTrack, DeepSORT, SORT, optical flow38- **Edge Deployment**: TensorRT, ONNX, OpenVINO optimization39- **3D Vision**: Stereo depth, point clouds, structure-from-motion4041### Hardware Integration42- **Flight Controllers**: Pixhawk, Ardupilot, PX4, DJI43- **Protocols**: MAVLink, DroneKit, MAVSDK44- **Edge Compute**: Jetson (Nano/Xavier/Orin), Coral TPU45- **Sensors**: IMU, GPS, barometer, LiDAR, depth cameras4647## Anti-Patterns to Avoid4849### 1. "Simulation-Only Syndrome"50**Wrong**: Testing only in Gazebo/AirSim, then deploying directly to real drone.51**Right**: Simulation → Bench test → Tethered flight → Controlled environment → Field.5253### 2. "EKF Overkill"54**Wrong**: Using Extended Kalman Filter when complementary filter suffices.55**Right**: Match filter complexity to requirements:56- Complementary filter: Basic stabilization, attitude only57- EKF: Multi-sensor fusion, GPS+IMU+baro58- UKF: Highly nonlinear systems, aggressive maneuvers5960### 3. "Max Resolution Assumption"61**Wrong**: Processing 4K frames at 30fps expecting real-time performance.62**Right**: Resolution trade-offs by altitude/speed:63| Altitude | Speed | Resolution | FPS | Rationale |64|----------|-------|------------|-----|-----------|65| <30m | Slow | 1920x1080 | 30 | Detail needed |66| 30-100m | Medium | 1280x720 | 30 | Balance |67| >100m | Fast | 640x480 | 60 | Speed priority |6869### 4. "Single-Thread Processing"70**Wrong**: Sequential detect → track → control in one loop.71**Right**: Pipeline parallelism:72```73Thread 1: Camera capture (async)74Thread 2: Object detection (GPU)75Thread 3: Tracking + state estimation76Thread 4: Control commands77```7879### 5. "GPS Trust"80**Wrong**: Assuming GPS is always accurate and available.81**Right**: Multi-source position estimation:82- GPS: 2-5m accuracy outdoor, unavailable indoor83- Visual odometry: 0.1-1% drift, lighting dependent84- AprilTags: cm-level accuracy where deployed85- IMU: Short-term only, drift accumulates8687### 6. "One Model Fits All"88**Wrong**: Using same YOLO model for all scenarios.89**Right**: Model selection by constraint:90| Constraint | Model | Notes |91|------------|-------|-------|92| Latency critical | YOLOv8n | 6ms inference |93| Balanced | YOLOv8s | 15ms, better accuracy |94| Accuracy first | YOLOv8x | 50ms, highest mAP |95| Edge device | YOLOv8n + TensorRT | 3ms on Jetson |9697## Problem-Solving Framework9899### 1. Constraint Analysis100- **Compute**: What hardware? (Jetson Nano = ~5 TOPS, Xavier = 32 TOPS)101- **Power**: Battery capacity? Flight time impact?102- **Latency**: Control loop rate? Detection response time?103- **Weight**: Payload capacity? Center of gravity?104- **Environment**: Indoor/outdoor? GPS available? Lighting conditions?105106### 2. Algorithm Selection Matrix107108| Problem | Classical Approach | Deep Learning | When to Use Each |109|---------|-------------------|---------------|------------------|110| Feature tracking | KLT optical flow | FlowNet | Classical: Real-time, limited compute. DL: Robust, more compute |111| Object detection | HOG+SVM | YOLO/SSD | Classical: Simple objects, no GPU. DL: Complex, GPU available |112| SLAM | ORB-SLAM | DROID-SLAM | Classical: Mature, debuggable. DL: Better in challenging scenes |113| Path planning | A*, RRT | RL-based | Classical: Known environments. DL: Complex, dynamic |114115### 3. Safety Checklist116- [ ] Kill switch tested and accessible117- [ ] Geofence configured118- [ ] Return-to-home altitude set119- [ ] Low battery action defined120- [ ] Signal loss action defined121- [ ] Propeller guards (if applicable)122- [ ] Pre-flight sensor calibration123- [ ] Weather conditions checked124125## Quick Reference Tables126127### MAVLink Message Types128| Message | Purpose | Frequency |129|---------|---------|-----------|130| HEARTBEAT | Connection alive | 1 Hz |131| ATTITUDE | Roll/pitch/yaw | 10-100 Hz |132| LOCAL_POSITION_NED | Position | 10-50 Hz |133| GPS_RAW_INT | Raw GPS | 1-10 Hz |134| SET_POSITION_TARGET | Commands | As needed |135136### Kalman Filter Tuning137| Matrix | High Values | Low Values |138|--------|-------------|------------|139| Q (process noise) | Trust measurements more | Trust model more |140| R (measurement noise) | Trust model more | Trust measurements more |141| P (initial covariance) | Uncertain initial state | Confident initial state |142143### Common Coordinate Frames144| Frame | Origin | Axes | Use |145|-------|--------|------|-----|146| NED | Takeoff point | North-East-Down | Navigation |147| ENU | Takeoff point | East-North-Up | ROS standard |148| Body | Drone CG | Forward-Right-Down | Control |149| Camera | Lens center | Right-Down-Forward | Vision |150151## Reference Files152153Detailed implementations in `references/`:154- `navigation-algorithms.md` - SLAM, path planning, localization155- `sensor-fusion-ekf.md` - Kalman filters, multi-sensor fusion156- `object-detection-tracking.md` - YOLO, ByteTrack, optical flow157158## Simulation Tools159160| Tool | Strengths | Weaknesses | Best For |161|------|-----------|------------|----------|162| Gazebo | ROS integration, physics | Graphics quality | ROS development |163| AirSim | Photorealistic, CV-focused | Windows-centric | Vision algorithms |164| Webots | Multi-robot, accessible | Less drone-specific | Swarm simulations |165| MATLAB/Simulink | Control design | Not real-time | Controller tuning |166167## Emerging Technologies (2024-2025)168169- **Event cameras**: 1μs temporal resolution, no motion blur170- **Neuromorphic computing**: Loihi 2 for ultra-low-power inference171- **4D Radar**: Velocity + 3D position, works in all weather172- **Swarm autonomy**: Decentralized coordination, emergent behavior173- **Foundation models**: SAM, CLIP for zero-shot detection174175## Integration Points176177- **drone-inspection-specialist**: Domain-specific detection (fire, damage, thermal)178- **metal-shader-expert**: GPU-accelerated vision processing, custom shaders179- **collage-layout-expert**: Report generation, visual composition180181---182183**Key Principle**: In drone systems, reliability trumps performance. A 95% accurate system that never crashes is better than 99% accurate that fails unpredictably. Always have fallbacks.
curiositech