Rust Ffmpeg

Rust FFmpeg

Guide for implementing FFmpeg functionality in Rust: library selection, code generation, and problem solving.

Selection Framework

Decision Logic

Layer 1: Integration Method

Default: Library Integration (ez-ffmpeg/ffmpeg-next/ffmpeg-sys-next)

• Better performance, type-safe API, direct Frame access

Alternative: Binary Approach (ffmpeg-sidecar) - consider when:

• Cannot install FFmpeg development libraries
• Restricted CI/CD environment without admin access
• Pure CLI batch processing (no real-time needs)

If installation constrained → Load ffmpeg_sidecar.md

Layer 2: Scenario Detection

Layer 3: Library Selection

CLI Migration: Both ez-ffmpeg and ffmpeg-sidecar support CLI-style APIs. Choose based on constraints below.

• ez-ffmpeg: CLI Migration Guide — library integration, native performance
• ffmpeg-sidecar: CLI to Rust Migration — process wrapper, no FFmpeg libs needed

1. Need async/await? → ez-ffmpeg (only library with native async)
2. Need custom Rust frame processing? → ez-ffmpeg FrameFilter (safe, simple API)
3. Need frame-level control? → ffmpeg-next for codec internals
4. Need custom I/O from memory? → ez-ffmpeg (read/write/seek callbacks); use ffmpeg-next/ffmpeg-sys-next for lower-level control
5. Need zero-copy or max performance? → ffmpeg-sys-next (requires unsafe code)
6. Cannot install FFmpeg libs? → ffmpeg-sidecar (process-based, stdin/stdout only, no custom I/O)

Note: FrameFilter (custom Rust frame processing) is separate from custom I/O callbacks (custom data sources/sinks). ez-ffmpeg supports both.

Quick Start

New to Rust FFmpeg? See quick_start.md for 5-minute setup.

Library References

• ez-ffmpeg - High-level API (sync and async)
• ffmpeg-next - Medium-level API for advanced control
• ffmpeg-sys-next - Low-level unsafe FFI
• ffmpeg-sidecar - CLI wrapper

Version Compatibility

Source: crates.io

Why not 8.x (ffmpeg-next / ffmpeg-sys-next)? Upstream rust-ffmpeg issue #246 reports a compilation error with FFmpeg 8.0 due to missing EXIF side data types (AV_FRAME_DATA_EXIF, AV_PKT_DATA_EXIF). We pin to 7.1.0 until it is resolved.

Installation Issues: installation.md

Guidelines for Claude

When this skill activates, follow this workflow:

For implementation tasks:

1. Identify task — Determine: video/audio/streaming/inspection? Simple or complex?
2. Select library — Apply Layer 3 decision logic. State which library and why in one sentence
3. Load references — Follow Layer 2 scenario detection to load the right reference files
4. Generate code — Production-ready: proper error handling, Result<> return types, no unwrap() in library code
5. Explain briefly — One-line summary of approach before the code block. No lengthy tutorials
6. Suggest next steps — If applicable: performance optimization, hardware acceleration, or testing

For evaluation/migration/selection tasks:

1. Load selection guide — Load library_selection.md and relevant scenario references
2. Assess requirements — Identify constraints: async needs, frame-level access, install limitations, existing code patterns
3. Compare options — Apply Layer 3 decision logic against the specific use case
4. Give verdict — Clear feasibility conclusion with rationale. If migration is possible, show key API differences. If not, explain why and recommend the alternative

Rules:

• Follow Layer 3 decision logic to select the library
• Always add Cargo.toml dependencies when introducing a new library
• Use async when the user's context is async (tokio/actix/axum)
• If the user's need is ambiguous between libraries, ask — don't guess
• For complex pipelines, break into steps with comments, not monolithic blocks

Interaction Examples

User: "Write a Rust function to extract audio from MP4" Claude: Identify → audio extraction → apply Layer 3 (general task, no special requirements) → load audio_extraction.md → generate function with proper error handling → include Cargo.toml dep

User: "I need frame-level access to decode H.264 and apply custom processing" Claude: Identify → frame-level + custom processing → ez-ffmpeg FrameFilter (safe, simple) or ffmpeg-next (codec internals) → ask user preference if unclear → load ffmpeg_next.md or ez_ffmpeg/filters.md → generate decode loop with proper EAGAIN handling

User: "How to do RTMP live streaming in Rust?" Claude: Identify → streaming + real-time → ez-ffmpeg (async + RTMP server) → load streaming_rtmp_hls.md → generate RTMP push/server example → mention jitter buffer for production use

User: "Can this video trimming code be converted to ez-ffmpeg? Fall back to ffmpeg-next if not" Claude: Identify → library migration feasibility → load library_selection.md + video_transcoding.md → review existing code against ez-ffmpeg API → assess feasibility → provide migration path or explain why ffmpeg-next is needed

User: "Which Rust FFmpeg library should I use for my project?" Claude: Identify → library selection → load library_selection.md → ask about requirements (async? frame-level? install constraints?) → apply Layer 3 decision logic → recommend with rationale

Best Practices

• Codec copy first: Use -c copy / stream copy when no re-encoding is needed — 10x faster, zero quality loss
• Keyframe alignment: Set force_key_frames for HLS/DASH segmentation to avoid playback glitches
• Error propagation: Return Result<T, Box<dyn Error>>, never panic in library code
• Resource cleanup: Use RAII patterns; FfmpegContext/Decoder/Encoder drop automatically
• Hardware acceleration: Probe availability at runtime before enabling — not all machines have GPU support
• Testing: Use testsrc / sine filters to generate synthetic test media instead of shipping binary fixtures

Converted and distributed by TomeVault — claim your Tome and manage your conversions.