PyAV Integration Guide
Use PyAV when Python needs direct access to FFmpeg libraries: containers, streams, packets, frames, codecs, filters, seeking, subtitles, and NumPy conversion. This SKILL is a lean orchestrator; the full API examples are preserved in references/pyav-recipes-and-api.md.
When to Use PyAV
Use PyAV for:
- Frame-level video/audio decoding and encoding
- Precise seeking and keyframe extraction
- NumPy/Pillow/OpenCV frame access without shelling out per frame
- Container inspection, packet-level remuxing, subtitle reads
- RTSP/network stream handling with library-level control
Use FFmpeg CLI/subprocess instead for simple transcodes, production command parity, or heavy hardware-accelerated pipelines.
Quick Reference
| Task |
PyAV pattern |
Notes |
| Open media |
av.open(path) |
Prefer context manager |
| Decode video |
container.decode(video=0) |
Yields VideoFrame |
| To NumPy |
frame.to_ndarray(format='rgb24') |
Convert to BGR for OpenCV |
| From NumPy |
av.VideoFrame.from_ndarray(arr, format='rgb24') |
Encode generated frames |
| Seek |
container.seek(offset) |
Usually keyframe-based |
| Encode |
stream.encode(frame) then container.mux(packet) |
Flush encoder at end |
| Close |
container.close() |
Critical in loops |
Minimal Patterns
import av
with av.open("input.mp4") as container:
stream = container.streams.video[0]
stream.thread_type = "AUTO"
for frame in container.decode(stream):
rgb = frame.to_ndarray(format="rgb24")
break
import av
with av.open("input.mp4") as container:
stream = container.streams.video[0]
container.seek(int(10.0 * av.time_base), backward=True)
for frame in container.decode(stream):
if frame.time >= 10.0:
image = frame.to_ndarray(format="rgb24")
break
Core Workflow
- Install
av from wheels unless you explicitly need a custom FFmpeg build.
- Check PyAV and FFmpeg library compatibility before relying on new FFmpeg 8.x features.
- Open containers with a context manager or
try/finally close.
- Select streams explicitly; decode only what you need.
- Convert frame pixel/sample formats intentionally.
- For encoding, set codec, dimensions, pixel format, rate, and options explicitly; flush encoders.
- Disable verbose logging in threaded applications and prefer file paths over Python file objects.
Key Gotchas
- PyAV frames can reference underlying buffers; call
.copy() on NumPy arrays you keep after container close.
thread_type='AUTO' can greatly improve decode speed but may add frame delay.- PyAV hardware acceleration is limited compared with CLI FFmpeg.
- Subtitle transcoding support is limited; use FFmpeg CLI for complex subtitle workflows.
Reference Map
references/pyav-recipes-and-api.md - Full preserved reference: installation, custom FFmpeg builds, FFmpeg 8 compatibility, decoding, NumPy conversion, encoding, audio, filters, seeking, remuxing, subtitles, RTSP, errors, memory, threading, performance, hardware caveats, common patterns.
Related Skills
ffmpeg-python-integration-reference - Type-safe FFmpeg parameter mappingffmpeg-opencv-integration - OpenCV pipelines and color conversionffmpeg-fundamentals-2025 - Core FFmpeg operations and codec choices
1---2name: ffmpeg-pyav-integration3description: Complete PyAV (Python FFmpeg bindings) integration guide. PROACTIVELY activate for: (1) PyAV installation on Ubuntu/Windows/macOS, (2) Building PyAV against custom FFmpeg, (3) FFmpeg 7.0/8.0+ compatibility, (4) av.open() video/audio decoding, (5) VideoFrame/AudioFrame NumPy conversion, (6) Filter graph processing, (7) Video encoding with H.264/H.265/AV1, (8) Seeking and keyframe extraction, (9) RTSP/network streaming with PyAV, (10) Memory management and thread safety, (11) Error handling with FFmpegError, (12) Subtitle extraction, (13) Container manipulation and remuxing, (14) Performance optimization and threading. Provides: Complete PyAV API patterns, installation guides for all Ubuntu versions, FFmpeg 8.0+ compatibility matrix, type-safe examples, memory management best practices, filter graph examples, encoding/decoding patterns.4---5
6# PyAV Integration Guide
7
8Use PyAV when Python needs direct access to FFmpeg libraries: containers, streams, packets, frames, codecs, filters, seeking, subtitles, and NumPy conversion. This SKILL is a lean orchestrator; the full API examples are preserved in `references/pyav-recipes-and-api.md`.
9
10## When to Use PyAV
11
12Use PyAV for:
13
14- Frame-level video/audio decoding and encoding
15- Precise seeking and keyframe extraction
16- NumPy/Pillow/OpenCV frame access without shelling out per frame
17- Container inspection, packet-level remuxing, subtitle reads
18- RTSP/network stream handling with library-level control
19
20Use FFmpeg CLI/subprocess instead for simple transcodes, production command parity, or heavy hardware-accelerated pipelines.
21
22## Quick Reference
23
24| Task | PyAV pattern | Notes |
25|---|---|---|
26| Open media | `av.open(path)` | Prefer context manager |
27| Decode video | `container.decode(video=0)` | Yields `VideoFrame` |
28| To NumPy | `frame.to_ndarray(format='rgb24')` | Convert to BGR for OpenCV |
29| From NumPy | `av.VideoFrame.from_ndarray(arr, format='rgb24')` | Encode generated frames |
30| Seek | `container.seek(offset)` | Usually keyframe-based |
31| Encode | `stream.encode(frame)` then `container.mux(packet)` | Flush encoder at end |
32| Close | `container.close()` | Critical in loops |
33
34## Minimal Patterns
35
36```python
37import av
38
39with av.open("input.mp4") as container:
40 stream = container.streams.video[0]
41 stream.thread_type = "AUTO"
42 for frame in container.decode(stream):
43 rgb = frame.to_ndarray(format="rgb24")
44 break
45```
46
47```python
48import av
49
50with av.open("input.mp4") as container:
51 stream = container.streams.video[0]
52 container.seek(int(10.0 * av.time_base), backward=True)
53 for frame in container.decode(stream):
54 if frame.time >= 10.0:
55 image = frame.to_ndarray(format="rgb24")
56 break
57```
58
59## Core Workflow
60
611. Install `av` from wheels unless you explicitly need a custom FFmpeg build.
622. Check PyAV and FFmpeg library compatibility before relying on new FFmpeg 8.x features.
633. Open containers with a context manager or `try/finally` close.
644. Select streams explicitly; decode only what you need.
655. Convert frame pixel/sample formats intentionally.
666. For encoding, set codec, dimensions, pixel format, rate, and options explicitly; flush encoders.
677. Disable verbose logging in threaded applications and prefer file paths over Python file objects.
68
69## Key Gotchas
70
71- PyAV frames can reference underlying buffers; call `.copy()` on NumPy arrays you keep after container close.
72- `thread_type='AUTO'` can greatly improve decode speed but may add frame delay.
73- PyAV hardware acceleration is limited compared with CLI FFmpeg.
74- Subtitle transcoding support is limited; use FFmpeg CLI for complex subtitle workflows.
75
76## Reference Map
77
78- `references/pyav-recipes-and-api.md` - Full preserved reference: installation, custom FFmpeg builds, FFmpeg 8 compatibility, decoding, NumPy conversion, encoding, audio, filters, seeking, remuxing, subtitles, RTSP, errors, memory, threading, performance, hardware caveats, common patterns.
79
80## Related Skills
81
82- `ffmpeg-python-integration-reference` - Type-safe FFmpeg parameter mapping
83- `ffmpeg-opencv-integration` - OpenCV pipelines and color conversion
84- `ffmpeg-fundamentals-2025` - Core FFmpeg operations and codec choices
josiahsiegel