Ffmpeg

FFmpeg Skill

FFmpeg is a comprehensive open-source multimedia framework that handles video, audio, and other multimedia files and streams. It provides command-line tools and libraries for recording, converting, and streaming audio and video.

When to Use This Skill

Use this skill when:

• Converting media files between formats (MP4, MKV, WebM, AVI, etc.)
• Encoding/decoding video with different codecs (H.264, H.265, VP9, AV1)
• Processing audio files (AAC, MP3, FLAC, Opus)
• Live streaming to platforms (Twitch, YouTube, RTMP servers)
• Creating HLS/DASH adaptive streams
• Extracting audio from video files
• Applying video/audio filters (scale, crop, denoise, volume)
• Screen recording and capture
• Optimizing video quality and file size
• Batch processing media files
• Analyzing media file properties

Core Components

Command-Line Tools

ffmpeg: Primary tool for transcoding, conversion, and streaming ffprobe: Media analysis and inspection tool ffplay: Minimalist multimedia player for testing

Libraries

• libavcodec: Encoding/decoding codecs
• libavformat: Container formats and I/O
• libavfilter: Audio/video filtering framework
• libavutil: Utility functions
• libswscale: Video scaling and color conversion
• libswresample: Audio resampling and mixing

Common Video Codecs

Modern Codecs

H.264 (libx264): Most widely supported, excellent compression/quality balance

• Best for: Universal compatibility, streaming, web video
• Quality range: CRF 17-28 (lower = better quality)

H.265/HEVC (libx265): 25-50% better compression than H.264

• Best for: 4K video, file size reduction
• Slower encoding, limited browser support

VP9: Royalty-free, WebM format

• Best for: YouTube, open-source projects
• Good quality/compression, Chrome/Firefox support

AV1 (libaom-av1, libsvtav1): Next-generation codec, best compression

• Best for: Future-proofing, ultimate quality/size ratio
• Very slow encoding (improving with SVT-AV1)

Legacy Codecs

• MPEG-4 (Xvid/DivX)
• MPEG-2
• VP8

Common Audio Codecs

AAC: Industry standard, excellent quality MP3: Universal compatibility, good quality Opus: Best quality at low bitrates, ideal for voice/streaming FLAC: Lossless compression, archival quality Vorbis: Open-source, good quality AC-3/DTS: Surround sound formats

Container Formats

MP4: Universal compatibility, streaming-friendly MKV (Matroska): Feature-rich, supports multiple tracks/subtitles WebM: Web-optimized (VP8/VP9 + Vorbis/Opus) AVI: Legacy format, broad compatibility MOV: Apple QuickTime format TS: Transport stream for broadcasting FLV: Flash video (legacy streaming)

Basic Operations

Format Conversion

Simple format conversion without re-encoding:

ffmpeg -i input.mkv -c copy output.mp4

Convert with specific codecs:

ffmpeg -i input.avi -c:v libx264 -c:a aac output.mp4

Quality-Based Encoding (CRF)

Constant Rate Factor - best for quality-focused encoding:

# H.264 encoding (CRF 23 = default, 17-28 recommended range)
ffmpeg -i input.mkv -c:v libx264 -preset slow -crf 22 -c:a copy output.mp4

# H.265 encoding (higher compression)
ffmpeg -i input.mkv -c:v libx265 -preset medium -crf 24 -c:a copy output.mp4

# VP9 encoding (WebM)
ffmpeg -i input.mkv -c:v libvpx-vp9 -crf 30 -b:v 0 -c:a libopus output.webm

CRF Scale:

• 0 = Lossless (huge file size)
• 17-18 = Visually lossless
• 20-23 = High quality (recommended)
• 24-28 = Medium quality
• 30+ = Low quality
• 51 = Worst quality

Presets (speed vs compression):

• ultrafast, superfast, veryfast, faster, fast
• medium (default)
• slow, slower, veryslow (better compression)
• placebo (not recommended - minimal gains)

Bitrate-Based Encoding (Two-Pass)

For targeting specific file sizes:

# Pass 1 (analysis)
ffmpeg -y -i input.mkv -c:v libx264 -b:v 2600k -pass 1 -an -f null /dev/null

# Pass 2 (encoding)
ffmpeg -i input.mkv -c:v libx264 -b:v 2600k -pass 2 -c:a aac -b:a 128k output.mp4

Audio Extraction

Extract audio without re-encoding:

ffmpeg -i video.mp4 -vn -c:a copy audio.m4a

Extract and convert to MP3:

ffmpeg -i video.mp4 -vn -q:a 0 audio.mp3

Extract with specific bitrate:

ffmpeg -i video.mp4 -vn -c:a libmp3lame -b:a 192k audio.mp3

Video Scaling/Resizing

Scale to specific dimensions:

ffmpeg -i input.mp4 -vf scale=1280:720 output.mp4

Scale maintaining aspect ratio:

# Width 1280, height auto-calculated
ffmpeg -i input.mp4 -vf scale=1280:-1 output.mp4

# Height 720, width auto-calculated
ffmpeg -i input.mp4 -vf scale=-1:720 output.mp4

Scale to half resolution:

ffmpeg -i input.mp4 -vf scale=iw/2:ih/2 output.mp4

Trimming/Cutting Video

Cut without re-encoding (fast but less precise):

ffmpeg -i input.mp4 -ss 00:01:30 -to 00:03:00 -c copy output.mp4

Cut with re-encoding (precise):

ffmpeg -i input.mp4 -ss 00:01:30 -t 00:01:30 -c:v libx264 -c:a aac output.mp4

Parameters:

• -ss: Start time (HH:MM:SS or seconds)
• -to: End time
• -t: Duration (instead of end time)

Concatenation

Method 1: Concat demuxer (same codec/format)

Create concat.txt:

file 'input1.mp4'
file 'input2.mp4'
file 'input3.mp4'

Run:

ffmpeg -f concat -safe 0 -i concat.txt -c copy output.mp4

Method 2: Concat filter (different formats)

ffmpeg -i input1.mp4 -i input2.mp4 -i input3.mp4 \
  -filter_complex "[0:v][0:a][1:v][1:a][2:v][2:a]concat=n=3:v=1:a=1[v][a]" \
  -map "[v]" -map "[a]" output.mp4

Advanced Filtering

Video Filters

Multiple filters (chain with comma):

ffmpeg -i input.mp4 -vf "scale=1280:720,hqdn3d" output.mp4

Denoise:

ffmpeg -i input.mp4 -vf hqdn3d output.mp4

Deinterlace:

ffmpeg -i input.mp4 -vf yadif output.mp4

Crop:

# Crop to 1280x720 from top-left (0,0)
ffmpeg -i input.mp4 -vf "crop=1280:720:0:0" output.mp4

# Auto-detect and remove black borders
ffmpeg -i input.mp4 -vf "cropdetect" output.mp4

Rotate:

# Rotate 90 degrees clockwise
ffmpeg -i input.mp4 -vf "transpose=1" output.mp4

# Rotate 180 degrees
ffmpeg -i input.mp4 -vf "transpose=1,transpose=1" output.mp4

Watermark/Logo:

ffmpeg -i video.mp4 -i logo.png \
  -filter_complex "overlay=10:10" output.mp4

Speed adjustment:

# 2x speed
ffmpeg -i input.mp4 -vf "setpts=0.5*PTS" -af "atempo=2.0" output.mp4

# 0.5x speed (slow motion)
ffmpeg -i input.mp4 -vf "setpts=2.0*PTS" -af "atempo=0.5" output.mp4

Audio Filters

Volume adjustment:

# Increase volume by 10dB
ffmpeg -i input.mp4 -af "volume=10dB" output.mp4

# Set to 50% volume
ffmpeg -i input.mp4 -af "volume=0.5" output.mp4

Normalize audio:

ffmpeg -i input.mp4 -af loudnorm output.mp4

Crossfade between audio:

ffmpeg -i audio1.mp3 -i audio2.mp3 \
  -filter_complex "[0][1]acrossfade=d=5" output.mp3

Mix multiple audio tracks:

ffmpeg -i input1.mp3 -i input2.mp3 \
  -filter_complex "[0:a][1:a]amix=inputs=2:duration=longest" output.mp3

Streaming

RTMP Streaming (Twitch/YouTube)

Basic stream:

ffmpeg -re -i input.mp4 -c:v libx264 -preset veryfast \
  -maxrate 3000k -bufsize 6000k -pix_fmt yuv420p -g 50 \
  -c:a aac -b:a 128k -ar 44100 \
  -f flv rtmp://live.twitch.tv/app/YOUR_STREAM_KEY

Screen capture + stream (Linux):

ffmpeg -f x11grab -s 1920x1080 -framerate 30 -i :0.0 \
  -f pulse -ac 2 -i default \
  -c:v libx264 -preset veryfast -tune zerolatency \
  -maxrate 2500k -bufsize 5000k -pix_fmt yuv420p \
  -c:a aac -b:a 128k -ar 44100 \
  -f flv rtmp://live.twitch.tv/app/YOUR_STREAM_KEY

Screen capture (Windows DirectShow):

ffmpeg -f dshow -i video="screen-capture-recorder":audio="Stereo Mix" \
  -c:v libx264 -preset ultrafast -tune zerolatency \
  -maxrate 750k -bufsize 3000k \
  -f flv rtmp://live.twitch.tv/app/YOUR_STREAM_KEY

Important streaming parameters:

• -re: Read input at native frame rate (real-time)
• -tune zerolatency: Optimize for low latency
• -g 50: Keyframe every 50 frames (2 seconds @ 25fps)
• -maxrate: Maximum bitrate
• -bufsize: Rate control buffer (typically 2x maxrate)

HLS Streaming

Generate adaptive HLS stream:

ffmpeg -i input.mp4 \
  -c:v libx264 -preset fast -crf 22 -g 48 -sc_threshold 0 \
  -c:a aac -b:a 128k \
  -f hls -hls_time 6 -hls_playlist_type vod \
  -hls_segment_filename "segment_%03d.ts" \
  playlist.m3u8

Multi-bitrate HLS:

ffmpeg -i input.mp4 \
  -map 0:v -map 0:a -map 0:v -map 0:a \
  -c:v libx264 -crf 22 -c:a aac -b:a 128k \
  -b:v:0 2000k -s:v:0 1280x720 -b:v:1 5000k -s:v:1 1920x1080 \
  -var_stream_map "v:0,a:0 v:1,a:1" \
  -master_pl_name master.m3u8 \
  -f hls -hls_time 6 -hls_list_size 0 \
  stream_%v/index.m3u8

UDP/RTP Streaming

UDP stream:

ffmpeg -re -i input.mp4 -c copy -f mpegts udp://192.168.1.100:1234

RTP audio stream:

ffmpeg -re -i audio.mp3 -c:a libopus -f rtp rtp://192.168.1.100:5004

Hardware Acceleration

NVIDIA NVENC

# H.264 with NVENC
ffmpeg -hwaccel cuda -i input.mp4 -c:v h264_nvenc -preset fast -crf 22 output.mp4

# H.265 with NVENC
ffmpeg -hwaccel cuda -i input.mp4 -c:v hevc_nvenc -preset fast -crf 24 output.mp4

Intel QuickSync (QSV)

ffmpeg -hwaccel qsv -c:v h264_qsv -i input.mp4 \
  -c:v h264_qsv -preset fast -global_quality 22 output.mp4

AMD VCE

ffmpeg -hwaccel auto -i input.mp4 \
  -c:v h264_amf -quality balanced -rc cqp -qp 22 output.mp4

Media Analysis

Inspect media file

# Detailed information
ffprobe -v quiet -print_format json -show_format -show_streams input.mp4

# Human-readable format
ffprobe input.mp4

# Get duration
ffprobe -v error -show_entries format=duration \
  -of default=noprint_wrappers=1:nokey=1 input.mp4

# Get resolution
ffprobe -v error -select_streams v:0 \
  -show_entries stream=width,height \
  -of csv=s=x:p=0 input.mp4

# Get bitrate
ffprobe -v error -show_entries format=bit_rate \
  -of default=noprint_wrappers=1:nokey=1 input.mp4

Batch Processing

Bash loop for batch conversion

# Convert all MKV files to MP4
for file in *.mkv; do
  ffmpeg -i "$file" -c:v libx264 -crf 22 -c:a aac "${file%.mkv}.mp4"
done

# Resize all videos to 720p
for file in *.mp4; do
  ffmpeg -i "$file" -vf scale=-1:720 "720p_${file}"
done

Parallel processing with GNU Parallel

# Install: sudo apt-get install parallel

# Process multiple files in parallel
ls *.mkv | parallel ffmpeg -i {} -c:v libx264 -crf 22 -c:a aac {.}.mp4

Common Patterns

Create GIF from video

# High quality GIF
ffmpeg -i input.mp4 -vf "fps=15,scale=640:-1:flags=lanczos,split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse" output.gif

# Simple GIF
ffmpeg -i input.mp4 -vf "fps=10,scale=320:-1" output.gif

Extract frames as images

# Extract all frames
ffmpeg -i input.mp4 frame_%04d.png

# Extract 1 frame per second
ffmpeg -i input.mp4 -vf fps=1 frame_%04d.png

# Extract single frame at 5 seconds
ffmpeg -ss 00:00:05 -i input.mp4 -frames:v 1 frame.png

Create video from images

# From numbered images
ffmpeg -framerate 30 -i frame_%04d.png -c:v libx264 -crf 22 -pix_fmt yuv420p output.mp4

# From pattern
ffmpeg -framerate 24 -pattern_type glob -i '*.png' -c:v libx264 output.mp4

Add subtitles

# Burn subtitles into video
ffmpeg -i video.mp4 -vf subtitles=subtitles.srt output.mp4

# Embed subtitle track (soft subtitles)
ffmpeg -i video.mp4 -i subtitles.srt -c copy -c:s mov_text output.mp4

Remove audio

ffmpeg -i input.mp4 -an -c:v copy output.mp4

Replace audio

ffmpeg -i video.mp4 -i audio.mp3 -c:v copy -c:a aac -map 0:v:0 -map 1:a:0 output.mp4

Create thumbnail

# Single thumbnail at 5 seconds
ffmpeg -i input.mp4 -ss 00:00:05 -vframes 1 -vf scale=320:-1 thumb.jpg

# Multiple thumbnails (one per minute)
ffmpeg -i input.mp4 -vf fps=1/60,scale=320:-1 thumb_%03d.jpg

Performance Optimization

Speed vs Quality Trade-offs

Fast encoding (real-time processing):

ffmpeg -i input.mp4 -c:v libx264 -preset ultrafast -crf 23 output.mp4

Balanced (default):

ffmpeg -i input.mp4 -c:v libx264 -preset medium -crf 22 output.mp4

High quality (archival):

ffmpeg -i input.mp4 -c:v libx264 -preset veryslow -crf 18 output.mp4

Multi-threading

FFmpeg automatically uses multiple CPU cores. Override with:

ffmpeg -threads 8 -i input.mp4 -c:v libx264 output.mp4

Best Practices

1. Use CRF for quality-focused encoding - Better than bitrate for most use cases
2. Two-pass for size targets - When file size is critical
3. Copy streams when possible - Use -c copy to avoid re-encoding
4. Choose appropriate presets - Balance speed vs compression
5. Match output to source - Don't upscale or use higher quality than source
6. Test short clips first - Verify settings before batch processing
7. Hardware acceleration - Use GPU encoding for faster processing
8. Audio matters - Don't neglect audio codec/bitrate choices
9. Use filters efficiently - Chain multiple filters with commas
10. Verify output - Always check quality after encoding

Common Parameters Reference

Video Parameters

• -c:v or -vcodec: Video codec
• -b:v: Video bitrate (e.g., 2M, 2500k)
• -crf: Constant Rate Factor (0-51)
• -preset: Encoding speed/quality (ultrafast to veryslow)
• -tune: Encoding optimization (film, animation, zerolatency)
• -pix_fmt: Pixel format (yuv420p for compatibility)
• -r: Frame rate (e.g., 30, 60)
• -g: GOP size (keyframe interval)

Audio Parameters

• -c:a or -acodec: Audio codec
• -b:a: Audio bitrate (e.g., 128k, 192k)
• -ar: Audio sample rate (e.g., 44100, 48000)
• -ac: Audio channels (1=mono, 2=stereo)
• -q:a: Audio quality for VBR (0=best for MP3)

General Parameters

• -i: Input file
• -y: Overwrite output without asking
• -n: Never overwrite output
• -t: Duration to encode
• -ss: Start time offset
• -to: End time
• -vf: Video filters
• -af: Audio filters
• -map: Stream selection
• -metadata: Set metadata
• -f: Force format

Troubleshooting

"Unknown encoder" errors

Install codec libraries:

# Ubuntu/Debian
sudo apt-get install ffmpeg libx264-dev libx265-dev libvpx-dev

# Check available encoders
ffmpeg -encoders

Output compatibility issues

Use safe defaults for maximum compatibility:

ffmpeg -i input.mkv -c:v libx264 -profile:v high -level 4.0 \
  -pix_fmt yuv420p -c:a aac -b:a 128k output.mp4

Performance issues

• Use hardware acceleration (-hwaccel)
• Choose faster presets (-preset fast)
• Reduce resolution with scale filter
• Use multiple threads explicitly

Quality issues

• Lower CRF value (18-22 for high quality)
• Use slower preset (-preset slow)
• Use two-pass encoding for better bitrate distribution
• Match or exceed source bitrate

Resources

• Official Documentation: https://ffmpeg.org/documentation.html
• FFmpeg Wiki: https://trac.ffmpeg.org/
• Codec Guides: https://trac.ffmpeg.org/wiki/Encode
• Filter Documentation: https://ffmpeg.org/ffmpeg-filters.html
• GitHub Repository: https://github.com/ffmpeg/ffmpeg
• Community Examples: https://trac.ffmpeg.org/wiki/

Implementation Checklist

When using FFmpeg:

• Verify FFmpeg installation (ffmpeg -version)
• Check available codecs for your use case
• Test encoding settings on short clip first
• Choose appropriate codec for target platform
• Set quality level (CRF or bitrate)
• Configure audio codec and bitrate
• Apply necessary filters (scale, crop, etc.)
• Consider hardware acceleration if available
• Verify output quality and file size
• Test output compatibility on target devices
• Document settings for reproducibility
• Create batch scripts for repetitive tasks