Screen Recording

Screen Recording

Create animated GIF demos that show a feature or workflow in action — with annotations, variable timing, and proper pacing. Useful for PR descriptions, documentation, and release notes.

When to Use This Skill

Use this skill when you need to:

• Record a multi-step UI interaction as an animated GIF
• Create a demo showing before/after behavior
• Build annotated walkthroughs for documentation or release notes
• Show a bug reproduction or fix in action

Prerequisites

pip install playwright Pillow imageio numpy scipy mss -q
playwright install chromium

Core Workflow

1. Capture frames

Use Playwright to step through the interaction and capture each frame:

from playwright.async_api import async_playwright

async def record_frames(url, steps, width=1400, height=900):
    """
    steps: list of dicts with 'action' (async callable taking page)
           and 'name' (frame filename)
    """
    async with async_playwright() as p:
        browser = await p.chromium.launch()
        page = await browser.new_page(viewport={"width": width, "height": height})
        await page.goto(url, wait_until="networkidle")

        for step in steps:
            if step.get("action"):
                await step["action"](page)
                await page.wait_for_timeout(step.get("wait", 500))
            await page.screenshot(path=step["name"])

        await browser.close()

2. Assemble GIF with imageio

Use imageio, not PIL, for GIF writing — PIL's GIF encoder merges visually similar frames, which kills animations.

import imageio.v3 as iio
from PIL import Image
import numpy as np

frames = []
durations = []

for frame_path, duration_ms in frame_list:
    img = Image.open(frame_path)
    frames.append(np.array(img))
    durations.append(duration_ms)

iio.imwrite("demo.gif", frames, duration=durations, loop=0)

3. Variable frame timing

Uniform timing makes everything feel either too fast or too slow. Use variable durations:

4. Annotate frames

Apply annotations to specific frames using the image-annotations skill:

from PIL import Image, ImageDraw, ImageFont

def annotate_frame(frame_path, annotations, out_path):
    img = Image.open(frame_path)
    draw = ImageDraw.Draw(img)

    for ann in annotations:
        # Apply annotation (rect, arrow, label, etc.)
        pass

    img.save(out_path)

5. Fade-in annotations

For smooth annotation appearance:

def apply_fade(base_frame, annotation_layer, alpha):
    """Blend annotation onto frame at given alpha (0.0 to 1.0)"""
    blended = Image.blend(
        base_frame.convert("RGBA"),
        annotation_layer.convert("RGBA"),
        alpha
    )
    return blended.convert("RGB")

# 2-frame pop-in at 10fps: 50% then 100%
faded_frames = [
    apply_fade(base, annotations, 0.5),  # frame 1: half opacity
    apply_fade(base, annotations, 1.0),  # frame 2: full opacity
]

At 10fps, use 2 fade frames (0.2s total). At 30fps, use 3-4 frames. Easing curves look bad at low FPS — simple pop-in is snappier and more readable.

Build as a Script

The annotation logic gets complex for anything beyond trivial demos. Write a dedicated script (e.g., annotate_gif.py) with functions instead of inline code. You'll iterate on timing and placement.

Testing Animations

Always test in isolation first — don't rebuild the full demo to test a fade tweak:

# Small test GIF: 10 bare frames → fade frames → 15 hold frames
# Add a frame counter overlay for debugging:
draw.text((10, height - 30), f"F{i}/{total} a={alpha:.0%} FADE",
          fill="white", font=small_font)

Desktop Screen Recording (mss)

For recording desktop apps, terminals, or anything outside a browser. Uses mss for fast screen capture.

import mss
from PIL import Image
import time

def record_gif(output_path, region=None, duration=5, fps=8):
    """Record screen region to GIF. region = {left, top, width, height} or None for full screen."""
    with mss.mss() as sct:
        if region is None:
            region = sct.monitors[1]  # primary monitor

        frames = []
        t_end = time.time() + duration
        while time.time() < t_end:
            t0 = time.time()
            shot = sct.grab(region)
            frames.append(Image.frombytes('RGB', shot.size, shot.rgb))
            time.sleep(max(0, 1 / fps - (time.time() - t0)))

    frames[0].save(output_path, save_all=True, append_images=frames[1:],
                   duration=int(1000 / fps), loop=0, optimize=True)
    return len(frames)

record_gif('demo.gif', region={'left': 0, 'top': 0, 'width': 800, 'height': 500}, duration=3)

Tested: 3s at 8fps → 24 frames, ~31KB. Keep fps ≤ 10 for reasonable file sizes.

Note: PIL.save(save_all=True) works for simple recordings but merges visually similar frames. For annotated GIFs with fade effects, use imageio.v3.imwrite instead.

Combining with window capture

# Find window rect, then record it as a GIF
# Reuse find_window() from the ui-screenshots skill
import ctypes
from ctypes import c_int, Structure, byref, windll

class RECT(Structure):
    _fields_ = [('left', c_int), ('top', c_int), ('right', c_int), ('bottom', c_int)]

hwnd = find_window('My App')[0][0]
rect = RECT()
windll.user32.GetWindowRect(hwnd, byref(rect))
region = {'left': rect.left, 'top': rect.top,
          'width': rect.right - rect.left, 'height': rect.bottom - rect.top}
record_gif('app-demo.gif', region=region, duration=5, fps=8)

Diff-Based Cluster Detection

Programmatically find changed regions between frames to decide what to annotate:

import numpy as np
from scipy import ndimage

def find_changed_clusters(frame_a, frame_b, threshold=30, min_pixels=300, dilate=5):
    """Find bounding boxes of changed regions between two frames."""
    diff = np.abs(frame_b.astype(float) - frame_a.astype(float)).max(axis=2)
    mask = diff > threshold
    dilated = ndimage.binary_dilation(mask, iterations=dilate)
    labeled, n = ndimage.label(dilated)
    clusters = []
    for i in range(1, n + 1):
        ys, xs = np.where(labeled == i)
        if len(ys) < min_pixels:
            continue
        clusters.append((xs.min(), ys.min(), xs.max(), ys.max(), len(ys)))
    return sorted(clusters, key=lambda c: -c[4])  # largest first

Format Compatibility

GIF is the only universally supported animated format across VS Code preview, GitHub markdown, and browsers.

Guidelines

1. Type → pause → annotate — during fast action, show NO annotation. Pause first, then annotate
2. Hero message gets the biggest font — 64pt+ for the main takeaway, 38pt for details
3. GIF palette does NOT kill gradients — 20 distinct alpha steps survive 256-color palette
4. 10fps minimum for typing/interaction — lower looks stuttery
5. Build iteratively — get the frame sequence right first, add annotations second, tune timing last

Limitations

• GIF is limited to 256 colors per frame — fine for UI screenshots, may show banding on photographic content
• Large GIFs (50+ frames at high resolution) can be several MB — consider cropping to the relevant area
• No audio support in GIF — use MP4 for narrated demos (but lose VS Code preview support)

Source: github/awesome-copilot → skills/screen-recording/SKILL.md