Video Outpainting Eval

video-outpainting-eval

Seen-to-Scene: Keep the Seen, Generate the Unseen for Video Outpainting — Inseok Jeon et al. (2026) (arXiv:2604.14648, 2026)

What this evaluates

Evaluates a model's ability to generate spatially and temporally consistent video content outside the original frame boundaries (video outpainting), while preserving source structure and visual realism.

Datasets

• DAVIS 2017 — total 90; splits: test (90)
• YouTube-VOS — total 60; splits: test (60)

Metrics

• FVD (primary) — range: other
  • Fréchet Video Distance measures the distributional distance between generated and original video feature embeddings in a pre-trained feature space. Lower values indicate better perceptual realism and temporal consistency.
• PSNR — range: dB
  • Peak Signal-to-Noise Ratio computed in dB between corresponding pixels of the generated and ground truth frames. Higher is better.
• SSIM — range: [0, 1]
  • Structural Similarity Index measuring luminance, contrast, and structure similarity between generated and ground truth frames. Ranges from 0 to 1, higher is better.
• LPIPS — range: [0, 1]
  • Learned Perceptual Image Patch Similarity using deep network features to measure perceptual distance. Ranges from 0 to 1, lower is better.

Input / output format

Input: Source video frames with horizontal masks applied at ratios of 0.25 and 0.66, resized to 256×256 resolution.

Output: Completed 256×256 video frames covering the full outpainted region.

Scoring recipe

def compute_metrics(predictions, ground_truth):
    # predictions and ground_truth are aligned video tensors (B, T, H, W, C)
    psnr = compute_psnr(predictions, ground_truth)
    ssim = compute_ssim(predictions, ground_truth)
    lpips = compute_lpips(predictions, ground_truth)
    fvd = compute_fvd(predictions, ground_truth) # FVD is principal
    return {'PSNR': psnr, 'SSIM': ssim, 'LPIPS': lpips, 'FVD': fvd}

Common pitfalls

• YouTube-VOS lacks a standardized evaluation protocol; prior works often use selected high-quality subsets causing selection bias. The paper mandates randomly sampling 60 videos from the official test set.
• Flow completion networks pre-trained for video inpainting suffer from a domain gap when applied to outpainting, leading to geometric distortions. Fine-tuning on outpainting data is required for reliable results.
• Evaluation must strictly use horizontal outpainting with mask ratios of exactly 0.25 and 0.66 to align with prior benchmarks.

Evidence (verbatim from paper)

We evaluate our method using standard quantitative metrics, including PSNR, SSIM, LPIPS, and FVD. Fréchet Video Distance (FVD) serves as the principal metric for video outpainting, as it captures both perceptual realism and temporal consistency by measuring the distributional distance between the generated and original videos. All evaluation metrics are computed following the same protocol as prior works. For DAVIS 2017, we test on 90 video sequences. ... we randomly sample 60 videos from the official test set. Following the standard protocol, we report horizontal outpainting results under mask ratios of 0.25 and 0.66 for all test videos.

Citation

@misc{jeon2026seentoscene,
  title={Seen-to-Scene: Keep the Seen, Generate the Unseen for Video Outpainting},
  author={Inseok Jeon et al. (2026)},
  year={2026},
  note={arXiv:2604.14648}
}

• arXiv: 2604.14648