Mobilei2v I2v Eval

mobilei2v-i2v-eval

MobileI2V: Fast and High-Resolution Image-to-Video on Mobile Devices — Zhang et al. (2025) (arXiv:2511.21475, 2025)

What this evaluates

Evaluates the visual quality and generation speed of image-to-video diffusion models optimized for mobile deployment. It probes the model's ability to generate temporally coherent 17-frame videos from a single reference image while maintaining high resolution and low latency on mobile hardware.

Datasets

• Unspecified (FVD benchmarks) — total ?; splits: (unstated)

Metrics

• FVDhum (primary) — range: other
  • Frechet Video Distance computed on human video clips. Lower is better. Calculated as the Fréchet distance between the multivariate Gaussian distributions of features extracted from a pretrained video classifier (typically I3D) for real and generated videos.
• FVDscen — range: other
  • Frechet Video Distance computed on scene video clips. Lower is better. Uses the same feature extraction and distance calculation as FVDhum but on a different subset of videos.

Input / output format

Input: A reference image and an optical flow/motion score condition.

Output: A 17-frame video at 1280×720 resolution.

Scoring recipe

def compute_fvd(real_videos, generated_videos, feature_extractor):
    real_features = feature_extractor(real_videos)
    gen_features = feature_extractor(generated_videos)
    mu_real, sigma_real = np.mean(real_features, axis=0), np.cov(real_features, rowvar=False)
    mu_gen, sigma_gen = np.mean(gen_features, axis=0), np.cov(gen_features, rowvar=False)
    diff = mu_real - mu_gen
    covmean, _ = scipy.linalg.sqrtm(sigma_real.dot(sigma_gen), disp=False)
    if np.iscomplexobj(covmean):
        covmean = covmean.real
    fvd = np.sum(diff**2) + np.trace(sigma_real + sigma_gen - 2.0 * covmean)
    return fvd

Common pitfalls

• FVD scores are highly sensitive to the batch size used for feature extraction and the specific version of the I3D classifier weights.
• Mobile latency measurements can vary significantly due to thermal throttling and background OS processes on the iPhone 16 Pro.
• The table reports FVD for 17-frame videos, but standard FVD benchmarks often use different frame counts; ensure consistent temporal sampling.

Evidence (verbatim from paper)

Table 1: Speed and quality comparison of different models performing 17-frame I2V tasks. | Model | Type | Steps | Params | Resolution | LatencyA100 | LatencyMobile | FVDhum↓ | FVDscen↓ |

Citation

@misc{zhang2025mobilei2v,
  title={MobileI2V: Fast and High-Resolution Image-to-Video on Mobile Devices},
  author={Zhang et al. (2025)},
  year={2025},
  note={arXiv:2511.21475}
}

• arXiv: 2511.21475