Worldcache Video Model Caching

Resource Definition

Four-module content-aware caching system for video diffusion model inference:

1. Causal Feature Caching (CFC): Motion-aware skip decisions
2. Saliency-Weighted Drift (SWD): Perceptual importance prioritization
3. Optimal Feature Approximation (OFA): Intelligent feature blending
4. Adaptive Threshold Scheduling (ATS): Time-dependent reuse aggressiveness

Efficiency Dimension

Primary optimization target: Wall-clock latency of generative step in video/image diffusion models.

Secondary targets: Memory bandwidth, GPU utilization during cached-feature reuse.

Trade-off axis: Reuse aggressiveness (throughput gain) vs. perceptual quality preservation.

Performance-Efficiency Tradeoff

Speedup vs. Quality

The framework achieves its dramatic speedup by investing in decision quality early, then "spending" the quality margin aggressively in later denoising phases where updates become small refinements.

Cosmos-Predict2.5 Results:

Competing Methods:

• DiCache: Only 1.3–1.4× speedup with visible quality degradation
• FasterCache: 1.6–1.7× speedup with noticeable artifacts

Overall Quality: PAI-Bench measurements confirm ~99.4% quality retention across diverse evaluation scenarios.

Four-Module Architecture

1. Causal Feature Caching (CFC)

Adapts skip thresholds based on input motion magnitude, preventing aggressive reuse during fast dynamics.

def causal_feature_caching(feature, motion_magnitude, base_threshold=0.02):
    """
    Motion-aware skip decision: reuse features when motion is small.
    During fast dynamics (high motion), enforce stricter thresholds.
    """
    # Estimate motion from optical flow or frame differencing
    motion_scale = compute_motion_magnitude(feature)

    # Scale threshold inversely with motion: high motion → stricter threshold
    adaptive_threshold = base_threshold / (1.0 + motion_scale)

    # Skip reuse decision
    should_reuse = motion_scale < adaptive_threshold
    return feature if should_reuse else recompute(feature)

2. Saliency-Weighted Drift (SWD)

Prioritizes perceptually important regions so errors on salient objects matter more than background noise.

def saliency_weighted_drift(feature, cached_feature, saliency_map):
    """
    Compute drift penalty weighted by perceptual importance.
    Salient regions incur high cost for mismatch; background errors are tolerated.
    """
    drift = feature - cached_feature
    weighted_loss = (drift ** 2) * saliency_map

    return weighted_loss.mean()

3. Optimal Feature Approximation (OFA)

Uses least-squares blending and motion-compensated warping instead of verbatim copying.

def optimal_feature_approximation(feature, cached_feature, motion_field):
    """
    Blend cached and current features via least-squares optimization.
    Compensate for motion-induced misalignment via optical flow warping.
    """
    # Motion-compensated warp of cached feature
    warped_cached = warp_features(cached_feature, motion_field)

    # Least-squares blend: find optimal α
    alpha = solve_lstsq(feature, warped_cached)
    approximated = alpha * feature + (1 - alpha) * warped_cached

    return approximated

4. Adaptive Threshold Scheduling (ATS)

Relaxes thresholds during late denoising refinement while maintaining strict control during structure formation.

def adaptive_threshold_scheduling(timestep, total_steps, base_threshold=0.02):
    """
    Threshold schedule: strict early (structure formation), relaxed late (refinement).
    Early: timestep/total_steps >> 0 (high noise)
    Late: timestep/total_steps << 0 (low noise, small updates)
    """
    progress = 1.0 - (timestep / total_steps)  # 0 at start, 1 at end

    # Early steps strict, late steps aggressive
    scheduled_threshold = base_threshold * (1.0 + 2.0 * progress)

    return scheduled_threshold

Practical Integration

When to Apply

• Generative video or image diffusion models
• Real-time or latency-sensitive inference scenarios
• Multi-frame prediction or frame interpolation tasks
• Hardware with bandwidth constraints (inference on edge devices)

Integration Steps

1. Instrument codebase: Extract motion magnitude and saliency per timestep
2. Profile baseline: Measure per-step latency and memory bandwidth on your hardware
3. Deploy CFC module: Start with motion-aware skip decisions; tune base_threshold for your model
4. Add SWD: Compute saliency map (via gradient or CLIP) and weight drift penalties
5. Implement OFA: Replace feature copying with least-squares blending + motion warping
6. Schedule thresholds: Adjust aggressiveness across denoising steps
7. Validate quality: Ensure metrics match 99.4%+ retention; adjust parameters if lower

Tuning Parameters

• base_threshold: Start at 0.02; lower for stricter reuse (quality-focused), higher for aggressive speedup
• motion_scale_factor: Control how motion magnitude scales threshold; typically 1.0–2.0
• saliency_weighting: Emphasize important regions; adjust based on task (faces more salient than backgrounds)
• late_phase_aggressiveness: Late denoising steps tolerate more approximation; typically 2×–4× base threshold