Abot Physworld Physics Aligned World Model

What This Skill Does

Replace standard diffusion model training objectives with a decoupled discriminator architecture and DPO (Direct Preference Optimization) post-training. This eliminates physically implausible behaviors like object interpenetration and anti-gravity motion while maintaining visual quality in robotic manipulation video generation.

The Component Swap

The old approach uses a single likelihood-based diffusion loss that prioritizes pixel-level reconstruction without explicit physics constraints:

# Old: standard diffusion training
# Loss = prediction_error(video_pred, video_real)
# No explicit physics modeling

The new ABot-PhysWorld approach decouples physics supervision from visual supervision using separate discriminator heads:

# New: decoupled discriminators for physics and visual quality
# Physics discriminator: classifies trajectory plausibility
physics_discriminator = DiscriminatorHead(
    input_dim=latent_dim,
    output_dim=1,  # Binary: physically plausible or not
    detects=['interpenetration', 'anti_gravity', 'contact_violations']
)

# Visual discriminator: classifies visual quality
visual_discriminator = DiscriminatorHead(
    input_dim=latent_dim,
    output_dim=1  # Binary: visually realistic or not
)

# DPO loss combines both signals
loss_dpo = dpo_loss(
    preferred=model_output_physics_plausible,
    rejected=model_output_physics_implausible,
    beta=0.1  # Preference strength
)

Post-training with DPO uses curated preference pairs to reinforce physically plausible trajectories while downweighting implausible ones. Parallel context blocks inject spatial action information without modifying core diffusion parameters:

# Parallel context injection for robotic action conditioning
action_embedding = embed(action_tokens)  # Action history
context_block = ParallelContextAttention(
    context=action_embedding,
    video_features=diffusion_features
)
# Applied in parallel to main diffusion UNet, not sequentially

Performance Impact

Physics plausibility: Achieves state-of-the-art results on EZSbench and PBench benchmarks, surpassing Veo 3.1 and Sora v2 Pro in physical realism and trajectory consistency.

Visual-physics trade-off: Maintains visual quality while improving physics plausibility (specific metrics not disclosed, but qualitative evaluation shows no degradation).

Scale: 14B parameter Diffusion Transformer enables fine-grained physics modeling at video-level.

Robotic performance: Enables zero-shot transfer to unseen robot-task-scene combinations through physically grounded representations.

When to Use

• Generating robotic manipulation videos where physical plausibility is critical
• Tasks requiring cross-embodiment generalization (different robot morphologies)
• Video diffusion models where standard training produces physically unrealistic outputs
• Scenarios with large curated datasets of physics-annotated manipulation clips

When NOT to Use

• Artistic or stylized video generation where physics realism is secondary
• Datasets without physics-aware annotations or labels
• Smaller models (<1B parameters) where discriminator overhead is prohibitive
• Scenarios where computational cost of DPO post-training is unacceptable

Implementation Checklist

To adopt this component swap:

1. Prepare physics-annotated dataset:

   # Requires trajectories labeled with physics plausibility
   # At minimum: {plausible, implausible} binary labels
   # Ideally: distractor videos with specific violations
   dataset = PhysicsManipulationDataset(
       videos=load_videos(),
       physics_labels={'interpenetration': False, 'gravity_plausible': True}
   )
   

2. Build decoupled discriminators:

   • Physics discriminator: detects common violations (object overlap, unsupported motion)
   • Visual discriminator: standard adversarial quality scoring
   • Ensure separate parameter spaces (no weight sharing)

3. Implement parallel context attention:

   • Don't modify core UNet; add parallel branches for action conditioning
   • Inject at multiple resolution levels for hierarchical control

4. Apply DPO post-training:

   # After standard diffusion pre-training
   preferred_outputs = model(video_real, action_real)
   rejected_outputs = model(video_fake, action_real)
   loss_dpo = -torch.log(sigmoid(beta * (score(preferred) - score(rejected))))
   

5. Verify physics quality:

   • Test on EZSbench (zero-shot evaluation on unseen scene-task-robot combinations)
   • Manual inspection of trajectories for interpenetration, gravity violations
   • Compare to Sora v2 Pro and Veo 3.1 baselines

6. Hyperparameter tuning:

   • beta (DPO preference strength): 0.05-0.2 (higher = stricter physics enforcement)
   • Physics discriminator loss weight: 0.1-1.0
   • Visual discriminator loss weight: 1.0 (reference)

Related Work

This builds on preference optimization methods (DPO, IPO) applied to video generation and relates to reward-guided diffusion approaches. Physics constraints in video generation have been explored, but decoupled discriminators represent a novel way to balance multiple objectives.