# Webworld > Train autoregressive world models on 1M+ real web interactions for accurate browser state prediction. Enables agent training with 100× more data than prior approaches, achieving GPT-4o comparable performance with format flexibility and cross-domain generalization. - Skill: `adu2021/webworld` (Agent Skill) - Install (CLI): `npx skillmds@latest add adu2021/webworld` - Raw SKILL.md: https://api.skillmd.com/api/skills/adu2021/webworld/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - License: MIT - Author: adu2021 (https://skillmd.com/u/adu2021) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/adu2021/webworld --- # WebWorld: Large-Scale Web Agent Training Environment ## Problem Context Web agent training requires massive interaction data that's expensive or impossible to collect at scale. Existing web world models use sandbox or closed environments with limited realism. Data scarcity forces agents to overfit to specific websites. Prior approaches generated only 10K-100K trajectories. Standard models trained on such small datasets don't capture real web complexity, limiting agent performance and generalization. ## Core Concept WebWorld is a **large-scale autoregressive world model** trained on over 1 million real-world web interaction trajectories. Rather than generating synthetic data or using sandboxes, it learns from authentic web behavior patterns. The key innovation: collecting data through three complementary strategies (randomized crawling, autonomous exploration, task-oriented execution) creates diverse, realistic training distributions. The model predicts next browser state given instruction and interaction history, enabling agents to train entirely in simulation. Multiple format support (A11y Trees, HTML, XML, Markdown) and explicit reasoning injection through chain-of-thought enable strong generalization to code, GUI, and game environments. ## Architecture Overview - **Autoregressive State Predictor**: Predicts next DOM/browser state given action and history - **Three-Level Data Collection**: Randomized crawling (293K), autonomous exploration (38K), task-oriented (94K) - **100× Scale Increase**: 1M+ trajectories vs 10K in prior work - **Multiple Format Support**: A11y Trees, HTML, XML, Markdown for flexibility - **Explicit Reasoning**: Chain-of-thought synthesis on subset for improved reasoning - **Model Sizes**: 8B, 14B, 32B parameters - **Extended Simulation**: Supports 30+ steps per episode - **Cross-Domain Transfer**: Generalizes to code, GUI, game environments ## Implementation Three-level hierarchical data collection: ```python class HierarchicalDataCollection: """ Collect web interactions through three complementary strategies to create diverse, large-scale training dataset. """ def __init__(self): self.trajectories = [] self.stats = { 'randomized_crawling': 0, 'autonomous_exploration': 0, 'task_oriented': 0 } def level_1_randomized_crawling(self, corpus_websites, num_trajectories=293000): """ Level 1: Randomized crawling of websites from pre-training corpora. Generates natural, unguided interactions. Creates 293K trajectories. """ print("Level 1: Randomized Crawling") for traj_idx in range(num_trajectories): # Sample random website from corpus website = random.choice(corpus_websites) # Start fresh session browser = WebBrowser() browser.navigate(website) trajectory = { 'website': website, 'interactions': [], 'type': 'randomized_crawling' } # Perform random interactions for step in range(random.randint(5, 30)): # Random action: click, type, scroll, etc. action = sample_random_action(browser) # Record before-state, action, after-state before_state = browser.get_state() browser.perform_action(action) after_state = browser.get_state() interaction = { 'action': action, 'before_state': before_state, 'after_state': after_state } trajectory['interactions'].append(interaction) self.trajectories.append(trajectory) self.stats['randomized_crawling'] += 1 if (traj_idx + 1) % 10000 == 0: print(f"Collected {traj_idx + 1} randomized trajectories") def level_2_autonomous_exploration(self, websites, num_trajectories=38000): """ Level 2: Autonomous agents generate their own tasks and explore. Creates 38K trajectories with agent-driven interaction. """ print("Level 2: Autonomous Exploration") for traj_idx in range(num_trajectories): # Sample website website = random.choice(websites) # Initialize autonomous agent agent = AutonomousExplorer() # Agent generates own objective objective = agent.generate_objective() trajectory = { 'website': website, 'objective': objective, 'interactions': [], 'type': 'autonomous_exploration' } # Agent explores to achieve objective for step in range(random.randint(10, 50)): # Agent decides action based on current state browser_state = get_browser_state() action = agent.decide_action(browser_state, objective) # Record interaction before_state = browser_state perform_web_action(action) after_state = get_browser_state() interaction = { 'action': action, 'before_state': before_state, 'after_state': after_state } trajectory['interactions'].append(interaction) # Check if objective achieved if agent.is_objective_achieved(): trajectory['success'] = True break self.trajectories.append(trajectory) self.stats['autonomous_exploration'] += 1 if (traj_idx + 1) % 5000 == 0: print(f"Collected {traj_idx + 1} autonomous trajectories") def level_3_task_oriented_execution(self, task_corpus, num_trajectories=94000): """ Level 3: Synthetic web tasks with human-written instructions. Creates 94K trajectories with structured objectives. """ print("Level 3: Task-Oriented Execution") for traj_idx in range(num_trajectories): # Sample task from corpus task = random.choice(task_corpus) website = task['website'] instruction = task['instruction'] trajectory = { 'website': website, 'instruction': instruction, 'interactions': [], 'type': 'task_oriented', 'success': False } # Execute task for step in range(max_steps := 30): # Get current state before_state = get_browser_state() # Use instruction to decide action action = decide_task_action(before_state, instruction) # Perform action perform_web_action(action) after_state = get_browser_state() interaction = { 'action': action, 'before_state': before_state, 'after_state': after_state } trajectory['interactions'].append(interaction) # Check task completion if check_task_completion(after_state, task): trajectory['success'] = True break self.trajectories.append(trajectory) self.stats['task_oriented'] += 1 if (traj_idx + 1) % 10000 == 0: print(f"Collected {traj_idx + 1} task-oriented trajectories") def get_collection_stats(self): """Return collection statistics.""" return { 'total': sum(self.stats.values()), 'breakdown': self.stats, 'diversity_score': self.compute_diversity() } def compute_diversity(self): """Measure dataset diversity across sources.""" websites = set() for traj in self.trajectories: websites.add(traj.get('website')) return len(websites) / len(self.trajectories) ``` Autoregressive world model architecture: ```python class AutoregressiveWorldModel(nn.Module): """ Predict next browser state given current state and action. Trained on 1M+ real web interaction trajectories. """ def __init__(self, vocab_size, hidden_dim=2048, num_layers=24, model_size='14B'): super().__init__() self.hidden_dim = hidden_dim self.model_size = model_size # Transformer backbone for state prediction self.backbone = TransformerLM( vocab_size=vocab_size, hidden_dim=hidden_dim, num_layers=num_layers, num_heads=32 ) # Action encoder self.action_encoder = nn.Embedding(num_actions := 1000, hidden_dim) def predict_next_state(self, current_state, action, interaction_history, format='html'): """ Predict next browser state in specified format. Supports A11y Trees, HTML, XML, Markdown. """ # Tokenize current state state_tokens = tokenize_state(current_state, format) # Encode action action_embedding = self.action_encoder(action) # Append action to state sequence full_sequence = torch.cat([ state_tokens, action_embedding.unsqueeze(0) ], dim=0) # Generate next state tokens autoregressively with torch.no_grad(): logits = self.backbone(full_sequence) # Sample next state tokens next_state_tokens = sample_next_tokens( logits, temperature=0.7, max_tokens=512) # Decode tokens to state next_state = decode_state(next_state_tokens, format) return next_state def generate_trajectory(self, initial_state, instruction, max_steps=30, format='html'): """ Generate complete trajectory given initial state and instruction. """ trajectory = { 'initial_state': initial_state, 'instruction': instruction, 'interactions': [], 'states': [initial_state] } current_state = initial_state for step in range(max_steps): # Decide action based on instruction and current state action = decide_action(current_state, instruction) # Predict next state next_state = self.predict_next_state( current_state, action, trajectory['interactions'], format=format) trajectory['interactions'].append({ 'action': action, 'from_state': current_state, 'to_state': next_state }) trajectory['states'].append(next_state) current_state = next_state return trajectory ``` Chain-of-thought synthesis for reasoning: ```python class ReasoningInjection: """ Inject explicit reasoning through chain-of-thought synthesis. Improves reasoning capability without requiring massive annotation. """ def __init__(self, cot_generator, sample_size=1000): self.cot_gen = cot_generator self.sample_size = sample_size def inject_reasoning_into_dataset(self, trajectories): """ Add CoT reasoning to subset of trajectories. Improves reasoning without expensive full annotation. """ # Sample subset for CoT synthesis sample = random.sample(trajectories, self.sample_size) annotated_trajectories = [] for traj in sample: # Generate reasoning for trajectory state = traj['interactions'][0]['before_state'] action = traj['interactions'][0]['action'] # Generate CoT explanation reasoning = self.cot_gen.explain_action( state, action, traj.get('instruction', '')) # Add reasoning to trajectory annotated_traj = copy.deepcopy(traj) annotated_traj['reasoning'] = reasoning annotated_trajectories.append(annotated_traj) return annotated_trajectories ``` Multi-format state representation: ```python class MultiFormatStateRepresentation: """ Support multiple state formats for flexibility. Enables transfer to code, GUI, game environments. """ @staticmethod def state_to_a11y_tree(dom_tree): """Convert DOM to accessibility tree (high-level structure).""" a11y = { 'type': 'root', 'children': [], 'text': '' } # Traverse DOM and build accessibility structure return a11y @staticmethod def state_to_html(dom_tree): """Convert to HTML representation.""" return dom_to_html_string(dom_tree) @staticmethod def state_to_markdown(dom_tree): """Convert to readable Markdown.""" markdown = [] for element in traverse_dom(dom_tree): if element.tag == 'h1': markdown.append(f"# {element.text}") elif element.tag == 'button': markdown.append(f"[Button: {element.text}]") return '\n'.join(markdown) @staticmethod def state_to_xml(dom_tree): """Convert to XML representation.""" return dom_to_xml_string(dom_tree) ``` ## Practical Guidance **When to use**: - Training web agents at scale - Need diverse, realistic interaction data - Want agents that generalize across websites - Have limited real API/browser access for training **Data collection strategy**: 1. **Level 1 (Randomized)**: 60-70% of total - Fast to collect - Natural interaction patterns - Budget: minimal (automated) 2. **Level 2 (Autonomous)**: 10-15% of total - Agent-driven objectives - Diverse exploration strategies - Budget: moderate (agent overhead) 3. **Level 3 (Task-Oriented)**: 20-30% of total - Structured objectives - Higher quality signal - Budget: higher (task synthesis/validation) **Model sizing recommendations**: - 8B: Fast training/inference, good for simple tasks - 14B: Balanced (recommended for most applications) - 32B: Highest quality, slower inference **Format selection**: - A11y Trees: Fast processing, good structure extraction - HTML: Detailed, but verbose - Markdown: Concise, human-readable - XML: Structured, good for element relationships - Use multiple formats during training for robustness **Reasoning injection**: - Annotate 5-10% of training data with CoT reasoning - Focus on complex state transitions - Use GPT-4 or similar for high-quality CoT - Include in trajectory dataset for training **Expected performance**: - WebArena benchmark: +9.2% over baselines - GPT-4o comparable performance on many tasks - Strong generalization to code, GUI, games - 30+ step trajectories maintain coherence - 1M trajectories approach 50 billion tokens **Training considerations**: - Batch size: 256-512 (depending on hardware) - Learning rate: 1e-4 for fine-tuning, 1e-5 for continued pre-training - Warmup steps: 5,000-10,000 - Eval frequency: every 5,000 steps - Total training: 100-200 hours on 8xH100 **Agent fine-tuning on WebWorld**: 1. Initialize agent with language model weights 2. Fine-tune on WebWorld-generated trajectories 3. Use behavior cloning for 1,000-5,000 steps 4. Then fine-tune with RL on real tasks 5. Achieves competitive performance with 10× less real data ## Reference Large-scale autoregressive world models trained on authentic web interactions enable efficient agent training in simulation. By combining randomized, autonomous, and task-oriented data collection with multi-format representation and reasoning injection, WebWorld creates a generalizable training environment that captures real web complexity at scale.