# Swe World Docker Free Training > Replace Docker environments with learned LLM surrogates comprising a lightweight sandbox for deterministic file operations, a transition model predicting execution feedback, and a reward model acting as virtual test runner. Reduces training infrastructure costs while scaling SWE agent training data. - Skill: `adu2021/swe-world-docker-free-training` (Agent Skill) - Install (CLI): `npx skillmds@latest add adu2021/swe-world-docker-free-training` - Raw SKILL.md: https://api.skillmd.com/api/skills/adu2021/swe-world-docker-free-training/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: DevOps & Infra - License: MIT - Author: adu2021 (https://skillmd.com/u/adu2021) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/adu2021/swe-world-docker-free-training --- # SWE-World: LLM-Based Surrogate Training Environments Training software engineering agents typically requires expensive infrastructure to run code in isolated Docker containers. SWE-World replaces this with learned surrogates that simulate execution feedback, dramatically reducing infrastructure overhead while enabling larger-scale training data. The key insight is that execution can be decomposed: deterministic file operations (navigation, editing) can be handled directly, while non-deterministic execution effects (stdout, exit codes, test results) are predicted by lightweight models. This hybrid approach maintains fidelity while eliminating Docker overhead. ## Core Concept SWE-World comprises three components working in tandem: 1. **Lightweight Sandbox**: Handles all deterministic state transitions (file system operations) directly without invoking external processes 2. **Transition Model (SWT)**: Predicts step-level execution output—stdout, stderr, exit codes—based on file state and command 3. **Reward Model (SWR)**: Acts as a virtual test runner, generating test reports and binary success signals This design preserves task semantics while replacing expensive runtime dependencies with efficient learned components. ## Architecture Overview - **Deterministic Sandbox**: Fast, stateless file operation handling (read, write, delete, navigate) with in-memory state - **SWT (Transition Model)**: Small LLM fine-tuned to predict execution trace outputs given command and file context - **SWR (Reward Model)**: Outputs test success/failure signals, trained on real test execution traces - **Training Loop**: Agents generate sequences of commands; sandbox updates file state; models predict execution effects - **Data Scaling**: Larger training sets possible since models are faster than real Docker execution ## Implementation ### Step 1: Build the Deterministic Sandbox Create a lightweight virtual file system that tracks state without external process invocation. ```python # Lightweight deterministic sandbox class DeterministicSandbox: def __init__(self): self.file_system = {} self.current_dir = "/" self.env_vars = {} def read_file(self, path: str) -> str: """Read file from virtual filesystem.""" full_path = self._resolve_path(path) if full_path not in self.file_system: return f"Error: File not found: {path}" return self.file_system[full_path] def write_file(self, path: str, content: str) -> str: """Write to virtual filesystem.""" full_path = self._resolve_path(path) self.file_system[full_path] = content return f"Wrote {len(content)} bytes to {path}" def list_files(self, path: str = ".") -> str: """List directory contents.""" full_path = self._resolve_path(path) files = [f for f in self.file_system.keys() if f.startswith(full_path)] return "\n".join(files) def _resolve_path(self, path: str) -> str: """Resolve relative path to absolute.""" if path.startswith("/"): return path return f"{self.current_dir}/{path}".replace("//", "/") ``` ### Step 2: Train the Transition Model (SWT) Fine-tune an LLM to predict execution outputs given command and file context. ```python # Transition model training def prepare_swe_training_data(execution_traces: List[dict]) -> List[dict]: """Convert execution traces to SWT training examples.""" examples = [] for trace in execution_traces: for step in trace['steps']: example = { 'input': f""" Command: {step['command']} Current directory: {step['cwd']} File context: {format_file_context(step['files'])} Environment: {step['env']} """, 'target': step['output'] # stdout/stderr } examples.append(example) return examples def train_transition_model(traces: List[dict], model_name: str = 'gpt-3.5-turbo'): """Fine-tune transition model on execution traces.""" train_data = prepare_swe_training_data(traces) # Create training dataset training_examples = [ f"Input: {ex['input']}\nOutput: {ex['target']}" for ex in train_data ] # Fine-tune on lightweight model # (Using API-based fine-tuning for illustration) response = openai.FineTuningJob.create( training_file=upload_to_openai(training_examples), model=model_name, n_epochs=3, batch_size=32 ) return response.fine_tuned_model ``` ### Step 3: Train the Reward Model (SWR) Create a model that predicts test success/failure based on execution state. ```python # Reward model training def prepare_reward_training_data(test_results: List[dict]) -> List[dict]: """Convert test results to SWR training examples.""" examples = [] for result in test_results: example = { 'input': f""" Test command: {result['command']} Exit code: {result['exit_code']} Output: {result['output']} Error: {result['error']} """, 'label': 1 if result['passed'] else 0 } examples.append(example) return examples def train_reward_model(test_results: List[dict]): """Train reward model on test execution outcomes.""" train_data = prepare_reward_training_data(test_results) # Prepare binary classification dataset texts = [ex['input'] for ex in train_data] labels = [ex['label'] for ex in train_data] # Use lightweight classifier fine-tuning model = AutoModelForSequenceClassification.from_pretrained( 'bert-base-uncased', num_labels=2 ) trainer = Trainer( model=model, args=TrainingArguments( output_dir='./reward_model', num_train_epochs=3, per_device_train_batch_size=32 ), train_dataset=Dataset.from_dict({ 'text': texts, 'labels': labels }) ) trainer.train() return trainer.model ``` ### Step 4: Integrate Sandbox and Models into Agent Training Loop Combine the sandbox and learned models to simulate execution during training. ```python # Agent training with surrogate environment def train_swe_agent_with_surrogate( agent_model: str, sandbox: DeterministicSandbox, transition_model, reward_model, dataset: List[dict] ): """Train SWE agent using surrogate environment.""" for episode in range(num_episodes): task = dataset[episode % len(dataset)] sandbox.reset() agent_trajectory = [] done = False step_count = 0 while not done and step_count < max_steps: # Get agent's next action state_repr = format_state(sandbox.file_system, sandbox.current_dir) action = agent_model.get_next_action( task_description=task['description'], current_state=state_repr, previous_actions=agent_trajectory ) # Execute in sandbox (deterministic file ops) file_effect = sandbox.execute_file_operation(action) # Predict execution output using SWT predicted_output = transition_model.predict( command=action, file_context=format_file_context(sandbox.file_system), cwd=sandbox.current_dir ) # Check if task is complete using SWR task_success = reward_model.predict( output=predicted_output, command=action, task=task['description'] ) agent_trajectory.append({ 'action': action, 'predicted_output': predicted_output, 'success': task_success }) if task_success > 0.5: # Threshold for success done = True step_count += 1 # Compute episode reward and update agent episode_reward = 1.0 if done else -0.1 * step_count agent_model.update( trajectory=agent_trajectory, reward=episode_reward ) return agent_model ``` ## Practical Guidance **When to use Docker-free surrogate training:** - Large-scale SWE agent training with 10K+ task instances - Development/prototyping where Docker overhead slows iteration - Scenarios where exact execution fidelity is less critical than scalability - Cost-constrained environments lacking containerization infrastructure **When to use real Docker:** - Production evaluation requiring exact execution behavior - Tasks with complex external dependencies (databases, services) - Security-critical scenarios requiring true isolation - High-consequence deployments where fidelity is paramount **Common Pitfalls:** - Train-inference mismatch: SWT/SWR trained on limited domains may hallucinate on novel code patterns - Context window saturation: Large file contents can exceed model input limits; use summary representations - Missing failure modes: Surrogates may not capture rare error conditions; validate on real environment periodically - Stale model assumptions: Periodically retrain models as agent behavior and command distributions drift **Hyperparameter Guidelines:** - SWT training data: Use 5K-10K execution traces; beyond 10K, diminishing returns - Context window budget: Reserve 1K-2K tokens for file content; summarize larger files - Reward threshold: Use 0.5-0.7 for binary success classification - Validation interval: Test agent trajectories on real Docker every 500 episodes to catch distribution shift ## Reference See the full paper at: https://arxiv.org/abs/2602.03419 Key results: Qwen2.5-Coder improved from 6.2% to 52.0% resolve rate via Docker-free SFT, reaching 68.2% with test-time scaling. Public code released; training scaled from thousands to 16.6K task instances.