aradotso Qwen-AgentWorld Language World Model
Skill by ara.so — AI Agent Skills collection.
What It Does
Qwen-AgentWorld is a native language world model that simulates agentic environments across seven unified domains: MCP (tool-calling), Search, Terminal, SWE (software engineering), Android, Web, and OS. Unlike traditional approaches that adapt language models post-hoc, Qwen-AgentWorld is trained from Continued Pre-Training (CPT) onward with environment modeling as the core objective.
Key Capabilities:
- Unified multi-domain simulation: Single model covers 7 agent interaction environments
- Controllable simulation: Inject perturbations, create fictional worlds, adapt environments
- Agent foundation model: Sim RL warm-up transfers to multi-turn, tool-calling agentic tasks
- Zero-shot generalization: Out-of-distribution environment handling (e.g., Claw Agent)
- Evaluation benchmark: AgentWorldBench with 5-dimensional rubric scoring
Model Variants:
Qwen-AgentWorld-35B-A3B(35B total, 3B active MoE, 256K context)Qwen-AgentWorld-397B-A17B(397B total, 17B active MoE, 256K context)
Installation
Requirements
# Core dependencies
pip install torch transformers accelerate
pip install openai # For API-based inference and evaluation
# For deployment (choose one)
pip install "sglang[all]" # SGLang (recommended)
# OR
pip install vllm # vLLM
# For evaluation
pip install huggingface_hub
Download Model & Benchmark
# Download model weights
huggingface-cli download Qwen/Qwen-AgentWorld-35B-A3B --local-dir ./models/Qwen-AgentWorld-35B-A3B
# Download evaluation benchmark
huggingface-cli download Qwen/AgentWorldBench --repo-type dataset --local-dir ./AgentWorldBench
# Alternative: Use ModelScope in China
export SGLANG_USE_MODELSCOPE=true
export VLLM_USE_MODELSCOPE=true
Deployment
SGLang (Recommended)
python -m sglang.launch_server \
--model-path Qwen/Qwen-AgentWorld-35B-A3B \
--port 8000 \
--tensor-parallel-size 4 \
--context-length 262144 \
--reasoning-parser qwen3
OpenAI-compatible API available at http://localhost:8000/v1
vLLM
vllm serve Qwen/Qwen-AgentWorld-35B-A3B \
--port 8000 \
--tensor-parallel-size 4 \
--max-model-len 262144 \
--reasoning-parser qwen3 \
--language-model-only \
--trust-remote-code
Note: --language-model-only is required because the model architecture includes visual component definitions but only language weights are included.
Core Usage Patterns
1. Basic Inference with Transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "Qwen/Qwen-AgentWorld-35B-A3B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype="auto",
device_map="auto",
)
# Terminal domain example
messages = [
{
"role": "system",
"content": "You are a language world model simulating a Linux terminal environment. "
"Given the user's command, predict the terminal output."
},
{
"role": "user",
"content": "Action: execute_bash\nCommand: ls -la /home/user/project/"
}
]
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=2048, temperature=0.6)
response = tokenizer.decode(
outputs[0][inputs.input_ids.shape[-1]:],
skip_special_tokens=True
)
print(response)
2. API-Based Inference
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:8000/v1",
api_key="EMPTY"
)
# MCP (tool-calling) domain example
response = client.chat.completions.create(
model="Qwen/Qwen-AgentWorld-35B-A3B",
messages=[
{
"role": "system",
"content": "You are a Tool World Model simulating MCP server responses. "
"Given tool calls, predict realistic environment observations."
},
{
"role": "user",
"content": """### Turn 1
**Action:**
```json
{
"tool_name": "read_file",
"arguments": {"path": "/data/config.json"}
}
```"""
}
],
temperature=0.6,
max_tokens=2048
)
print(response.choices[0].message.content)
3. Domain-Specific System Prompts
The repository includes templates in prompts/{domain}/system_prompt.txt. Each benchmark sample carries its own system prompt in the system_str field.
# Load domain-specific system prompt template
with open("prompts/terminal/system_prompt.txt") as f:
terminal_system_prompt = f.read()
messages = [
{"role": "system", "content": terminal_system_prompt},
{"role": "user", "content": "Action: execute_bash\nCommand: cat /etc/os-release"}
]
4. Multi-Turn Trajectory Simulation
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
trajectory = []
system_prompt = "You are a language world model simulating a web browser environment."
for turn in range(5):
# Agent action
action = f"Action: click_element\nSelector: button#{turn}\n"
# Get environment observation from world model
messages = [{"role": "system", "content": system_prompt}] + trajectory + [
{"role": "user", "content": action}
]
response = client.chat.completions.create(
model="Qwen/Qwen-AgentWorld-35B-A3B",
messages=messages,
temperature=0.6,
max_tokens=2048
)
observation = response.choices[0].message.content
trajectory.append({"role": "user", "content": action})
trajectory.append({"role": "assistant", "content": observation})
print(f"\n=== Turn {turn+1} ===")
print(f"Action: {action}")
print(f"Observation: {observation}")
Evaluation on AgentWorldBench
Data Format
Each domain has a JSONL file (mcp_test.jsonl, terminal_test.jsonl, etc.):
{
"task": "terminal",
"id": 145256090131919,
"prompt": ["### Turn 1\n**Action:**\n...", "### Turn 2\n**Action:**\n..."],
"response": ["**Environment Observation:**\n...", "**Environment Observation:**\n..."],
"current_prompt": "### Turn 1\n**Action:**\nexecute_bash\nCommand: ls",
"system_str": "You are a language world model simulating a Linux terminal...",
"turn_idx": 1,
"total_turns": 5
}
Run Evaluation Pipeline
cd eval
# Step 1: Generate predictions from world model
python eval.py \
--data_file ../AgentWorldBench/terminal_test.jsonl \
--base_url http://localhost:8000/v1 \
--model Qwen/Qwen-AgentWorld-35B-A3B \
--output_file results/terminal_predictions.jsonl \
--max_workers 16 \
--temperature 0.6 \
--max_tokens 8192
# Step 2: Score predictions with LLM judge
python eval.py \
--data_file results/terminal_predictions.jsonl \
--base_url http://localhost:8000/v1 \
--model Qwen/Qwen-AgentWorld-35B-A3B \
--output_file results/terminal_scores.jsonl \
--max_workers 16 \
--temperature 0.0 \
--max_tokens 512 \
--judge
# Step 3: Compute metrics
python eval.py \
--data_file results/terminal_scores.jsonl \
--aggregate
Evaluation Dimensions
AgentWorldBench scores predictions on 5 dimensions:
- Format: Structural correctness (JSON schema, markdown formatting)
- Factuality: Technical accuracy of simulated behavior
- Consistency: Coherence with previous trajectory turns
- Realism: Plausibility vs. real environment responses
- Quality: Overall utility for agent training
Scores are 0-100 per dimension, aggregated as mean.
Advanced: Controllable Simulation
Environment Adaptation (MCP Domain)
Inject perturbations to expose agent weaknesses:
control_instruction = """
Simulate an environment where:
- 30% of file operations return "Permission denied"
- Network requests have 200ms random latency
- Database queries occasionally timeout
"""
messages = [
{
"role": "system",
"content": f"{mcp_system_prompt}\n\n{control_instruction}"
},
{
"role": "user",
"content": '{"tool_name": "write_file", "arguments": {"path": "/tmp/test.txt", "content": "data"}}'
}
]
Fictional-World Construction (Search Domain)
Train agents in fully invented, self-consistent worlds:
fictional_world = """
Simulate a search engine for a fictional universe where:
- Physics constants differ (speed of light = 500,000 km/s)
- Historical events are alternative (Rome never fell)
- Technology evolved differently (steam-powered computers)
Maintain internal consistency across all search results.
"""
messages = [
{
"role": "system",
"content": f"{search_system_prompt}\n\n{fictional_world}"
},
{
"role": "user",
"content": "Query: history of computing in the Roman Empire"
}
]
Sim RL for Agent Training
Basic Sim RL Setup
import json
from openai import OpenAI
world_model = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
agent_model = OpenAI(base_url="http://localhost:8001/v1", api_key="EMPTY")
# Generate synthetic trajectories
for episode in range(1000):
trajectory = []
state = "Initial terminal state"
for step in range(10):
# Agent generates action
action_response = agent_model.chat.completions.create(
model="agent-model-name",
messages=[
{"role": "system", "content": "You are a Linux terminal agent."},
{"role": "user", "content": f"Current state: {state}\nChoose an action."}
]
)
action = action_response.choices[0].message.content
# World model simulates next state
next_state_response = world_model.chat.completions.create(
model="Qwen/Qwen-AgentWorld-35B-A3B",
messages=[
{"role": "system", "content": "You are a terminal world model."},
{"role": "user", "content": f"Action: {action}"}
]
)
next_state = next_state_response.choices[0].message.content
trajectory.append({"action": action, "state": next_state})
state = next_state
# Save trajectory for RL training
with open(f"trajectories/episode_{episode}.json", "w") as f:
json.dump(trajectory, f)
Out-of-Distribution Scaling (Claw Agent Example)
# Generate 4k synthetic Claw environments
python scripts/generate_claw_environments.py \
--world_model http://localhost:8000/v1 \
--num_environments 4000 \
--output_dir claw_sim_data/
# Train agent with PPO on synthetic data
python scripts/train_ppo.py \
--data_dir claw_sim_data/ \
--agent_model Qwen/Qwen3.5-35B-A3B \
--epochs 3
# Evaluate on real Claw-Eval benchmark
python scripts/eval_claw.py \
--model trained_agent/ \
--benchmark claw_eval
Configuration
World Model Parameters
# Temperature: Controls randomness in environment simulation
# - 0.0: Deterministic (for evaluation)
# - 0.6-0.8: Realistic variance (for training)
temperature = 0.6
# Max tokens: Depends on domain complexity
# - Terminal/MCP: 2048-4096
# - SWE/Web/OS: 8192-16384
# - Search: 4096
max_tokens = 8192
# Context length: Use full 262K for long trajectories
context_length = 262144
Tensor Parallelism
# For 35B-A3B model:
# - 1x A100 80GB: --tensor-parallel-size 1
# - 2x A100 80GB: --tensor-parallel-size 2
# - 4x A100 40GB: --tensor-parallel-size 4
# For 397B-A17B model:
# - 8x A100 80GB: --tensor-parallel-size 8
Common Patterns
Pattern 1: Domain-Specific Evaluation Loop
import json
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
def evaluate_domain(domain_file, model_name):
results = []
with open(domain_file) as f:
for line in f:
sample = json.loads(line)
response = client.chat.completions.create(
model=model_name,
messages=[
{"role": "system", "content": sample["system_str"]},
{"role": "user", "content": sample["current_prompt"]}
],
temperature=0.6,
max_tokens=8192
)
results.append({
"id": sample["id"],
"prediction": response.choices[0].message.content,
"ground_truth": sample["response"][sample["turn_idx"]-1]
})
return results
# Run on all 7 domains
domains = ["mcp", "search", "terminal", "swe", "android", "web", "os"]
for domain in domains:
results = evaluate_domain(f"AgentWorldBench/{domain}_test.jsonl", "Qwen/Qwen-AgentWorld-35B-A3B")
with open(f"results/{domain}_results.json", "w") as f:
json.dump(results, f)
Pattern 2: Batch Inference with Concurrent Requests
from concurrent.futures import ThreadPoolExecutor
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
def predict_single(sample):
response = client.chat.completions.create(
model="Qwen/Qwen-AgentWorld-35B-A3B",
messages=[
{"role": "system", "content": sample["system_str"]},
{"role": "user", "content": sample["current_prompt"]}
],
temperature=0.6,
max_tokens=8192
)
return response.choices[0].message.content
# Process 100 samples with 16 concurrent workers
with ThreadPoolExecutor(max_workers=16) as executor:
predictions = list(executor.map(predict_single, samples))
Pattern 3: Custom Rubric Scoring
def score_prediction(prediction, ground_truth, judge_client):
judge_prompt = f"""
Rate the predicted environment observation on 5 dimensions (0-100):
**Ground Truth:**
{ground_truth}
**Prediction:**
{prediction}
Provide scores in JSON format:
{{"format": 85, "factuality": 90, "consistency": 80, "realism": 88, "quality": 87}}
"""
response = judge_client.chat.completions.create(
model="Qwen/Qwen-AgentWorld-35B-A3B",
messages=[
{"role": "system", "content": "You are an expert judge evaluating world model predictions."},
{"role": "user", "content": judge_prompt}
],
temperature=0.0,
max_tokens=512
)
return json.loads(response.choices[0].message.content)
Troubleshooting
Out-of-Memory Errors
# Reduce context length
--max-model-len 131072 # 128K instead of 256K
# Increase tensor parallelism
--tensor-parallel-size 8
# Enable quantization (vLLM)
--quantization awq
Slow Inference
# SGLang: Enable RadixAttention caching
--enable-radix-cache
# vLLM: Adjust max concurrent requests
--max-num-seqs 128
# Use smaller batch sizes for long contexts
--max-num-batched-tokens 32768
Model Loading Fails
# Ensure --language-model-only flag (vLLM)
# The model config includes vision components but weights are language-only
# For HuggingFace Hub access issues in China
export HF_ENDPOINT=https://hf-mirror.com
# OR use ModelScope
export VLLM_USE_MODELSCOPE=true
Evaluation Script Hangs
# Add timeout to OpenAI client
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:8000/v1",
api_key="EMPTY",
timeout=60.0 # 60 second timeout
)
JSON Parsing Errors in Judge Scores
import re
import json
def extract_json_from_response(text):
# Try direct parse
try:
return json.loads(text)
except:
pass
# Extract JSON from markdown code blocks
match = re.search(r'```json\s*(\{.*?\})\s*```', text, re.DOTALL)
if match:
return json.loads(match.group(1))
# Extract first JSON object
match = re.search(r'\{[^}]*"format"[^}]*\}', text)
if match:
return json.loads(match.group(0))
raise ValueError(f"No valid JSON found in: {text}")
Environment Variables
# ModelScope (China alternative to HuggingFace)
export SGLANG_USE_MODELSCOPE=true
export VLLM_USE_MODELSCOPE=true
# HuggingFace mirror
export HF_ENDPOINT=https://hf-mirror.com
# CUDA optimization
export CUDA_VISIBLE_DEVICES=0,1,2,3
# Logging
export SGLANG_LOG_LEVEL=INFO
export VLLM_LOGGING_LEVEL=INFO
Performance Benchmarks
AgentWorldBench Overall Scores (5-dimension mean):
- Qwen-AgentWorld-397B-A17B: 58.71 (SOTA)
- GPT-5.4: 58.25
- Claude Opus 4.8: 56.59
- Qwen-AgentWorld-35B-A3B: 56.39 (+8.66 vs. base Qwen3.5-35B-A3B)
Sim RL Transfer (35B model on out-of-domain tasks):
- Terminal-Bench 2.0: 39.55 (+6.30)
- SWE-Bench Verified: 67.86 (+3.39)
- Claw-Eval: 64.88 (+11.28)
- BFCL v4: 71.25 (+8.96)
Resources
- Technical Report: arXiv:2606.24597
- Blog Post: Qwen Blog
- HuggingFace Collection: Qwen/qwen-agentworld
- ModelScope (China): Qwen-AgentWorld
- GitHub Issues: Report bugs
- Discussions: Community forum