tsinghua-fib-lab Create Dataset
Overview
Create, validate, package, and upload datasets to the AgentSociety platform. Produces a ZIP package conforming to the platform's schema for sharing or admin review.
Use the Python interpreter from .env. See CLAUDE.md for setup.
When to Use
- User wants to publish or share a dataset on the AgentSociety platform
- Packaging survey results, agent profiles, or simulation outputs for upload
- User says "create dataset", "upload dataset", or "publish data"
Do NOT use when:
- User wants to download or browse datasets (use
agentsociety-use-dataset) - User is working with experiment config files only (use
experiment-config)
Quick Reference
| Command | Auth | Description |
|---|---|---|
login |
No | Casdoor Device Code Flow authentication |
logout |
No | Clear saved credentials |
init <name> |
No | Create dataset directory structure |
validate <path> |
No | Validate dataset dir or ZIP |
pack <dir> |
No | Package directory into ZIP |
upload <zip> |
Yes | Upload to AgentSociety platform |
submit <dataset_id> |
Yes | Submit for admin review |
Run commands from the workspace root through .agentsociety/bin/ags.py.
Workflow
digraph create_dataset_flow {
rankdir=LR;
node [shape=box, style=filled, fillcolor="#E8F4FD"];
login [label="login\nif upload needed"];
init [label="init dataset dir"];
add [label="add data files"];
meta [label="edit README.md\nand dataset.json"];
validate [label="validate"];
pack [label="pack zip"];
upload [label="upload"];
submit [label="submit for review"];
login -> init -> add -> meta -> validate -> pack -> upload -> submit;
}
Dataset Format
A valid dataset is a directory containing:
<name>/
├── README.md # Required. Describes the dataset.
├── dataset.json # Required. Metadata (id, name, description, category, etc.)
└── data/ # Recommended. Dataset files (CSV, JSON, etc.)
Constraints:
idmust match^[a-z0-9_-]+$and be unique on the platformcategory: one ofagent_profiles,surveys,experiments,literature,simulation_results,other- Total package size: max 5TB (files ≥200MB use multipart direct-to-OSS upload)
For the full dataset.json schema, see references/dataset-schema.md.
Common Mistakes
| Mistake | Fix |
|---|---|
Using invalid slug format in id |
Must match ^[a-z0-9_-]+$ (lowercase, digits, hyphens, underscores) |
| Exceeding 5TB package size | Split into multiple datasets or compress data files |
| Missing README.md or dataset.json | Both are required at the package root |
| Not running validate before pack | Always validate then pack to catch errors early |
| Skipping login before upload/submit | Run login first; credentials persist in ~/.agentsociety/ |
Pipeline Position
Predecessors: None (standalone)
Successors: experiment-config (when external data needed), agentsociety-use-dataset (for download after upload)