Agentsociety Analysis

Analysis

Use this skill for interactive experiment analysis. You (Claude Code) own orchestration, scientific judgment, and user dialogue. The harness CLI enforces structural reproducibility and phase gates; it does not replace your reasoning.

Quality before gates: Read references/analysis-quality.md at the start of each phase. A validate-* PASS with weak science is a failure — improve claims, evidence, and prose first, then attest.

Harness vs LLM (read first)

After each phase: (1) do the analytical work with the user, (2) run validate-<phase>, (3) run record-attestation with rubric from references/phase-attestation.md, (4) gate-status or advance.

After release or synthesis, run the experience loop from references/experience-memory.md when the user wants learning: draft-reflection → ask user feedback / record-feedback → review-reflection → promote-reflection. Promoted memory is active by default in intake, status, and run-loop; inspect it with memory-context. Do not promote user preferences unless the user explicitly confirms them.

References: references/harness-contract.md, references/phase-attestation.md, references/analysis-quality.md, references/experience-memory.md, references/json-payloads.md.

JSON payloads: --payload accepts a JSON object or a .json file path. The CLI repairs minor syntax issues via json-repair before Pydantic validation — still fix wrong fields, not escaping.

Overview

Analyze experiment results from AgentSociety simulations. Explore data interactively, produce targeted visualizations or composite figures, and write bilingual reports with optional cross-experiment synthesis.

The plotting and figure-assembly parts of this skill follow a contract-first structure adapted to the agentsociety2 analysis stack: fixed Python backend, replay-backed evidence tracking, ags.py analysis tool surface, and report asset management under presentation/ and synthesis/.

When to Use

• An experiment run has completed and either run/replay/_schema.json or legacy run/sqlite.db exists in the run directory.
• The user asks to analyze results, explore data, visualize data, combine multiple assets into one figure, create charts, or write analysis reports.
• The user references a hypothesis or experiment ID and wants to understand outcomes.

Do NOT use when:

• The experiment run has not completed (no run/replay/_schema.json).
• The user only wants to configure or launch experiments (use run-experiment instead).

Quick Reference

Use the Python interpreter from .env. The analysis and plotting environment must be installed from the SDK's declared analysis extra (uv sync --extra analysis in the SDK project); do not treat an ad hoc Matplotlib install as a reproducible workspace setup. See CLAUDE.md and references/eda.md for setup and capability checks.

All mutating analysis commands return outcome.status and write atomic receipts under .agentsociety/analysis/runs/. Treat exit code 2 as a fixable phase/gate/business block and exit code 1 as execution failure. Inspect one receipt with analysis status --workspace . --run-id RUN_ID; never infer success from a nested gate status alone. Equivalent mutating calls are single-flight across processes. A duplicate successful non-repeatable call returns UNCHANGED; an overlapping call returns BLOCKED with reason=operation_already_running. Use run-loop's recommended_operations, completed_operations, and in_flight_runs instead of blindly repeating commands.

First Move: Analysis Contract Before Plotting

Before generating any chart code or combining existing assets, establish the contract below:

1. Core finding: what specific claim should the next visual defend?
2. Evidence source: which table, query, aggregation, or prior chart supports it?
3. Figure scope: is the artifact a single chart or composite figure?
4. Figure archetype: choose the simplest structure that carries the message.
5. Output contract: decide filenames, export bundle, and where the artifact will appear in the report.

The highest-priority rule is: each visual artifact must strengthen an already identified analytical finding. Generic EDA should stay in Stage 2 unless it is promoted into Stage 3 by an explicit finding.

Common Mistakes

Workflow

digraph analysis_flow {
    rankdir=LR;
    node [shape=box, style=filled, fillcolor="#E8F4FD"];
    frame [label="1 frame\nplan"];
    explore [label="2 explore\nrun-eda"];
    claims [label="3 claims"];
    refine [label="4 refine\ncharts"];
    produce [label="5 produce\nreport"];
    synth [label="6 synthesis\nrequired"];

    frame -> explore -> claims -> refine -> produce -> synth;
}

Pipeline Position

Predecessors: run-experiment (completed run with run/replay/_schema.json) Optional inputs: web-research (supplementary context for interpretation), use-dataset (external datasets for comparison) Successors: paper-toolkit Also feeds: hypothesis (refinement cycle when analysis informs hypothesis revision)

Stage Notes

• stages/01_frame.md: intake, write-plan, validate-plan
• stages/02_data_explore.md: narrow exploration + run-eda + validate-explore
• stages/03_claims.md: record-claim, validate-claims
• stages/04_refine.md: figure contracts, chart generation, validate-chart
• stages/05_produce.md: bilingual report, validate-release
• stages/06_synthesis.md: required workspace synthesis, validate-synthesis

After Stage 5 or Stage 6, use references/experience-memory.md when the user wants the system to learn project lessons, reusable analysis methods, or durable preferences.

Shared References

• Figure contract: references/figure-contract.md
• Python backend rule: references/backend-selection.md
• Chart selection and chart families: references/charts.md
• Plot API and palettes: references/api.md
• Plot layout patterns: references/common-patterns.md
• Composite figure assembly: references/composite-figures.md
• Chart QA and export checks: references/harness.md#qa-contract
• Analysis methods: references/analysis-methods.md
• Worked examples: references/tutorials.md
• Demo routing: references/demos.md
• Output layout: references/harness-contract.md
• Report self-check: checklists/quality.md
• Harness contract: references/harness-contract.md
• Phase attestation rubrics: references/phase-attestation.md
• Quality standard (LLM-owned): references/analysis-quality.md
• Experience memory and preferences: references/experience-memory.md
• JSON templates: references/json-payloads.md
• EDA → report integration: references/report-integration.md
• Independent review: references/report-review.md
• Embeddings (figures, tables, EDA): references/report-embeddings.md
• Interactive EDA in HTML reports: references/eda.md
• HTML design cues and bundled support: references/reports.md, references/integrations.md; polish via support/frontend-design/ + support/frontend-design/references/analysis-reports.md
• Simulation report template: references/report-template-simulation.md
• Progressive assembly and LLM-authored HTML: references/reports.md, assets/report-shell.reference.html

CLI Tool

Use $PYTHON_PATH .agentsociety/bin/ags.py analysis for all mechanical operations:

• load-context, data-summary, list-tables, query-data, run-eda, compose-figure, collect-assets

Subagent Delegation

Subagents return file paths and key_findings; they do not call advance, record-attestation, or pipeline update-stage.

Orchestrator-only: frame (user intent), claims approval with user, final attestation wording.

Produce flow:

1. Orchestrator: prepare-produce after refine gate passes to build report context.
2. Dispatch report-producer with paths to report_context.md, evidence_index.json, claims, charts/assets.
3. Re-run prepare-produce after report asset references are final; this is the explicit write/sync step.
4. Dispatch report-reviewer (independent subagent). On REVISE/FAIL, loop producer → reviewer until PASS.
5. record-report-review then the presentation-read-only validate-release (it records gate state but never rewrites report artifacts).
6. Orchestrator record-attestation only after review PASS and user-aligned narrative.

HTML (if requested) is written inside report-producer as native HTML using assets/report-shell.reference.html — never mechanical Markdown conversion.

Runtime Contract

• Start with Stage 1 (intake + write-plan) for every new analysis request.
• Do not mark analysis complete until validate-synthesis passes.
• Do not skip directly to charting before the relevant tables and questions are clear.
• Use .agentsociety/bin/ags.py analysis ... instead of inventing parallel helper scripts.
• Keep intermediate reasoning in the conversation, not inside generated helper code.
• Stage 6 synthesis is required; write synthesis_brief.json plus bilingual synthesis reports.
• Produce stage: write report_outline.json (sections + figure captions) — harness does not grep report headings.
• Use the Python toolchain exclusively for analysis plotting and figure assembly in this skill.

Hard Constraints

• No default chart cap (max_charts: 0 in harness state = unlimited). Set max_charts in state.yaml only when the user wants a hard budget.
• Prefer fewer strong charts over many weak ones — quality bar, not a fixed count.
• Every chart plan starts from a finding contract before code generation.
• Every chart that appears in the report must have a one-line description directly below it.
• When one finding requires multiple coordinated views or pre-rendered assets, assemble them into a single figure_{nn}_{description}.png with panel labels instead of pasting separate screenshots into the report.
• Run EDA only for tables explicitly selected during Stage 2.
• Write single-experiment output to presentation/hypothesis_{id}/ (data/, charts/, assets/ only — no analysis/, figures/, or eda/ folders there).
• Harness state goes to .agentsociety/analysis/hypothesis_{id}/ (created by intake).
• Write synthesis output only to the dedicated synthesis/ root directory, never under presentation/.
• Reports are bilingual, Chinese-first by default. Required for gate pass: report_zh.md, report_en.md, report_zh.html, and report_en.html under presentation/hypothesis_{id}/. Embed images only via assets/; run prepare-produce, sync-report-assets, or collect-assets before the presentation-read-only validate-release.
• Generated matplotlib charts must use the Agg backend, English-only legend text, and a publication-safe rcParams block that keeps SVG text editable.
• Final charts and reports must pass references/harness.md#qa-contract before being described as complete.
• Scripts are mechanical helpers only; Claude Code remains the orchestrator.

Progress Tracking

Only after validate-synthesis passes:

$PYTHON_PATH .agentsociety/bin/ags.py research-pipeline update-stage analysis completed