# Generate Report

> Synthesize experiment results into a structured research report in the research/ folder. Supports --inconclusive flag.

- Skill: `talont-org/generate-report` (Agent Skill)
- Install (CLI): `npx skillmds@latest add talont-org/generate-report`
- Raw SKILL.md: https://api.skillmd.com/api/skills/talont-org/generate-report/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Docs & Writing
- Author: TalonT-Org (https://skillmd.com/u/talont-org)
- Updated: 2026-09-10
- Page: https://skillmd.com/skills/talont-org/generate-report

---


# Write Report Skill

Synthesize scope findings, experiment design, and experiment results into a
structured research report. The report is committed to the `research/` directory
in the worktree and becomes the primary deliverable of the research recipe.

This skill handles both conclusive and inconclusive outcomes — inconclusive
results are valid findings, not failures.

## When to Use

- As the reporting step of the `research` recipe (phase 2)
- After `/autoskillit:run-experiment` has produced results (or after retry exhaustion)

## Arguments

```
/autoskillit:generate-report {worktree_path} {results_path} [--inconclusive]
[--output-mode {local|pr}] [--issue-url {url}]
[--experiment-type {type}] [--methodology-traditions {tradition}]
```

- `{worktree_path}` — Absolute path to the worktree (required). First path-like
  token after the skill name.
- `{results_path}` — Absolute path to the experiment results file (required).
  Second path-like token.
- `--inconclusive` — Optional flag indicating experiments were inconclusive
  (retry exhaustion or insufficient evidence). When present, the report
  emphasizes what was learned and why evidence was insufficient, rather than
  framing as a failure.
- `--output-mode {local|pr}` — Optional. When 'local', the report header must include an
  issue reference link if `--issue-url` is also provided.
- `--issue-url {url}` — Optional. GitHub issue URL. When output_mode=local and this is
  supplied, inject a blockquote reference at the top of report.md. In pr mode, skip —
  the PR body handles the issue link.
- `--experiment-type {type}` — Optional. Experiment type classification (e.g., "benchmark",
  "causal_inference"). When absent or "null", write `experiment_type: null` in frontmatter.
- `--methodology-traditions {tradition}` — Optional. Methodology tradition slug from Tier-C
  routing (e.g., "controlled_intervention"). When absent or "null", write
  `methodology_traditions: []` in frontmatter; otherwise wrap in a YAML list.
- `--design-review-verdict {verdict}` — Optional. GO/REVISE/STOP verdict from design review.
- `--disambiguation-rule-applied {rule}` — Optional. Disambiguation rule applied during Tier-C routing.
- `--tier-c-lens {lens}` — Optional. Tier-C vis-lens selected.
- `--classification-timestamp {timestamp}` — Optional. UTC timestamp of design classification.

## Inputs

In addition to the arguments above, this skill reads from the worktree:
- `${RESEARCH_DIR}/visualization-plan.md` — figure inventory and `yaml:figure-spec`
  blocks produced by `plan-visualization`. Read in Step 2.5 to drive plot generation.
- `${RESEARCH_DIR}/report-plan.md` — section outline mapping figure IDs to report
  sections. Read in Step 3 to place figure references correctly.

## Critical Constraints

**NEVER:**
- Modify source code files outside the `research/` directory
- Fabricate or embellish results — report exactly what was measured
- Omit the methodology section — reproducibility requires it
- Frame inconclusive results as failures — they are valid findings
- Create the report outside the worktree's `research/` directory
- Run subagents in the background (`run_in_background: true` is prohibited)

**ALWAYS:**
- Use `model: "sonnet"` when spawning all subagents via the Task tool
- Write the report to `research/` in the worktree root
- Include experiment scripts inline as fenced code blocks for reproducibility
- Commit the report to the worktree before returning
- Include a "What We Learned" section regardless of outcome
- Link back to the originating GitHub issue if an issue number is available

## Context Limit Behavior

When context is exhausted mid-execution, the report may be partially written
but not yet committed to the worktree. The recipe routes to `on_context_limit: test`,
preserving whatever was written.

**Before emitting structured output tokens:**
1. Run `git -C {worktree_path} status --porcelain`
2. If any files are dirty: `git -C {worktree_path} add -A && git -C {worktree_path} commit -m "chore: commit partial report before context limit"`
3. Only then emit the structured output tokens

This ensures the partial report is committed and the downstream test step can
evaluate whatever research output was produced.

## Workflow

### Step 1 — Gather All Artifacts

Read all available artifacts from the worktree:
1. Experiment plan: `{{AUTOSKILLIT_TEMP}}/experiment-plan.md`
2. Scope report: `{{AUTOSKILLIT_TEMP}}/scope/` (if available in worktree)
3. Experiment results: `{results_path}`
4. Any raw data files in `{{AUTOSKILLIT_TEMP}}/run-experiment/`
5. Standardized metrics: scan `{{AUTOSKILLIT_TEMP}}/run-experiment/` for
   `*_metrics.json` files (e.g., `accuracy_metrics.json`, `parity_metrics.json`).
   If present, read them — they will populate the Standardized Metrics Assessment
   section of the report.
6. Experiment code: scan the worktree for scripts, fixtures, and tools
   added during implementation

### Step 1.5 — Inject Issue Reference Header (local mode only)

Parse arguments for `--output-mode` and `--issue-url`.

If `--output-mode local` AND `--issue-url {url}` is present:
1. Parse the issue number from the URL (last numeric path segment).
2. Prepend this line to the **top** of `report.md`, before the title:
   ```
   > This research addresses [Issue #{N}]({url})

   ```
   (Include a blank line after the blockquote before the title.)

In pr mode: omit this step entirely — the PR body contains the issue link.

### Step 2 — Determine Report Type

Based on the `--inconclusive` flag and the experiment results status:

**Conclusive (no --inconclusive flag):**
- Full report with definitive findings
- Clear answer to the research question
- Recommendations based on evidence

**Inconclusive (--inconclusive flag or status = INCONCLUSIVE/FAILED):**
- Read `visualization-plan.md` if it exists.
- For each figure-spec: if `data_source.path` exists and has data, generate
  the plot (run Step 2.5 normally for that figure).
- If data is absent: emit the placeholder block instead of the figure:
  ```
  > **[Figure {id} not produced]** — experiment concluded inconclusively;
  > data required for this figure was not produced.
  ```
  Preserve the original `yaml:figure-spec` YAML block in the report for
  reproducibility, indented under a `<details>` collapsible.
- Emphasize what was learned despite lack of definitive answer
- Document boundary conditions established
- Clearly state what additional work would produce a conclusive result
- Distinguish between "negative result" (evidence against hypothesis) and
  "inconclusive" (insufficient evidence either way)

### Step 2.5 — Produce Visualizations

If `${RESEARCH_DIR}/visualization-plan.md` exists:

1. Read `visualization-plan.md`. If it contains zero figure specs (empty plan),
   omit all sub-steps and proceed to Step 3.

2. Identify the experiment's Docker image. The image tag is `research-{slug}` where
   `{slug}` is the experiment directory name. Verify the image exists:
   ```bash
   docker image inspect "research-{slug}" > /dev/null 2>&1 || \
       (cd "${RESEARCH_DIR}" && docker build --build-arg MAMBA_ENV={slug} -t "research-{slug}" .)
   ```

3. For each `yaml:figure-spec` block in `visualization-plan.md`:
   a. Write a Python plotting script to
      `${RESEARCH_DIR}/scripts/fig{N}_{slug}.py`
      that reads from `data_source.path` (or scans `results/` and `data/` if
      the path does not exist — treat `data_source.path` as a hint).
   b. Run the script inside the experiment container (volume-mount research dir):
      ```bash
      docker run --rm \
        -v "${RESEARCH_DIR}:/workspace" \
        "research-{slug}" \
        bash -c "pip install --quiet matplotlib seaborn plotly kaleido 2>/dev/null; \
                 python /workspace/scripts/fig${N}_${slug}.py"
      ```
   c. Confirm output exists at `${RESEARCH_DIR}/images/fig-${N}.{png,svg}`.
   d. On failure: emit `MISSING: fig-${N} — {error summary}` to stdout and
      continue with remaining figures. Do not abort the skill.

4. Commit scripts and images (if any were produced):
   ```bash
   git add research/ && git commit -m "Add visualization scripts and figures"
   ```

### Step 3 — Write Report

### Domain-Adaptive Section Ordering

The default section order below suits software engineering and computational
research. Adjust to match domain conventions when the research question
originates from a non-engineering field:

- **Default (engineering / computational):** Executive Summary → Background and
  Research Question → Methodology → Results → Observations → Analysis →
  What We Learned → Conclusions → Recommendations
- **Biology / biomedical:** Adapted from IMRaD — Background → Methodology →
  Results → Discussion and Future Directions → What We Learned. Use "Discussion
  and Future Directions" instead of "Recommendations". For journals that move
  Methods to supplementary material (e.g., Nature style), omit the Methodology
  section from the main body and note its location.
- **Economics / social science:** Background → Methodology → Results → Analysis →
  Discussion and Future Directions → What We Learned.

**Rules:**
- Executive Summary is always first; Appendices are always last. Exception: for
  academic journal submissions (biology, medical, social science) where Executive
  Summary is not a standard format, replace it with a structured Abstract at the
  document start.
- All mandatory sections (Data Scope Statement, Metrics Provenance Check, Gate
  Enforcement, What We Learned) must appear regardless of reordering.
- When reordering, only the section position changes — content requirements are
  unchanged.

Create the report directory and file:
```
research/YYYY-MM-DD-{slug}/
  report.md       # The main research report
  scripts/        # Extracted experiment scripts (optional, if complex)
```

The `{slug}` is a kebab-case summary of the research topic (max 40 chars).

<<<<<<< HEAD
=======
Write a YAML frontmatter block (fenced with `---`) at the very top of report.md,
before the title heading. Use the values from `--experiment-type` and
`--methodology-traditions` flags. If a flag is absent or its value is the literal
string `null`, write the key with value `null` (not omitted). Always include
`generated_at`. If `--output-mode local` with `--issue-url`, the issue blockquote
goes AFTER the frontmatter, before the title.

>>>>>>> b64b32f7 (fix(review): align --methodology-tradition synopsis to plural form)
The report structure:

```markdown
---
experiment_type: {value from --experiment-type, or null}
methodology_traditions:
  - {value from --methodology-traditions, or empty list if null}
disambiguation_rule_applied: {value from --disambiguation-rule-applied, or null}
tier_c_lens: {value from --tier-c-lens, or null}
design_review_verdict: {value from --design-review-verdict, or null}
classification_timestamp: {value from --classification-timestamp, or null}
audit_trail_path:
  design_review: research/{slug}/audit/design-review-dashboard.md
  visualization_trace: research/{slug}/audit/visualization-plan-trace.md
generated_at: {ISO 8601 timestamp}
---

# {Research Title}

> Research report for [Issue #{N}]({issue_url}) — {date}

## Executive Summary

### Data Scope Statement (mandatory — include at start of Executive Summary)

Every report must begin the Executive Summary with a Data Scope Statement:

> **Data Scope:** All benchmarks were conducted on {comma-separated list of data types
> used, e.g., "synthetic Gaussian blobs (10K–100K points)"}. {Domain target} data was
> {present and used | absent — all results derive from synthetic data | partial — only
> {subset} was available}.

**Rules:**
- If ALL benchmarks used ONLY synthetic data and the research task was domain-specific:
  state this explicitly. Do not claim domain-specific performance improvements derived
  from synthetic data without this qualifier.
- If some hypotheses were marked N/A or BLOCKED due to missing data: state which
  hypotheses were affected and why.
- Read the experiment plan's `data_manifest` (if available) to determine what data was
  planned vs. what was actually used.

{2-3 paragraph overview: what was investigated, key methodology, headline
finding, and recommendation. Written last, placed first.}

## Background and Research Question

{Context: why this investigation was initiated, what decision it informs,
what was known before this experiment.}

## Methodology

### Experimental Design
{From the experiment design: hypothesis, variables, controls. Include
enough detail for independent reproduction.}

### Environment
- **Repository commit:** {output of `git rev-parse HEAD` — the exact commit this experiment ran against}
- **Branch:** {current branch name}
- **Package versions:** {output of the project's package manager — e.g., `pip freeze`, `conda list`, or the contents of lock files (e.g., `requirements.txt`, `environment.yml`). Include ALL relevant dependency versions, not just top-level.}
- **Hardware/OS:** {if relevant to the experiment}
- **Custom environment:** {if a micromamba/conda environment.yml was used, note it and its location}

### Procedure
{Step-by-step description of what was executed.}

## Results

{Present data from the experiment. Use tables, code blocks, or whatever
format best represents the measurements. No interpretation in this
section — just facts.}

### Figure References

Reference figures by ID and caption only. NEVER embed images with `![](...)` syntax.
The HTML report rendered by `bundle-local-report` reads `yaml:figure-spec` metadata
and inserts `<img>` tags at the correct sections. Markdown prose uses:
> "Figure 1 shows ..."  or  "(see Figure 1)"

### Metrics Provenance Check (mandatory before including any metrics)

Before including data from any `*_metrics.json` file:

1. **Check generation timestamp**: The file's modification time must be within the
   current experiment's execution window. If the file predates the experiment run,
   it is stale.
2. **Check content relevance**: Verify the metrics file's contents relate to the
   hypotheses under test. If a metrics file contains data from a different subsystem
   or experiment, it is irrelevant.
3. **Disposition:**
   - **Current and relevant**: Include normally.
   - **Stale**: Disclose in the report: "Note: {filename} predates the current
     experiment run and was not regenerated. Excluded from analysis."
   - **Irrelevant**: Disclose: "Note: {filename} contains {description of actual
     contents} which is unrelated to the hypotheses under test. Excluded."
   - **NEVER** silently drop a metrics file. Always disclose the reason for exclusion.

### Standardized Metrics

{Include this section when `*_metrics.json` files are present in
`{{AUTOSKILLIT_TEMP}}/run-experiment/`. Omit entirely if no metrics JSON was produced.}

| Metric | Dimension | Dataset | Value | Threshold | Status |
|--------|-----------|---------|-------|-----------|--------|
| {metric_name} | {Accuracy/Parity} | {dataset} | {value} | {threshold} | ✅ PASS / ❌ FAIL |

{If any metrics failed: note which solver level or dataset showed the failure
and whether it is within acceptable range for the experiment's scope.}

## Observations

{Notable patterns, anomalies, unexpected behaviors discovered during
the experiment.}

### Gate Enforcement (mandatory for all hypothesis results)

When reporting on pre-specified hypothesis gates:

1. **Use the gate threshold from the experiment plan**, not a different hypothesis's
   threshold. Each hypothesis has its own pre-specified success criterion — do not
   conflate them.
2. **When a gate is NOT met**: State this as a failure. Example: "H6 targeted ≥3×
   speedup at n=100K. Measured: 2.04× at n=50K (estimated ~1.95× at n=100K). **FAIL.**"
3. **When recommending GO**: The GO recommendation must reference the specific gate(s)
   that were met and their measured values. A GO on H1 (which has a ≥1.5× threshold)
   does not satisfy H6 (which has a ≥3× threshold).
4. **NEVER** silently substitute one hypothesis's threshold for another. If H6's gate
   is not met, H6 is a FAIL regardless of whether H1's lower threshold was met by the
   same measurement.

## Analysis

{Interpret the results. Compare against the hypothesis. Explain anomalies.
Connect findings to the original research question. Include statistical
analysis if relevant to the experiment type.}

## What We Learned

{Regardless of outcome, document:}
- {Key insight 1}
- {Key insight 2}
- {Boundary conditions established}
- {Methodology learnings for future experiments}

## Conclusions

{Direct answer to the research question.}

## Design Review Summary

**Verdict:** {design_review_verdict}

The design review evaluated the experiment plan against {experiment_type} criteria.
The review confirmed the experimental design meets quality standards.

For detailed evaluation dimensions, scorecard, and adversarial findings, see
[`audit/design-review-dashboard.md`](audit/design-review-dashboard.md).

For visualization tier routing decisions and methodology tradition analysis, see
[`audit/visualization-plan-trace.md`](audit/visualization-plan-trace.md).

## Recommendations

*For biology and social science domains, use "Discussion and Future Directions" as the section title instead.*

{Actionable next steps based on findings — what to keep, revert, modify,
or investigate further. Include justification for each recommendation.
In non-engineering domains, frame this section as "Discussion and Future Directions":
interpret findings in the context of existing literature, explain implications,
and propose follow-on studies rather than prescriptive next steps.}

## Data Availability

{Optional — include when required by the target journal or domain conventions
(biology, medical, social science). State where datasets, code, and supplementary
materials are available, or explain any access restrictions (e.g., patient data,
proprietary datasets). Omit this section for internal engineering reports.}

## Appendix: Experiment Scripts

{Include key experiment scripts as fenced code blocks. These are preserved
for reproducibility even after the worktree is cleaned up.}

### {script_name.ext}
```{language}
{script content}
```

## Appendix: Visualization Scripts

{Enumerate each script in `${RESEARCH_DIR}/scripts/fig*.py` produced during
Step 2.5. Include the full script as a fenced Python code block. These are
preserved for figure reproducibility even after the worktree is cleaned up.}

## Appendix: Raw Data

{If raw data is small enough, include inline. Otherwise, reference the
files committed alongside this report.}
```

### Step 4 — Commit and Emit

1. Create the research directory in the worktree:
   `mkdir -p research/YYYY-MM-DD-{slug}/`
2. Write `report.md` to that directory.
3. If experiment scripts are complex (>50 lines), also save them as separate
   files in `research/YYYY-MM-DD-{slug}/scripts/`.
4. Commit to the worktree:
   ```
   git add research/
   git commit -m "Add research report: {brief title}"
   ```

After committing, emit the structured output token as the very last line of
your text output:

> **IMPORTANT:** Emit the structured output tokens as **literal plain text with no
> markdown formatting on the token names**. Do not wrap token names in `**bold**`,
> `*italic*`, or any other markdown. The adjudicator performs a regex match on the
> exact token name — decorators cause match failure.

```
report_path = {absolute_path_to_report.md}
```