Reporting And Archiving Findings

Reporting and Archiving Findings

Overview

Complete an investigation by confirming reproducibility, presenting clear options, handling the chosen one, and archiving everything needed to reproduce the result.

Core principle: Confirm reproducibility -> separate confirmatory from exploratory -> present options -> execute choice -> archive code + data + environment + pre-registration.

Announce at start: "I'm using the reporting-and-archiving-findings skill to complete this work."

Step 1: Verify Reproducibility

Before reporting anything, confirm the whole analysis reproduces from immutable raw data with the fixed seed in the pinned environment.

# From a clean state: re-run the pipeline end to end
# Confirm headline numbers match what you intend to report

Use science-superpowers:verifying-results-before-claiming. If it doesn't reproduce, stop — fix reproducibility (possibly via science-superpowers:investigating-anomalous-results) before reporting. Don't report a number you can't regenerate.

For any confirmatory claim, also run the pre-registration audit and keep its output for the report:

# ships with science-superpowers:preregistering-analysis
<skills root>/preregistering-analysis/prereg.sh audit

If the audit fails, the confirmatory label is indefensible — the registration changed after its freeze, or outputs predate it. Relabel the affected analyses exploratory (or pre-register a fresh test on unused data) before reporting.

Step 2: Detect Environment

GIT_DIR=$(cd "$(git rev-parse --git-dir)" 2>/dev/null && pwd -P)
GIT_COMMON=$(cd "$(git rev-parse --git-common-dir)" 2>/dev/null && pwd -P)
BRANCH=$(git branch --show-current)

• GIT_DIR == GIT_COMMON: normal repo, no worktree cleanup needed
• GIT_DIR != GIT_COMMON, named branch: worktree, provenance-based cleanup (Step 5)
• Detached HEAD: reduced menu (no local merge), no cleanup (externally managed)

Step 3: Determine Base Branch

git merge-base HEAD main 2>/dev/null || git merge-base HEAD master 2>/dev/null

Or ask: "This branch split from main — correct?"

Step 4: Present Options

Normal repo / named-branch worktree — present exactly these 4 options:

Analysis complete and reproducible. What would you like to do?

1. Merge the analysis back to <base-branch> locally
2. Write up and share (report / preprint / pull request)
3. Keep the branch as-is (I'll handle it later)
4. Discard this work

Which option?

Detached HEAD — present these 3 (no local merge):

Analysis complete and reproducible (externally managed workspace).

1. Push as a new branch and open a pull request / share
2. Keep as-is
3. Discard this work

Which option?

Don't add explanation — keep options concise.

Step 5: Execute Choice

Option 1: Merge Locally

MAIN_ROOT=$(git -C "$(git rev-parse --git-common-dir)/.." rev-parse --show-toplevel)
cd "$MAIN_ROOT"
git checkout <base-branch> && git pull && git merge <feature-branch>
# Re-run the pipeline on the merged result; confirm it still reproduces

Then cleanup worktree (Step 6), then git branch -d <feature-branch>.

Option 2: Write Up and Share

Produce the report (see "Report Content" below). If sharing via PR:

git push -u origin <feature-branch>
gh pr create --title "<title>" --body "$(cat <<'EOF'
## Question
<the research question>

## What was done
<the pre-registered analysis, and any documented deviations>

## Findings
<confirmatory results with effect sizes + intervals>

## Exploratory (not confirmatory)
<clearly separated leads>

## Reproducibility
- Pre-registration: <path/commit>
- Environment: <lockfile>
- Seed: <value>
- Re-run: <command>
EOF
)"

Do NOT clean up the worktree — it's needed for iteration on feedback.

Option 3: Keep As-Is

Report: "Keeping branch . Worktree preserved at ." No cleanup.

Option 4: Discard

Confirm first:

This will permanently delete:
- Branch <name>
- All commits: <list>
- Worktree at <path>

Type 'discard' to confirm.

Wait for the exact word. Then cleanup worktree (Step 6) and git branch -D <feature-branch>.

Step 6: Cleanup Workspace

Only for Options 1 and 4. Options 2 and 3 preserve the worktree.

WORKTREE_PATH=$(git rev-parse --show-toplevel)

• GIT_DIR == GIT_COMMON: normal repo, nothing to clean up.
• Worktree under .worktrees/, worktrees/, or ~/.config/superpowers/worktrees/: we own it.

  MAIN_ROOT=$(git -C "$(git rev-parse --git-common-dir)/.." rev-parse --show-toplevel)
  cd "$MAIN_ROOT"
  git worktree remove "$WORKTREE_PATH"
  git worktree prune
  

• Otherwise: the harness owns it. Don't remove it.

Report Content

Every report MUST:

• Separate confirmatory from exploratory. Confirmatory results are those that match the frozen pre-registration, run as registered. Everything else goes under an "Exploratory" heading and is described as a lead, not a conclusion.
• Report effect sizes and intervals, not just p-values. Describe null results as inconclusive vs. evidence of absence honestly.
• Document every deviation from the pre-registration and note that the affected analysis is exploratory.
• State limitations and threats to validity carried from the design.
• Avoid over-claiming — no causal language from observational data; no generalizing past the sample.

Archive for Reproducibility

Whatever the option, ensure the archive contains everything needed to regenerate the result:

• The code (committed)
• The environment lockfile and runtime version
• The fixed seed
• The frozen pre-registration (commit reference) and the passing prereg.sh audit output
• Data provenance + checksums for raw inputs (and the data itself, or instructions to obtain it, per any data-sharing constraints)
• The exact command(s) to re-run

Red Flags

Never:

• Report a number you haven't just reproduced
• Blur confirmatory and exploratory results
• Report a p-value without an effect size and interval
• Use causal language for an observational finding
• Hide a deviation from the pre-registration
• Remove a worktree before confirming a merge succeeded
• Clean up a worktree you didn't create

Always:

• Verify reproducibility before offering options
• Present exactly 4 options (3 for detached HEAD)
• Get typed confirmation for discard
• Archive code + env + seed + pre-registration together