Audit Script — Systematic Code Review for Data Science
This skill systematically evaluates data analysis scripts for correctness, analytical soundness,
and quality. It surfaces bugs, questionable analytical choices, data handling problems, style
issues, and reproducibility gaps — producing a structured audit report with severity levels and
action items.
Unlike /learn-code (which teaches students to understand code), this skill is for critical
evaluation — finding what's wrong, fragile, or misleading. The user is a collaborator, not a
student. The tone is direct and analytical.
Core Philosophy: Simplicity First
The goal of an audit is NOT to make scripts handle every possible edge case. Data science
scripts should be simple, clean, easy to read, and well-annotated. Adding defensive code for
hypothetical problems makes scripts harder to read, which is the opposite of what we want.
The audit should:
- Flag real bugs that produce wrong results in the script's actual use case
- Flag analytical decisions that affect interpretation (undocumented, questionable, or missing)
- Flag clarity problems — code that's hard to follow, poorly annotated, or unnecessarily complex
- Note theoretical issues as awareness items, not action items — "be aware that
get(load(f))
is fragile if files contain multiple objects" is useful context; "rewrite to use new.env()"
is over-engineering a one-time script
- Actively flag over-engineering — unnecessary validation, defensive code for impossible cases,
and abstraction-for-its-own-sake are style findings, not good practices
Context matters. A one-time conversion script that processes a known, fixed dataset needs
different treatment than a reusable pipeline that will see unknown inputs. The audit must
calibrate its recommendations to the script's actual role.
Execution Model — get fresh eyes without leaving the chat
The biggest threat to a single-script audit is author contamination: if this chat wrote the
script, it rationalizes its own choices. But unlike a skill audit, much of this skill's value needs
the orchestrator's powers — running diagnostics against your real data and live env, and the
interactive walk-through. So split read from do:
- If this chat created/edited the script → delegate the cold read to a fresh Auditor
subagent (
templates/auditor-prompt.md): it reads the script cold, does the Domain Verification
research (WebSearch/WebFetch), applies the 5 categories, and returns candidate findings with line
numbers — running no code. The orchestrator then does the doing: runs the diagnostics that
confirm or refute each candidate against real data (anti_join, dim, re-running chunks),
discusses with you, and saves the report. The "subagent can't run your code" limit is a feature
here — the orchestrator's diagnostics are the verification of the cold read.
- If a clean chat is doing the audit (it didn't write the script) → it already has fresh eyes;
read inline, no Auditor subagent needed.
- Per mode: fast / report-only use the Auditor subagent for the read, then the orchestrator
confirms + (fast) discusses. Thorough stays the live pair-review (you are the fresh perspective),
but you may run the Auditor first as an independent cross-check whose candidates seed the walk-through.
- Never delegate the diagnostics, interaction, or report-saving, and never use an agent team —
only the read. The gated Adversarial Verification phase (refuter + completeness, below) still
applies on top, over findings already formed.
Entry Flow
When the skill is invoked (via /audit-script or auto-loaded from context):
1. Identify the Script
Check for:
- An IDE selection (highlighted code in the editor)
- A currently open file in the editor
- A file path mentioned in conversation
If none, ask: "Which script would you like to audit?"
Read the full script before proceeding.
2. Ask Mode
Use AskUserQuestion:
- Thorough (default) — Collaborative, section-by-section deep audit. You are a co-auditor: reading code, running chunks, inspecting data, catching issues yourself. I'll guide the systematic walk-through and add my own observations. This is pair code review.
- Fast — I'll read the whole script and identify issues independently, then we'll discuss my findings together.
- Report only — I'll audit independently and produce a report. You read it on your own time.
3. Ask for Specific Concerns
"Is there anything in particular you're worried about or want me to focus on?"
This lets the user flag known weak points, steer attention to a specific category, or provide
context about what the script is supposed to do.
Domain Verification Phase
This phase runs in all modes (thorough, fast, report-only) before the code audit begins.
Its purpose is to close the gap between code-level review and domain-specific correctness by
researching the actual tools, file formats, and analytical methods the script uses — then
auditing the code against that verified knowledge rather than relying on background familiarity.
Why This Matters
The most dangerous bugs in bioinformatics and data science aren't code bugs — they're
misunderstandings of what the tools and data actually do. A script can be syntactically
correct, logically clean, and still produce wrong results because the author (or reviewer)
didn't know that:
- BAM files store each alignment as a separate record (naive iteration overcounts multimappers)
- Cell Ranger uses MAPQ 255 for unique mapping (non-standard; SAM spec uses ≤60)
inner_join silently drops unmatched rows
- GFF3 coordinates are 1-based inclusive, BED is 0-based half-open
These are domain assumptions — facts about tools, formats, and methods that the code
depends on but doesn't state. The domain verification phase makes them explicit and checks them.
How It Works
1. Inventory Tools, Formats, and Methods
After reading the script, identify every external dependency the code relies on:
- File formats being read or written (BAM/SAM, BED, GFF3, VCF, FASTA, CSV, H5AD, etc.)
- Bioinformatics tools called via subprocess or library (minimap2, STAR, Cell Ranger,
BLAST, samtools, pysam, scanpy, Seurat, etc.)
- Statistical methods or analytical approaches (normalization, clustering, differential
expression, multiple testing correction, etc.)
- Library-specific behaviors (how pysam iterates BAM records, how pandas handles NAs in
groupby, how ggplot2 drops NAs in aesthetics, etc.)
2. Research Critical Assumptions
For each tool/format/method, use WebSearch and WebFetch to pull the relevant documentation
and identify the critical behaviors the code must handle correctly. Focus on:
- Record structure: What does one "row" or "record" represent? (A read? An alignment?
A gene? A transcript?)
- Coordinate systems: 0-based vs 1-based? Half-open vs closed? Does the code convert
correctly?
- Default behaviors: What does the tool do silently? (Drop unmapped reads? Merge
overlapping features? Sort output?)
- Flag/field semantics: What do specific values mean? (MAPQ 255, SAM flags, GFF3
attribute encoding)
- Edge cases: What happens with empty input, missing values, duplicate keys, very long
sequences, special characters?
- Known gotchas: What do people commonly get wrong with this tool/format? (Community
forums, GitHub issues, tool FAQs)
Produce a Domain Assumptions Checklist — a concrete list of facts that the code depends on,
each verified against documentation. Format:
DOMAIN ASSUMPTIONS CHECKLIST
─────────────────────────────
Tool/Format: pysam + BAM
✓ Each multimapped read appears as multiple records (primary + secondary)
✓ Iterating bam.fetch() yields alignments, not reads — must deduplicate by query_name
✓ MAPQ 255 = uniquely mapped (Cell Ranger convention; standard SAM caps at 60)
✓ is_secondary (flag 0x100) vs is_supplementary (flag 0x800) are different categories
? PCR duplicate marking in Cell Ranger BAMs — need to verify
Tool/Format: BED
✓ 0-based, half-open coordinates (start inclusive, end exclusive)
✓ Converting from GFF3 (1-based inclusive): subtract 1 from start, keep end as-is
Method: minimap2 cross-species mapping
✓ -k 10 appropriate for short ncRNAs (default k=15 misses tRNAs)
✓ --secondary=yes needed for multi-copy genes (rRNA arrays)
? Alignment quality thresholds for cross-species mapping — worth checking
Mark each assumption: ✓ (verified against docs), ✗ (contradicted by docs — potential BUG),
? (couldn't verify — flag for manual review).
3. Audit Code Against Checklist
With the checklist in hand, trace through the code and verify that each assumption is handled
correctly. This is where domain verification feeds into the standard audit:
- An assumption marked ✗ becomes a BUG or CONCERN finding
- An assumption marked ? becomes a WARNING with "needs manual domain review"
- An assumption marked ✓ that the code handles incorrectly becomes a BUG
- An assumption marked ✓ that the code handles correctly is noted as a Good Practice
4. Recommend Assumption Blocks
After the audit, recommend that the script include an explicit ASSUMPTIONS block documenting
the critical domain assumptions the code depends on. This makes future audits faster and helps
students understand what the code takes for granted:
# ASSUMPTIONS (verified against Cell Ranger 9.0 docs, SAM spec v1.6):
# - BAM iteration yields alignments, not reads; we deduplicate via seen_reads set
# - MAPQ 255 = uniquely mapped (Cell Ranger convention, not standard SAM)
# - PCR duplicates are NOT marked in possorted_genome_bam.bam
# - is_secondary and is_supplementary alignments are skipped (primary only)
# - GFF3 coordinates are 1-based inclusive; converted to 0-based for pysam fetch
Depth Scaling
The depth of domain verification scales with audit mode and script complexity:
- Report-only: Quick checklist from background knowledge + targeted web searches for
unfamiliar tools. Flag unknowns as ? rather than spending time researching deeply.
- Fast: Full research phase with web searches. Produce verified checklist. Flag remaining
unknowns for discussion.
- Thorough: Full research phase, then walk through the checklist with the user before
starting the code audit. The user adds domain knowledge ("we verified this threshold
experimentally"), resolves ? items, and may flag additional assumptions the checklist missed.
The 5 Audit Categories
Every section of the script is evaluated against these categories. Each finding is tagged with
its category and severity.
1. Correctness (bugs)
- Off-by-one errors, wrong variable references, typos in column names
- Logic errors (wrong condition, inverted filter, incorrect formula)
- Functions used incorrectly (wrong arguments, misunderstood return values)
- Race conditions or order dependencies
- Mismatches between what the code does and what the comments say
2. Analytical Reasoning
- Is the statistical test appropriate for this data and question?
- Are assumptions checked (normality, independence, homoscedasticity)?
- Are thresholds justified or arbitrary?
- Is the normalization/correction method appropriate for the experimental design?
- Are comparisons properly controlled?
- Could the analysis be misleading even if technically correct?
- Are there alternative approaches that would be more appropriate?
3. Data Handling
- Silent row/column drops (joins, filters, NA removal)
- Unvalidated assumptions about data structure
- Missing input validation (expected columns, types, ranges)
- Unchecked NAs propagating through calculations
- Joins that could introduce duplicates or lose rows
- Aggregation that hides important variation
4. Style & Organization
- Code clarity and readability
- Variable naming
- Comments (too few, misleading, or unnecessary)
- Function decomposition (repeated code that should be a function)
- Script flow (is the order logical?)
- Magic numbers without explanation
- Over-engineering — unnecessary defensive code, validation for impossible cases,
abstractions that add complexity without benefit. Simple, readable code is a feature.
5. Reproducibility
- Hardcoded paths or values
- Missing seed setting for random operations
- Environment dependencies (packages not loaded, conda env not specified)
- Missing input file documentation
- Output not clearly tied to input versions
- Platform-specific code without fallbacks
Severity Levels
- BUG — Incorrect behavior; produces wrong results in the script's actual use case. Must fix.
- CONCERN — Analytically questionable; may produce misleading results. Should investigate.
- WARNING — Not wrong, but fragile or risky. Should address.
- NOTE — Style, clarity, or minor improvement. Nice to fix.
- FYI — A pattern or assumption worth being aware of, but not something to change. Used for
theoretical fragilities that don't apply to the script's actual context (e.g., a function that
would break with different input, but the input is known and fixed). These are informational —
the author should understand them but not act on them.
Severity Calibration
Before assigning severity, consider:
- Is this a real problem or a theoretical one? If the script processes a known, fixed dataset
and the "issue" only manifests with different input, it's FYI, not BUG.
- Would the fix make the script simpler or more complex? If more complex, the cure may be
worse than the disease. Defensive code that handles impossible cases is a style problem.
- Is this a one-time script or a reusable pipeline? One-time scripts should be simple and
correct for their specific task. Reusable pipelines need more robustness.
Thorough Mode: Collaborative Section-by-Section Audit
The user is a co-auditor. Claude does NOT pre-digest the script — both work through it together.
The process of finding issues is as valuable as the findings themselves.
1. Script Overview
Read the script and present:
- What it does (plain language)
- What data it processes and what it produces
- A numbered map of logical sections
Ask: "Does this match your understanding of what this script should do?" Mismatches between
intent and implementation are a finding category.
2. Domain Verification (collaborative)
Run the full Domain Verification Phase (see above). In thorough mode:
- Research tools and formats, produce the Domain Assumptions Checklist
- Present the checklist to the user before starting the code walk-through
- Walk through each assumption: "I found that Cell Ranger uses MAPQ 255 for unique mapping —
does that match your understanding?"
- The user adds domain knowledge, resolves ? items, and may flag assumptions the checklist missed
- This step builds shared understanding of what the code should do before examining whether
it actually does
3. Section-by-Section Audit
For each logical section:
a. Present the code chunk (~15-20 lines max at a time)
b. User reads and runs it. Encourage the user to:
- Read the code before Claude explains anything
- Run the chunk in their console
- Inspect intermediate objects (
str(), dim(), head(), summary() in R;
.info(), .head(), .shape, .describe() in Python)
- Flag anything that looks off or that they don't understand
c. Claude probes and suggests checks — targeted to what's most likely to go wrong with
this specific type of code. Ask questions, suggest diagnostics, and raise concerns adapted to the
operation at hand. The user adds domain context and responds. Findings are documented as they
emerge.
For data loading/input:
- "Let's verify the dimensions — how many rows and columns did we get? Is that what you expect?"
- "Are there NAs in the key columns? Let's check before we go further"
- "Does the data structure match what the rest of the script assumes?"
For joins/merges:
- "This is an inner join — let's run
anti_join() to see what gets dropped and whether that's acceptable"
- "Could this join introduce duplicates? Let's check
nrow() before and after"
- "Are the join keys the right level of granularity?"
For filtering/subsetting:
- "How many rows survive this filter? Is that a reasonable fraction?"
- "Are we losing any categories entirely? Let's check what's left"
- "What happens to NAs — are they silently excluded?"
For statistical tests/modeling:
- "What assumptions does this test make about the data? Let's check if they hold"
- "Are there actually significant results? Let's look at the distribution of p-values"
- "Is the sample size sufficient for this test to have power?"
- "Why this test and not [alternative]? Is there a reason?"
For normalization/transformation:
- "Let's compare the distribution before and after — does the transformation do what we expect?"
- "Are there edge cases (zeros, negatives, NAs) that this transformation handles poorly?"
- "Is this the right normalization for this experimental design?"
For plotting/output:
- "Does this plot accurately represent the underlying data, or could it be misleading?"
- "Are the axis scales, labels, and legends correct?"
- "Is anything being visually hidden (e.g., overplotting, truncated axes)?"
The user adds their own observations and domain knowledge throughout. Their context may resolve
concerns ("this threshold was chosen because of the experimental design") or raise new ones.
d. Run diagnostics together when something is suspicious:
- "Let's check — run
anti_join() on these two tables and see how many rows don't match"
- "Try
summary() on this column — is the distribution what you'd expect?"
- "Comment out this filter and re-run — how does the downstream result change?"
e. Document findings — tag with category, severity, lines, and recommendation.
4. Cross-Section Analysis
After all individual sections:
- Trace data flow across the full script together
- Check: do transformations in section A affect correctness in section D?
- Look for cascading issues (e.g., silent drop early → wrong denominator later)
- Verify the overall analytical argument holds together
- Look for things the script should be doing but isn't (missing validation, missing checks)
5. Produce and Save Audit Report
Compile findings documented throughout into the structured report format (see below).
Save the report to .claude/audit_reports/ (see "Audit Report Format" for details).
Pacing
Every 2-3 sections, briefly check in: "How's the depth? Want to go faster or deeper?"
Fast Mode: Claude-Driven Audit with Discussion
Claude works through the script independently, then discusses findings with the user.
1. Full Script Read
Read the entire script.
2. Domain Verification
Run the full Domain Verification Phase (see above). In fast mode:
- Research tools and formats via web searches
- Produce verified Domain Assumptions Checklist
- Flag remaining unknowns (?) for discussion with the user
- Audit code against the checklist as part of the systematic analysis
3. Systematic Analysis
Apply the 5-category checklist across all sections:
- Trace data flow from input to output
- Check analytical reasoning and statistical assumptions
- Look for silent data loss, unvalidated joins, missing checks
- Evaluate style, organization, and reproducibility
- Run diagnostics where possible (dimension checks, NA counts, join validation)
- Check code against the Domain Assumptions Checklist — verify each assumption is handled
4. Produce and Save Audit Report
Full structured report with all findings, including Domain Assumptions Checklist.
Save the report to .claude/audit_reports/ (see "Audit Report Format" for details).
5. Collaborative Review of Findings
Present findings to the user, ordered by severity (BUG first):
- For each finding: show the code, explain the issue, discuss implications
- Present unresolved domain assumptions (? items) for the user's domain input
- User adds context, agrees/disagrees, reclassifies severity
- Together decide: fix now, defer, mark as acceptable
- New issues can surface during discussion
- Report updated with collaborative decisions and final dispositions
Report-Only Mode
Same as fast mode steps 1-4. No collaborative review. Produces the report and saves it
to .claude/audit_reports/. Findings are marked as "Unreviewed" in the status column.
In report-only mode, domain verification uses background knowledge + targeted web searches.
Unknown assumptions are flagged as ? in the checklist for the user to review independently.
Best for: batch auditing multiple scripts, quick quality snapshots, or when the user will
review the report in a separate session.
Diagnostic Capabilities
When auditing, Claude should actively run diagnostics (in Claude-driven modes) or suggest them
(in collaborative mode):
- Dimension checks:
dim(), nrow() before and after key operations
- NA propagation: Track where NAs enter and how they flow through the script
- Join validation:
anti_join() to check unmatched rows on both sides
- Distribution checks:
summary(), hist() for key variables, especially before statistical tests
- Duplication checks: Are there unexpected duplicates after joins or reshaping?
- Edge case probing: What happens with empty groups, single-observation groups, all-NA columns?
Optional: Adversarial Verification & Completeness (gated — requires the Workflow tool)
Gated — optional enhancement, not a requirement. The audit is complete and valid
without this phase. Run it ONLY when both hold:
- the Workflow tool is available in this environment — it is not present in every
Claude Code setup (lab-plugin users on a standard install will not have it), and
- at least one opt-in signal: ultracode is on, the user explicitly asked for thorough /
adversarial / "double-check" verification, or you are producing a saved handoff report
for another session.
If the Workflow tool is absent or no opt-in signal is present, skip this entire phase
and proceed with your solo findings. Never block, delay, or weaken the audit waiting on a
capability the environment may not have, and don't mention the workflow to users who can't
run it.
Run this after you've gathered findings in any mode (thorough / fast / report-only) and
before finalizing the report. Fan out over the findings you've already formed solo:
- Adversarially verify each BUG / CONCERN — one agent per finding, each re-reading the
actual script lines (and, where relevant, re-checking the tool/format docs behind the
Domain Assumptions Checklist) and prompted to refute: is the bug real and does it fire
in the script's actual use case, or is it a false positive / FYI-level theoretical
issue? This is the highest-value use — it kills plausible-but-wrong bug reports before
they reach the user, and lets a verifier downgrade a finding (to FYI), not only
confirm it, honoring this skill's simplicity-first philosophy.
- Completeness critic — one agent re-reading the script fresh, told what you already
found, hunting only for what you missed: silent data-flow drops, unverified domain
assumptions, edge cases, missing validation, analytical-reasoning gaps.
- (Report-only / handoff mode) draft fix artifacts — concrete diffs per confirmed
finding, so the implementing session can apply without re-deriving.
Fold the verified / refuted / newly-found findings into the report and note that an
adversarial pass was run. This is the "adversarial verify + completeness critic" pattern from
the Workflow tool's quality-patterns guidance — see that tool's docs for the fan-out
mechanics. The fan-out is read-only verification: never use it to run the user's code or
save the report (see Claude Code Behavior).
Audit Report Format
# Script Audit Report: {script_name}
**Date:** {date}
**Script:** {path/to/script}
**Auditor:** Claude Code {+ user name, if collaborative}
**Mode:** {Thorough / Fast / Report only}
## Summary
- **Total findings:** {N}
- **By severity:** {N} BUG, {N} CONCERN, {N} WARNING, {N} NOTE, {N} FYI
- **By category:** {N} Correctness, {N} Analytical, {N} Data Handling, {N} Style, {N} Reproducibility
- **Overall assessment:** {1-2 sentence summary of script quality and most critical issues}
- **Outputs trustworthy?** {Yes / Yes with caveats / No — must rerun after fixes}
{1-2 sentences explaining why. For "Yes with caveats", state which outputs are affected
and whether the impact is minor (e.g., cosmetic label error) or could change
interpretation. For "No", identify which bugs invalidate the outputs.
For scripts that don't produce analytical outputs (utilities, migration scripts, etc.),
state "N/A — script does not produce analytical outputs."}
## Domain Assumptions Checklist
| Tool/Format | Assumption | Verified? | Code Handles? | Finding |
|-------------|-----------|:---------:|:-------------:|---------|
| {tool} | {assumption} | ✓ / ✗ / ? | Yes / No / N/A | {ref or "OK"} |
## Findings
### BUG-1: {Short description}
- **Category:** {Correctness / Analytical / Data Handling / Style / Reproducibility}
- **Section:** {section name}
- **Lines:** {line range}
- **Description:** {What the issue is}
- **Impact:** {What goes wrong because of this}
- **Recommendation:** {How to fix it}
- **Status:** {Open / Discussed — {outcome} / Fixed / Unreviewed}
### CONCERN-1: {Short description}
...
### WARNING-1: {Short description}
...
### NOTE-1: {Short description}
...
### FYI-1: {Short description}
- **Category:** {category}
- **Lines:** {line range}
- **Description:** {What the pattern is and why it's worth knowing about}
- **Why not an action item:** {Why this doesn't need to change in this script's context}
## Sections Reviewed
| Section | Lines | Issues Found | Notes |
|---------|-------|-------------|-------|
| {name} | {range} | BUG-1, WARN-2 | {brief note} |
| {name} | {range} | None | Clean |
## Analytical Decisions Inventory
| Section | Decision | Current Choice | Justification | Alternatives | Risk Level |
|---------|----------|---------------|---------------|-------------|------------|
| ... | ... | ... | ... | ... | ... |
## Action Items
| Priority | Finding | Action | Owner |
|----------|---------|--------|-------|
| 1 | BUG-1 | Fix immediately | {name} |
| 2 | CONCERN-1 | Investigate | {name} |
Always save the report to .claude/audit_reports/{script_name}_audit_report.md in the
project root. Create the .claude/audit_reports/ directory if it doesn't exist. Every audit
must produce a saved report file — this is not optional.
Audit Principles
- Trace the data, not just the code. The most important bugs in data science are data flow bugs — silent drops, wrong joins, incorrect baselines. Follow the data from input to output.
- Question analytical defaults. Just because
method = "BH" is common doesn't mean it's right for this data. Every default is a choice.
- Check what's NOT in the script. Missing validation, missing checks, missing documentation are findings too.
- Severity is about impact, not aesthetics. A confusing variable name is a NOTE. A confusing variable name that leads someone to use the wrong column is a BUG.
- Be specific. "This join might lose rows" is not helpful. "This inner_join on line 47 drops 23 rows because gene_names has entries not in mdata" is actionable.
- Run diagnostics, don't guess. When something looks suspicious, actually run the code to verify before reporting it as a finding.
- Credit good practices. Note when the script does something well — especially clean structure, good documentation, or thoughtful analytical choices.
- Verify domain assumptions, don't assume. When the code depends on tool/format behavior, look it up rather than relying on background knowledge. A verified assumption is worth ten educated guesses.
- Simplicity is a virtue, not a gap. A script that does its job cleanly without handling every edge case is well-written, not incomplete. Recommend adding code only when it solves a real problem. If a finding's recommended fix would make the script longer and harder to read, reconsider whether it's worth reporting as an action item — it may be better as an FYI.
- Calibrate to the script's role. A one-time conversion script on a known dataset needs different rigor than a reusable pipeline. Don't treat every script as if it will be rerun on unknown inputs.
Claude Code Behavior
When this skill is active:
- Be direct, not hedging. "This join silently drops 50 rows" not "This join might potentially have some issues with row counts."
- Show evidence. When flagging an issue, show the specific code and explain exactly what goes wrong. Run diagnostics where possible.
- Distinguish fact from opinion. "This uses an inner join that drops rows" (fact) vs. "I think a left join would be better here" (opinion/recommendation). Both are valid but should be clearly distinguished.
- Don't over-report. Not every line needs a finding. If a section is clean, say so and move on. Audit fatigue from low-severity noise degrades the value of real findings.
- Protect simplicity. The audit should never push scripts toward unnecessary complexity. If a recommendation would make the code longer and harder to read to handle a theoretical edge case, use FYI severity instead. Actively flag existing over-engineering as a style finding — unnecessary defensive code is clutter.
- Respect the author's context. In collaborative mode, the author may have reasons for choices that aren't documented. Ask before assuming something is wrong.
- Track uncertainty. If you're not sure whether something is a bug or intentional, say so. "This might be intentional, but if not, it would cause..." is better than a false positive or a missed bug.
- In thorough mode: don't pre-digest. Let the user read and run the code first. Ask questions, don't give answers. The user finding issues themselves is the point.
- In fast mode: be comprehensive. You're working alone — don't skip sections or categories. The user is counting on your thoroughness because they're not reading every line.
- Split read from do (see Execution Model). Diagnostics, the interactive walk-through, and
saving the report stay with the orchestrator — subagents can't reliably run your code/data or save
files. But the cold read is delegable: when this chat authored the script, hand the static review
- domain research to a fresh Auditor subagent to escape the chat's own rationalization, then run
the confirming diagnostics yourself. A clean chat already has fresh eyes — read inline. The optional
gated verification (refuter + completeness over findings already formed) may fan out via the Workflow
tool when present + opted-in. Never use a subagent for the walk-through, for running the user's code,
or for saving the report; never use an agent team.
1---2name: audit-script3description: Systematic audit of data analysis scripts for bugs, analytical reasoning, data handling, style, and reproducibility. Includes domain verification phase that researches tools, file formats, and methods to catch domain-specific errors (not just code bugs). Use when auditing a script, reviewing code for correctness, checking for bugs, preparing a script for publication, or when the user says "audit this script", "review this code", "check this for bugs", or "is this script correct". Three modes: thorough (collaborative section-by-section), fast (Claude-driven with discussion), and report-only. Do NOT load for quick one-off questions about a single line or function.4---56# Audit Script — Systematic Code Review for Data Science78This skill systematically evaluates data analysis scripts for correctness, analytical soundness,9and quality. It surfaces bugs, questionable analytical choices, data handling problems, style10issues, and reproducibility gaps — producing a structured audit report with severity levels and11action items.1213Unlike `/learn-code` (which teaches students to understand code), this skill is for **critical14evaluation** — finding what's wrong, fragile, or misleading. The user is a collaborator, not a15student. The tone is direct and analytical.1617### Core Philosophy: Simplicity First1819**The goal of an audit is NOT to make scripts handle every possible edge case.** Data science20scripts should be simple, clean, easy to read, and well-annotated. Adding defensive code for21hypothetical problems makes scripts harder to read, which is the opposite of what we want.2223The audit should:24- **Flag real bugs** that produce wrong results in the script's actual use case25- **Flag analytical decisions** that affect interpretation (undocumented, questionable, or missing)26- **Flag clarity problems** — code that's hard to follow, poorly annotated, or unnecessarily complex27- **Note theoretical issues as awareness items**, not action items — "be aware that `get(load(f))`28 is fragile if files contain multiple objects" is useful context; "rewrite to use `new.env()`"29 is over-engineering a one-time script30- **Actively flag over-engineering** — unnecessary validation, defensive code for impossible cases,31 and abstraction-for-its-own-sake are style findings, not good practices3233**Context matters.** A one-time conversion script that processes a known, fixed dataset needs34different treatment than a reusable pipeline that will see unknown inputs. The audit must35calibrate its recommendations to the script's actual role.3637---3839## Execution Model — get fresh eyes without leaving the chat4041The biggest threat to a single-script audit is **author contamination**: if this chat wrote the42script, it rationalizes its own choices. But unlike a skill audit, much of this skill's value needs43the orchestrator's powers — running diagnostics against your real data and live env, and the44interactive walk-through. So split **read** from **do**:4546- **If this chat created/edited the script** → delegate the *cold read* to a fresh **Auditor47 subagent** (`templates/auditor-prompt.md`): it reads the script cold, does the Domain Verification48 research (WebSearch/WebFetch), applies the 5 categories, and returns **candidate findings with line49 numbers** — running no code. The **orchestrator then does the doing**: runs the diagnostics that50 confirm or refute each candidate against real data (`anti_join`, `dim`, re-running chunks),51 discusses with you, and saves the report. The "subagent can't run your code" limit is a *feature*52 here — the orchestrator's diagnostics *are* the verification of the cold read.53- **If a clean chat is doing the audit** (it didn't write the script) → it already has fresh eyes;54 read inline, no Auditor subagent needed.55- **Per mode:** *fast* / *report-only* use the Auditor subagent for the read, then the orchestrator56 confirms + (fast) discusses. *Thorough* stays the live pair-review (you are the fresh perspective),57 but you may run the Auditor first as an independent cross-check whose candidates seed the walk-through.58- **Never** delegate the diagnostics, interaction, or report-saving, and **never use an agent team** —59 only the read. The gated **Adversarial Verification** phase (refuter + completeness, below) still60 applies on top, over findings already formed.6162---6364## Entry Flow6566When the skill is invoked (via `/audit-script` or auto-loaded from context):6768### 1. Identify the Script6970Check for:71- An IDE selection (highlighted code in the editor)72- A currently open file in the editor73- A file path mentioned in conversation7475If none, ask: "Which script would you like to audit?"7677Read the full script before proceeding.7879### 2. Ask Mode8081Use AskUserQuestion:8283- **Thorough (default)** — Collaborative, section-by-section deep audit. You are a co-auditor: reading code, running chunks, inspecting data, catching issues yourself. I'll guide the systematic walk-through and add my own observations. This is pair code review.84- **Fast** — I'll read the whole script and identify issues independently, then we'll discuss my findings together.85- **Report only** — I'll audit independently and produce a report. You read it on your own time.8687### 3. Ask for Specific Concerns8889"Is there anything in particular you're worried about or want me to focus on?"9091This lets the user flag known weak points, steer attention to a specific category, or provide92context about what the script is supposed to do.9394---9596## Domain Verification Phase9798**This phase runs in all modes** (thorough, fast, report-only) before the code audit begins.99Its purpose is to close the gap between code-level review and domain-specific correctness by100researching the actual tools, file formats, and analytical methods the script uses — then101auditing the code against that verified knowledge rather than relying on background familiarity.102103### Why This Matters104105The most dangerous bugs in bioinformatics and data science aren't code bugs — they're106**misunderstandings of what the tools and data actually do.** A script can be syntactically107correct, logically clean, and still produce wrong results because the author (or reviewer)108didn't know that:109- BAM files store each alignment as a separate record (naive iteration overcounts multimappers)110- Cell Ranger uses MAPQ 255 for unique mapping (non-standard; SAM spec uses ≤60)111- `inner_join` silently drops unmatched rows112- GFF3 coordinates are 1-based inclusive, BED is 0-based half-open113114These are **domain assumptions** — facts about tools, formats, and methods that the code115depends on but doesn't state. The domain verification phase makes them explicit and checks them.116117### How It Works118119#### 1. Inventory Tools, Formats, and Methods120121After reading the script, identify every external dependency the code relies on:122123- **File formats** being read or written (BAM/SAM, BED, GFF3, VCF, FASTA, CSV, H5AD, etc.)124- **Bioinformatics tools** called via subprocess or library (minimap2, STAR, Cell Ranger,125 BLAST, samtools, pysam, scanpy, Seurat, etc.)126- **Statistical methods** or analytical approaches (normalization, clustering, differential127 expression, multiple testing correction, etc.)128- **Library-specific behaviors** (how pysam iterates BAM records, how pandas handles NAs in129 groupby, how ggplot2 drops NAs in aesthetics, etc.)130131#### 2. Research Critical Assumptions132133For each tool/format/method, use **WebSearch and WebFetch** to pull the relevant documentation134and identify the critical behaviors the code must handle correctly. Focus on:135136- **Record structure:** What does one "row" or "record" represent? (A read? An alignment?137 A gene? A transcript?)138- **Coordinate systems:** 0-based vs 1-based? Half-open vs closed? Does the code convert139 correctly?140- **Default behaviors:** What does the tool do silently? (Drop unmapped reads? Merge141 overlapping features? Sort output?)142- **Flag/field semantics:** What do specific values mean? (MAPQ 255, SAM flags, GFF3143 attribute encoding)144- **Edge cases:** What happens with empty input, missing values, duplicate keys, very long145 sequences, special characters?146- **Known gotchas:** What do people commonly get wrong with this tool/format? (Community147 forums, GitHub issues, tool FAQs)148149Produce a **Domain Assumptions Checklist** — a concrete list of facts that the code depends on,150each verified against documentation. Format:151152```153DOMAIN ASSUMPTIONS CHECKLIST154─────────────────────────────155Tool/Format: pysam + BAM156 ✓ Each multimapped read appears as multiple records (primary + secondary)157 ✓ Iterating bam.fetch() yields alignments, not reads — must deduplicate by query_name158 ✓ MAPQ 255 = uniquely mapped (Cell Ranger convention; standard SAM caps at 60)159 ✓ is_secondary (flag 0x100) vs is_supplementary (flag 0x800) are different categories160 ? PCR duplicate marking in Cell Ranger BAMs — need to verify161162Tool/Format: BED163 ✓ 0-based, half-open coordinates (start inclusive, end exclusive)164 ✓ Converting from GFF3 (1-based inclusive): subtract 1 from start, keep end as-is165166Method: minimap2 cross-species mapping167 ✓ -k 10 appropriate for short ncRNAs (default k=15 misses tRNAs)168 ✓ --secondary=yes needed for multi-copy genes (rRNA arrays)169 ? Alignment quality thresholds for cross-species mapping — worth checking170```171172Mark each assumption: ✓ (verified against docs), ✗ (contradicted by docs — potential BUG),173? (couldn't verify — flag for manual review).174175#### 3. Audit Code Against Checklist176177With the checklist in hand, trace through the code and verify that each assumption is handled178correctly. This is where domain verification feeds into the standard audit:179180- An assumption marked ✗ becomes a **BUG** or **CONCERN** finding181- An assumption marked ? becomes a **WARNING** with "needs manual domain review"182- An assumption marked ✓ that the code handles incorrectly becomes a **BUG**183- An assumption marked ✓ that the code handles correctly is noted as a **Good Practice**184185#### 4. Recommend Assumption Blocks186187After the audit, recommend that the script include an explicit **ASSUMPTIONS block** documenting188the critical domain assumptions the code depends on. This makes future audits faster and helps189students understand what the code takes for granted:190191```python192# ASSUMPTIONS (verified against Cell Ranger 9.0 docs, SAM spec v1.6):193# - BAM iteration yields alignments, not reads; we deduplicate via seen_reads set194# - MAPQ 255 = uniquely mapped (Cell Ranger convention, not standard SAM)195# - PCR duplicates are NOT marked in possorted_genome_bam.bam196# - is_secondary and is_supplementary alignments are skipped (primary only)197# - GFF3 coordinates are 1-based inclusive; converted to 0-based for pysam fetch198```199200### Depth Scaling201202The depth of domain verification scales with audit mode and script complexity:203204- **Report-only:** Quick checklist from background knowledge + targeted web searches for205 unfamiliar tools. Flag unknowns as ? rather than spending time researching deeply.206- **Fast:** Full research phase with web searches. Produce verified checklist. Flag remaining207 unknowns for discussion.208- **Thorough:** Full research phase, then walk through the checklist with the user before209 starting the code audit. The user adds domain knowledge ("we verified this threshold210 experimentally"), resolves ? items, and may flag additional assumptions the checklist missed.211212---213214## The 5 Audit Categories215216Every section of the script is evaluated against these categories. Each finding is tagged with217its category and severity.218219### 1. Correctness (bugs)220- Off-by-one errors, wrong variable references, typos in column names221- Logic errors (wrong condition, inverted filter, incorrect formula)222- Functions used incorrectly (wrong arguments, misunderstood return values)223- Race conditions or order dependencies224- Mismatches between what the code does and what the comments say225226### 2. Analytical Reasoning227- Is the statistical test appropriate for this data and question?228- Are assumptions checked (normality, independence, homoscedasticity)?229- Are thresholds justified or arbitrary?230- Is the normalization/correction method appropriate for the experimental design?231- Are comparisons properly controlled?232- Could the analysis be misleading even if technically correct?233- Are there alternative approaches that would be more appropriate?234235### 3. Data Handling236- Silent row/column drops (joins, filters, NA removal)237- Unvalidated assumptions about data structure238- Missing input validation (expected columns, types, ranges)239- Unchecked NAs propagating through calculations240- Joins that could introduce duplicates or lose rows241- Aggregation that hides important variation242243### 4. Style & Organization244- Code clarity and readability245- Variable naming246- Comments (too few, misleading, or unnecessary)247- Function decomposition (repeated code that should be a function)248- Script flow (is the order logical?)249- Magic numbers without explanation250- **Over-engineering** — unnecessary defensive code, validation for impossible cases,251 abstractions that add complexity without benefit. Simple, readable code is a feature.252253### 5. Reproducibility254- Hardcoded paths or values255- Missing seed setting for random operations256- Environment dependencies (packages not loaded, conda env not specified)257- Missing input file documentation258- Output not clearly tied to input versions259- Platform-specific code without fallbacks260261---262263## Severity Levels264265- **BUG** — Incorrect behavior; produces wrong results *in the script's actual use case*. Must fix.266- **CONCERN** — Analytically questionable; may produce misleading results. Should investigate.267- **WARNING** — Not wrong, but fragile or risky. Should address.268- **NOTE** — Style, clarity, or minor improvement. Nice to fix.269- **FYI** — A pattern or assumption worth being aware of, but not something to change. Used for270 theoretical fragilities that don't apply to the script's actual context (e.g., a function that271 would break with different input, but the input is known and fixed). These are informational —272 the author should understand them but not act on them.273274### Severity Calibration275276Before assigning severity, consider:277- **Is this a real problem or a theoretical one?** If the script processes a known, fixed dataset278 and the "issue" only manifests with different input, it's FYI, not BUG.279- **Would the fix make the script simpler or more complex?** If more complex, the cure may be280 worse than the disease. Defensive code that handles impossible cases is a style problem.281- **Is this a one-time script or a reusable pipeline?** One-time scripts should be simple and282 correct for their specific task. Reusable pipelines need more robustness.283284---285286## Thorough Mode: Collaborative Section-by-Section Audit287288The user is a co-auditor. Claude does NOT pre-digest the script — both work through it together.289The process of finding issues is as valuable as the findings themselves.290291### 1. Script Overview292293Read the script and present:294- What it does (plain language)295- What data it processes and what it produces296- A numbered map of logical sections297298Ask: "Does this match your understanding of what this script should do?" Mismatches between299intent and implementation are a finding category.300301### 2. Domain Verification (collaborative)302303Run the full Domain Verification Phase (see above). In thorough mode:304- Research tools and formats, produce the Domain Assumptions Checklist305- Present the checklist to the user before starting the code walk-through306- Walk through each assumption: "I found that Cell Ranger uses MAPQ 255 for unique mapping —307 does that match your understanding?"308- The user adds domain knowledge, resolves ? items, and may flag assumptions the checklist missed309- This step builds shared understanding of what the code *should* do before examining whether310 it actually does311312### 3. Section-by-Section Audit313314For each logical section:315316**a. Present the code chunk** (~15-20 lines max at a time)317318**b. User reads and runs it.** Encourage the user to:319- Read the code before Claude explains anything320- Run the chunk in their console321- Inspect intermediate objects (`str()`, `dim()`, `head()`, `summary()` in R;322 `.info()`, `.head()`, `.shape`, `.describe()` in Python)323- Flag anything that looks off or that they don't understand324325**c. Claude probes and suggests checks** — targeted to what's most likely to go wrong with326this specific type of code. Ask questions, suggest diagnostics, and raise concerns adapted to the327operation at hand. The user adds domain context and responds. Findings are documented as they328emerge.329330**For data loading/input:**331- "Let's verify the dimensions — how many rows and columns did we get? Is that what you expect?"332- "Are there NAs in the key columns? Let's check before we go further"333- "Does the data structure match what the rest of the script assumes?"334335**For joins/merges:**336- "This is an inner join — let's run `anti_join()` to see what gets dropped and whether that's acceptable"337- "Could this join introduce duplicates? Let's check `nrow()` before and after"338- "Are the join keys the right level of granularity?"339340**For filtering/subsetting:**341- "How many rows survive this filter? Is that a reasonable fraction?"342- "Are we losing any categories entirely? Let's check what's left"343- "What happens to NAs — are they silently excluded?"344345**For statistical tests/modeling:**346- "What assumptions does this test make about the data? Let's check if they hold"347- "Are there actually significant results? Let's look at the distribution of p-values"348- "Is the sample size sufficient for this test to have power?"349- "Why this test and not [alternative]? Is there a reason?"350351**For normalization/transformation:**352- "Let's compare the distribution before and after — does the transformation do what we expect?"353- "Are there edge cases (zeros, negatives, NAs) that this transformation handles poorly?"354- "Is this the right normalization for this experimental design?"355356**For plotting/output:**357- "Does this plot accurately represent the underlying data, or could it be misleading?"358- "Are the axis scales, labels, and legends correct?"359- "Is anything being visually hidden (e.g., overplotting, truncated axes)?"360361The user adds their own observations and domain knowledge throughout. Their context may resolve362concerns ("this threshold was chosen because of the experimental design") or raise new ones.363364**d. Run diagnostics together** when something is suspicious:365- "Let's check — run `anti_join()` on these two tables and see how many rows don't match"366- "Try `summary()` on this column — is the distribution what you'd expect?"367- "Comment out this filter and re-run — how does the downstream result change?"368369**e. Document findings** — tag with category, severity, lines, and recommendation.370371### 4. Cross-Section Analysis372373After all individual sections:374- Trace data flow across the full script together375- Check: do transformations in section A affect correctness in section D?376- Look for cascading issues (e.g., silent drop early → wrong denominator later)377- Verify the overall analytical argument holds together378- Look for things the script should be doing but isn't (missing validation, missing checks)379380### 5. Produce and Save Audit Report381382Compile findings documented throughout into the structured report format (see below).383Save the report to `.claude/audit_reports/` (see "Audit Report Format" for details).384385### Pacing386387Every 2-3 sections, briefly check in: "How's the depth? Want to go faster or deeper?"388389---390391## Fast Mode: Claude-Driven Audit with Discussion392393Claude works through the script independently, then discusses findings with the user.394395### 1. Full Script Read396397Read the entire script.398399### 2. Domain Verification400401Run the full Domain Verification Phase (see above). In fast mode:402- Research tools and formats via web searches403- Produce verified Domain Assumptions Checklist404- Flag remaining unknowns (?) for discussion with the user405- Audit code against the checklist as part of the systematic analysis406407### 3. Systematic Analysis408409Apply the 5-category checklist across all sections:410- Trace data flow from input to output411- Check analytical reasoning and statistical assumptions412- Look for silent data loss, unvalidated joins, missing checks413- Evaluate style, organization, and reproducibility414- Run diagnostics where possible (dimension checks, NA counts, join validation)415- **Check code against the Domain Assumptions Checklist** — verify each assumption is handled416417### 4. Produce and Save Audit Report418419Full structured report with all findings, including Domain Assumptions Checklist.420Save the report to `.claude/audit_reports/` (see "Audit Report Format" for details).421422### 5. Collaborative Review of Findings423424Present findings to the user, ordered by severity (BUG first):425- For each finding: show the code, explain the issue, discuss implications426- **Present unresolved domain assumptions (? items)** for the user's domain input427- User adds context, agrees/disagrees, reclassifies severity428- Together decide: fix now, defer, mark as acceptable429- New issues can surface during discussion430- Report updated with collaborative decisions and final dispositions431432---433434## Report-Only Mode435436Same as fast mode steps 1-4. No collaborative review. Produces the report and saves it437to `.claude/audit_reports/`. Findings are marked as "Unreviewed" in the status column.438439In report-only mode, domain verification uses background knowledge + targeted web searches.440Unknown assumptions are flagged as ? in the checklist for the user to review independently.441442Best for: batch auditing multiple scripts, quick quality snapshots, or when the user will443review the report in a separate session.444445---446447## Diagnostic Capabilities448449When auditing, Claude should actively run diagnostics (in Claude-driven modes) or suggest them450(in collaborative mode):451452- **Dimension checks:** `dim()`, `nrow()` before and after key operations453- **NA propagation:** Track where NAs enter and how they flow through the script454- **Join validation:** `anti_join()` to check unmatched rows on both sides455- **Distribution checks:** `summary()`, `hist()` for key variables, especially before statistical tests456- **Duplication checks:** Are there unexpected duplicates after joins or reshaping?457- **Edge case probing:** What happens with empty groups, single-observation groups, all-NA columns?458459---460461## Optional: Adversarial Verification & Completeness (gated — requires the Workflow tool)462463> **Gated — optional enhancement, not a requirement.** The audit is complete and valid464> without this phase. Run it ONLY when **both** hold:465> - the **Workflow tool is available** in this environment — it is **not** present in every466> Claude Code setup (lab-plugin users on a standard install will not have it), **and**467> - **at least one opt-in signal:** ultracode is on, the user explicitly asked for thorough /468> adversarial / "double-check" verification, or you are producing a saved handoff report469> for another session.470>471> **If the Workflow tool is absent or no opt-in signal is present, skip this entire phase**472> and proceed with your solo findings. Never block, delay, or weaken the audit waiting on a473> capability the environment may not have, and don't mention the workflow to users who can't474> run it.475476Run this **after** you've gathered findings in any mode (thorough / fast / report-only) and477**before** finalizing the report. Fan out over the findings you've **already formed solo**:4784791. **Adversarially verify each BUG / CONCERN** — one agent per finding, each re-reading the480 actual script lines (and, where relevant, re-checking the tool/format docs behind the481 Domain Assumptions Checklist) and prompted to **refute**: is the bug real and does it fire482 in the script's *actual* use case, or is it a false positive / FYI-level theoretical483 issue? This is the highest-value use — it kills plausible-but-wrong bug reports before484 they reach the user, and lets a verifier **downgrade** a finding (to FYI), not only485 confirm it, honoring this skill's simplicity-first philosophy.4862. **Completeness critic** — one agent re-reading the script fresh, told what you already487 found, hunting only for what you **missed**: silent data-flow drops, unverified domain488 assumptions, edge cases, missing validation, analytical-reasoning gaps.4893. **(Report-only / handoff mode) draft fix artifacts** — concrete diffs per confirmed490 finding, so the implementing session can apply without re-deriving.491492Fold the verified / refuted / newly-found findings into the report and note that an493adversarial pass was run. This is the "adversarial verify + completeness critic" pattern from494the Workflow tool's quality-patterns guidance — see that tool's docs for the fan-out495mechanics. The fan-out is **read-only verification**: never use it to run the user's code or496save the report (see Claude Code Behavior).497498---499500## Audit Report Format501502```markdown503# Script Audit Report: {script_name}504505**Date:** {date}506**Script:** {path/to/script}507**Auditor:** Claude Code {+ user name, if collaborative}508**Mode:** {Thorough / Fast / Report only}509510## Summary511512- **Total findings:** {N}513- **By severity:** {N} BUG, {N} CONCERN, {N} WARNING, {N} NOTE, {N} FYI514- **By category:** {N} Correctness, {N} Analytical, {N} Data Handling, {N} Style, {N} Reproducibility515- **Overall assessment:** {1-2 sentence summary of script quality and most critical issues}516- **Outputs trustworthy?** {Yes / Yes with caveats / No — must rerun after fixes}517 {1-2 sentences explaining why. For "Yes with caveats", state which outputs are affected518 and whether the impact is minor (e.g., cosmetic label error) or could change519 interpretation. For "No", identify which bugs invalidate the outputs.520 For scripts that don't produce analytical outputs (utilities, migration scripts, etc.),521 state "N/A — script does not produce analytical outputs."}522523## Domain Assumptions Checklist524525| Tool/Format | Assumption | Verified? | Code Handles? | Finding |526|-------------|-----------|:---------:|:-------------:|---------|527| {tool} | {assumption} | ✓ / ✗ / ? | Yes / No / N/A | {ref or "OK"} |528529## Findings530531### BUG-1: {Short description}532- **Category:** {Correctness / Analytical / Data Handling / Style / Reproducibility}533- **Section:** {section name}534- **Lines:** {line range}535- **Description:** {What the issue is}536- **Impact:** {What goes wrong because of this}537- **Recommendation:** {How to fix it}538- **Status:** {Open / Discussed — {outcome} / Fixed / Unreviewed}539540### CONCERN-1: {Short description}541...542543### WARNING-1: {Short description}544...545546### NOTE-1: {Short description}547...548549### FYI-1: {Short description}550- **Category:** {category}551- **Lines:** {line range}552- **Description:** {What the pattern is and why it's worth knowing about}553- **Why not an action item:** {Why this doesn't need to change in this script's context}554555## Sections Reviewed556557| Section | Lines | Issues Found | Notes |558|---------|-------|-------------|-------|559| {name} | {range} | BUG-1, WARN-2 | {brief note} |560| {name} | {range} | None | Clean |561562## Analytical Decisions Inventory563564| Section | Decision | Current Choice | Justification | Alternatives | Risk Level |565|---------|----------|---------------|---------------|-------------|------------|566| ... | ... | ... | ... | ... | ... |567568## Action Items569570| Priority | Finding | Action | Owner |571|----------|---------|--------|-------|572| 1 | BUG-1 | Fix immediately | {name} |573| 2 | CONCERN-1 | Investigate | {name} |574```575576**Always save the report** to `.claude/audit_reports/{script_name}_audit_report.md` in the577project root. Create the `.claude/audit_reports/` directory if it doesn't exist. Every audit578must produce a saved report file — this is not optional.579580---581582## Audit Principles5835841. **Trace the data, not just the code.** The most important bugs in data science are data flow bugs — silent drops, wrong joins, incorrect baselines. Follow the data from input to output.5852. **Question analytical defaults.** Just because `method = "BH"` is common doesn't mean it's right for this data. Every default is a choice.5863. **Check what's NOT in the script.** Missing validation, missing checks, missing documentation are findings too.5874. **Severity is about impact, not aesthetics.** A confusing variable name is a NOTE. A confusing variable name that leads someone to use the wrong column is a BUG.5885. **Be specific.** "This join might lose rows" is not helpful. "This inner_join on line 47 drops 23 rows because gene_names has entries not in mdata" is actionable.5896. **Run diagnostics, don't guess.** When something looks suspicious, actually run the code to verify before reporting it as a finding.5907. **Credit good practices.** Note when the script does something well — especially clean structure, good documentation, or thoughtful analytical choices.5918. **Verify domain assumptions, don't assume.** When the code depends on tool/format behavior, look it up rather than relying on background knowledge. A verified assumption is worth ten educated guesses.5929. **Simplicity is a virtue, not a gap.** A script that does its job cleanly without handling every edge case is well-written, not incomplete. Recommend adding code only when it solves a real problem. If a finding's recommended fix would make the script longer and harder to read, reconsider whether it's worth reporting as an action item — it may be better as an FYI.59310. **Calibrate to the script's role.** A one-time conversion script on a known dataset needs different rigor than a reusable pipeline. Don't treat every script as if it will be rerun on unknown inputs.594595---596597## Claude Code Behavior598599When this skill is active:600601- **Be direct, not hedging.** "This join silently drops 50 rows" not "This join might potentially have some issues with row counts."602- **Show evidence.** When flagging an issue, show the specific code and explain exactly what goes wrong. Run diagnostics where possible.603- **Distinguish fact from opinion.** "This uses an inner join that drops rows" (fact) vs. "I think a left join would be better here" (opinion/recommendation). Both are valid but should be clearly distinguished.604- **Don't over-report.** Not every line needs a finding. If a section is clean, say so and move on. Audit fatigue from low-severity noise degrades the value of real findings.605- **Protect simplicity.** The audit should never push scripts toward unnecessary complexity. If a recommendation would make the code longer and harder to read to handle a theoretical edge case, use FYI severity instead. Actively flag existing over-engineering as a style finding — unnecessary defensive code is clutter.606- **Respect the author's context.** In collaborative mode, the author may have reasons for choices that aren't documented. Ask before assuming something is wrong.607- **Track uncertainty.** If you're not sure whether something is a bug or intentional, say so. "This might be intentional, but if not, it would cause..." is better than a false positive or a missed bug.608- **In thorough mode: don't pre-digest.** Let the user read and run the code first. Ask questions, don't give answers. The user finding issues themselves is the point.609- **In fast mode: be comprehensive.** You're working alone — don't skip sections or categories. The user is counting on your thoroughness because they're not reading every line.610- **Split read from do (see Execution Model).** Diagnostics, the interactive walk-through, and611 saving the report stay with the orchestrator — subagents can't reliably run your code/data or save612 files. But the *cold read* is delegable: when this chat authored the script, hand the static review613 + domain research to a fresh **Auditor subagent** to escape the chat's own rationalization, then run614 the confirming diagnostics yourself. A clean chat already has fresh eyes — read inline. The optional615 gated verification (refuter + completeness over findings already formed) may fan out via the Workflow616 tool when present + opted-in. Never use a subagent for the walk-through, for running the user's code,617 or for saving the report; never use an agent team.