wolverin0 Skill: Audit Tambon Hunt
This skill detects the three LLM-specific bug patterns documented in Tambon et al. 2025 — patterns that are nearly absent from human-written code. Finding them is double signal: each occurrence is a correctness defect AND evidence of unreviewed AI generation.
The full taxonomy and detection guidance is in
@.claude/context/audit-tambon-signatures.md. Load it before starting.
Process
Pre-flight
view @.claude/context/audit-rules.md
view @.claude/context/audit-tambon-signatures.md
Phase 1 — Static type-check (the heavy lifter)
The single most effective Tambon detector is a strict type-checker. Run the appropriate one(s):
# Python
which mypy &>/dev/null && mypy --strict src/ 2>&1 | tee /tmp/mypy.log
which pyright &>/dev/null && pyright src/ 2>&1 | tee /tmp/pyright.log
# TypeScript
which tsc &>/dev/null && tsc --noEmit --strict 2>&1 | tee /tmp/tsc.log
# If nothing is installed, that's a finding in its own right —
# "no static type checking configured." Continue with grep-based hunts.
Filter the output for the relevant error categories:
- mypy:
has no attribute,Module has no attribute,is not defined - pyright:
Cannot access attribute,is not a known attribute,is not defined - tsc:
Property '...' does not exist,Cannot find name,has no exported member
Each unique error of those classes is a candidate Tambon finding. Verify by reading the cited path:line — sometimes type-checker output points at a real Tambon, sometimes at a missing type stub for a real library. Distinguish:
- "Cannot find module X" where X is a real package = missing types, not Tambon
- "Property foo does not exist on type Bar" where Bar is a real class and foo is misspelled = Tambon (Wrong Attribute)
- "Module X has no attribute foo" where X is a real package = could be Hallucinated Object OR a recent API change. Cross-reference with the pinned version in the manifest.
Phase 2 — Silly Mistake hunt
Static type checkers don't catch most Silly Mistakes. Use linters:
# Python
ruff check --select=PLR src/ 2>&1 | tee /tmp/ruff.log
pylint src/ --disable=all \
--enable=duplicate-code,no-else-return,redundant-condition,unnecessary-pass \
2>&1 | tee /tmp/pylint.log
# JS/TS
npx eslint src/ \
--rule 'no-dupe-else-if: error' \
--rule 'no-self-compare: error' \
--rule 'no-constant-condition: error' \
--rule 'no-useless-return: error' \
2>&1 | tee /tmp/eslint.log
For patterns the linter doesn't catch (e.g., both branches of an if returning the same thing), use grep heuristics:
# Both-branch-returns-same — heuristic
# Find functions where the same identifier is returned in both branches
grep -rEnA15 "if .*:\s*$" src/ --include="*.py" | \
awk '/return / { print prev "\n" $0 } { prev = $0 }' | \
uniq -d
Phase 3 — Mixed-version-API hunt (Wrong Attribute, common case)
Pydantic v1/v2 mixing is endemic in vibe-coded Python:
# v1 patterns
grep -rEn "\.dict\(\)|\.parse_obj\(|class Config:" src/ --include="*.py"
# v2 patterns
grep -rEn "\.model_dump\(|\.model_validate\(|model_config\s*=" src/ --include="*.py"
# Both present in same project = Wrong Attribute waiting to happen
SQLAlchemy 1.x/2.x mixing is also common:
# 1.x async style (deprecated)
grep -rEn "create_async_engine.*sessionmaker.*class_=AsyncSession" src/
# 2.x async style (correct)
grep -rEn "async_sessionmaker" src/
Phase 4 — Manual confirmation pass
The static analysis surfaces candidates. For each candidate that's NOT obviously a false positive, read the actual code to confirm (R2):
- Open the file at the cited line
- Verify the pattern is real
- Classify as Hallucinated Object / Wrong Attribute / Silly Mistake
- Assess severity:
- Test file only: Medium
- Utility code: High
- Critical-flow code (auth/payment/delete): Critical
Drop candidates that turn out to be:
- Type stub gaps (real library, missing types)
- Intentional dead code (clearly marked with comment)
- Test fixtures that look weird but are deliberate
Output format
═══════════════════════════════════════════════════════════════════════
TAMBON SIGNATURE HUNT
═══════════════════════════════════════════════════════════════════════
Scan scope: <path>
Total LoC: <N>
Tooling used:
Static type checker: <mypy / pyright / tsc / NONE — finding>
Linter: <ruff / pylint / eslint / NONE — finding>
────────────────────────────────────────────────────────────────────────
SIGNATURE 1 — Hallucinated Object
────────────────────────────────────────────────────────────────────────
Total occurrences: <N>
Top occurrences (severity-sorted):
F-T1.1 — <path:line>
Pattern: <what was hallucinated>
Severity: Critical | High | Medium
Evidence:
<the exact code line>
Fix: <one sentence>
F-T1.2 — <path:line>
[same shape]
[Up to 10 top findings; if more, append]
[+ <N> additional Hallucinated Object findings — see appendix]
────────────────────────────────────────────────────────────────────────
SIGNATURE 2 — Wrong Attribute
────────────────────────────────────────────────────────────────────────
Total occurrences: <N>
[Same shape]
────────────────────────────────────────────────────────────────────────
SIGNATURE 3 — Silly Mistake
────────────────────────────────────────────────────────────────────────
Total occurrences: <N>
[Same shape]
────────────────────────────────────────────────────────────────────────
TAMBON DENSITY
────────────────────────────────────────────────────────────────────────
Total signatures: <H + W + S>
Total LoC: <N>
Density: <total / N * 1000> per 1000 LoC
Density band:
[ ] 0–1 per 1000 LoC: code is reviewed
[ ] 2–5 per 1000 LoC: light review
[ ] 6–15 per 1000 LoC: most code unreviewed
[ ] >15 per 1000 LoC: AI-as-shipped
Critical-path findings (in auth/payment/delete code):
<count> findings
[list each with path:line if any]
────────────────────────────────────────────────────────────────────────
INTERPRETATION
────────────────────────────────────────────────────────────────────────
[2-3 sentences. Density band → what it implies about the codebase →
recommended response. Examples:]
"Tambon density of 8.3/kLoC places this codebase in the 'most code is
unreviewed AI output' band. Combined with the 4 Critical findings in
the auth path, this suggests a substantial portion of the
authentication code was generated and never line-reviewed by a human."
"Tambon density of 0.4/kLoC indicates the code has been line-reviewed
thoroughly. The 2 occurrences are isolated and don't cluster in any
specific module."
[SECTION COMPLETE: audit-tambon-hunt]
What this skill does NOT do
- Fix the findings (that's the dev's job, with /audit-fix)
- Run the code (audit is read-only)
- Speculate about which model generated the code (irrelevant; the density band is what matters)
- Skip occurrences in test files (test code matters too — broken tests give false confidence)
Failure modes to refuse
- ❌ Reporting Tambon density without running a type-checker (the density number requires the static analysis)
- ❌ Counting type-checker errors that are actually missing-stub issues as Tambon findings
- ❌ Calling something "Hallucinated Object" without verifying the package/method actually doesn't exist (R2 — quote before cite)
- ❌ Producing a 50-finding list of trivial Silly Mistakes that obscures the 3 real Critical-path Hallucinated Objects (severity sort matters)