production-placeholder-audit
Find implementation artifacts that make production behavior less real than it looks: mocks, stubs, fake paths, test-only modules, inert adapters, placeholder branches, dummy credentials, and comments that imply future behavior while current code silently succeeds.
Critical Constraints
- Evidence first. Every finding needs a concrete file:line, quoted or summarized signal, and a reason it is reachable or relevant. Do not report vibes.
- Production reachability matters. Test fixtures under test-only paths are not findings unless they are imported by production code, packaged in a runtime artifact, selected by config, or exposed through a public path.
- Do not count harmless TODOs alone. A placeholder is actionable only when behavior is missing, fake, misleading, externally visible, or able to mask failure.
- Separate suspicious from confirmed. Keep weak signals in a "needs follow-up" section instead of presenting them as defects.
- Stay read-only by default. This skill audits and reports; it does not rewrite code or delete test helpers.
Workflow / Methodology
Phase 1: Map the production surface
Identify where production code enters and what gets packaged:
rg -n --hidden -S "main\\(|entry|router|routes|handler|server|export|module.exports|pub fn main|func main|package|deploy|build|dist|release" .
rg -n --hidden -S "include|exclude|files|testMatch|pytest|go test|cargo test|jest|vitest|mocha|webpack|vite|rollup|tsconfig|package.json|pyproject|Cargo.toml|go.mod" .
Record the directories that are production by default (src/, app/, cmd/, lib/, internal/, deployment configs) and the directories that are test-only by default (test/, tests/, fixtures/, __mocks__/, spec/). Treat generated and vendored code as out of scope unless it is explicitly shipped or edited in the project.
Phase 2: Scan for high-signal terms
Start broad, then narrow:
rg -n --hidden -S "\\b(mock|stub|fake|dummy|fixture|placeholder|todo|hack|temporary|not implemented|no-op|noop|sample|example)\\b" .
rg -n --hidden -S "(__mocks__|test-only|testonly|Mock[A-Z]|Fake[A-Z]|Stub[A-Z]|Dummy[A-Z]|Fixture[A-Z])" .
rg -n --hidden -S "(localhost|127\\.0\\.0\\.1|example\\.com|/tmp/|/dev/null|changeme|password123|api[_-]?key|token)" .
rg -n --hidden -S "(throw new Error\\(['\\\"]TODO|panic!\\(['\\\"]TODO|panic\\(['\\\"]TODO|return null|return nil|return None|return \\{\\}|return \\[\\])" .
Then inspect import sites and packaging configs for each suspicious symbol or file:
rg -n --hidden -S "SuspiciousName|suspicious/path|__mocks__|fixtures?" .
git ls-files
Phase 3: Classify the evidence
Use these finding classes:
| Class |
Confirming evidence |
| Production mock/stub |
Mock/fake/stub implementation imported, constructed, or selected by production entrypoints |
| Test-only leak |
Test helper, fixture, or __mocks__ module reachable from non-test code or shipped package config |
| Fake path/config |
Hard-coded fake URL, local path, dummy token, or sample endpoint in runtime config |
| Evidence-free placeholder |
Code claims success, returns empty/default data, or suppresses errors without a real implementation |
| Bypass branch |
Env flag, build mode, or feature toggle swaps real behavior for fake behavior in production-capable paths |
| Misleading artifact |
Docs, generated output, or API response advertises behavior that code does not implement |
Phase 4: Prove or dismiss each candidate
For each candidate, answer:
- Is it under production, test, generated, vendored, or deployment-only scope?
- What imports, config, route, command, package manifest, or build rule makes it reachable?
- What user-visible or system-visible behavior is fake, missing, or misleading?
- What minimal change would remove the risk: delete, gate, rename, move to tests, fail closed, wire real implementation, or document as intentional?
Dismiss candidates that are test-local, explicitly excluded from packages, behind a safe non-production gate, or clearly documented as an intentional simulation.
Output Specification
Return a markdown report:
## Findings
| Severity | Class | Location | Evidence | Reachability | Impact | Recommended fix |
|---|---|---|---|---|---|---|
| High | Production mock/stub | path/file.ext:42 | FakeClient returned for default config | imported by app entrypoint | real API calls never happen | inject real client by default; keep fake in tests |
## Needs Follow-up
- path/file.ext:10: suspicious placeholder, but production reachability is not proven.
## Excluded
- tests/fixtures/user.json: fixture is test-local and not packaged.
Severity guide:
- Critical: fake behavior can corrupt data, bypass security, report false success, or ship to customers by default.
- High: production path uses mock/stub/fake behavior or silently drops required work.
- Medium: production-capable config or branch can select fake behavior without strong guardrails.
- Low: misleading placeholder with limited blast radius and clear remediation.
Quality Rubric
Examples
Confirmed finding:
High | Test-only leak | src/client.ts:18 | imports "../tests/fixtures/users" | src/client.ts is exported by package.json "main" | production bundle returns fixture users | move fixture import behind test factory and fail closed when real source is absent
Not a finding:
tests/payment.mock.ts is under tests/, only imported by *.test.ts, and excluded from package files.
Troubleshooting
| Problem |
Cause |
Response |
| Too many TODO hits |
Text search is too broad |
Only keep TODOs tied to runtime behavior, false success, or missing implementation |
| Unsure if code ships |
Packaging/build config unclear |
Mark as needs follow-up and name the missing proof |
| Mock is intentional |
It is gated to tests or local dev |
Exclude it and cite the gate |
| Fake endpoint appears in docs |
Docs may be examples |
Report only if docs drive generated config or production defaults |
| Empty return is idiomatic |
Some APIs use empty collections safely |
Report only if it masks required behavior or contradicts contract |
See Also / References
review skill for broader PR/code review.security skill when fake credentials, bypasses, or auth placeholders affect access control.test skill when findings need regression tests.refactor or implement skills for follow-up fixes after findings are accepted.
1---2name: production-placeholder-audit3description: Use when finding mocks, stubs, fake paths, or placeholders leaking into production code. Triggers:4---5
6# production-placeholder-audit
7
8Find implementation artifacts that make production behavior less real than it looks: mocks, stubs, fake paths, test-only modules, inert adapters, placeholder branches, dummy credentials, and comments that imply future behavior while current code silently succeeds.
9
10## Critical Constraints
11
12- **Evidence first.** Every finding needs a concrete file:line, quoted or summarized signal, and a reason it is reachable or relevant. Do not report vibes.
13- **Production reachability matters.** Test fixtures under test-only paths are not findings unless they are imported by production code, packaged in a runtime artifact, selected by config, or exposed through a public path.
14- **Do not count harmless TODOs alone.** A placeholder is actionable only when behavior is missing, fake, misleading, externally visible, or able to mask failure.
15- **Separate suspicious from confirmed.** Keep weak signals in a "needs follow-up" section instead of presenting them as defects.
16- **Stay read-only by default.** This skill audits and reports; it does not rewrite code or delete test helpers.
17
18## Workflow / Methodology
19
20### Phase 1: Map the production surface
21
22Identify where production code enters and what gets packaged:
23
24```bash
25rg -n --hidden -S "main\\(|entry|router|routes|handler|server|export|module.exports|pub fn main|func main|package|deploy|build|dist|release" .
26rg -n --hidden -S "include|exclude|files|testMatch|pytest|go test|cargo test|jest|vitest|mocha|webpack|vite|rollup|tsconfig|package.json|pyproject|Cargo.toml|go.mod" .
27```
28
29Record the directories that are production by default (`src/`, `app/`, `cmd/`, `lib/`, `internal/`, deployment configs) and the directories that are test-only by default (`test/`, `tests/`, `fixtures/`, `__mocks__/`, `spec/`). Treat generated and vendored code as out of scope unless it is explicitly shipped or edited in the project.
30
31### Phase 2: Scan for high-signal terms
32
33Start broad, then narrow:
34
35```bash
36rg -n --hidden -S "\\b(mock|stub|fake|dummy|fixture|placeholder|todo|hack|temporary|not implemented|no-op|noop|sample|example)\\b" .
37rg -n --hidden -S "(__mocks__|test-only|testonly|Mock[A-Z]|Fake[A-Z]|Stub[A-Z]|Dummy[A-Z]|Fixture[A-Z])" .
38rg -n --hidden -S "(localhost|127\\.0\\.0\\.1|example\\.com|/tmp/|/dev/null|changeme|password123|api[_-]?key|token)" .
39rg -n --hidden -S "(throw new Error\\(['\\\"]TODO|panic!\\(['\\\"]TODO|panic\\(['\\\"]TODO|return null|return nil|return None|return \\{\\}|return \\[\\])" .
40```
41
42Then inspect import sites and packaging configs for each suspicious symbol or file:
43
44```bash
45rg -n --hidden -S "SuspiciousName|suspicious/path|__mocks__|fixtures?" .
46git ls-files
47```
48
49### Phase 3: Classify the evidence
50
51Use these finding classes:
52
53| Class | Confirming evidence |
54|---|---|
55| Production mock/stub | Mock/fake/stub implementation imported, constructed, or selected by production entrypoints |
56| Test-only leak | Test helper, fixture, or `__mocks__` module reachable from non-test code or shipped package config |
57| Fake path/config | Hard-coded fake URL, local path, dummy token, or sample endpoint in runtime config |
58| Evidence-free placeholder | Code claims success, returns empty/default data, or suppresses errors without a real implementation |
59| Bypass branch | Env flag, build mode, or feature toggle swaps real behavior for fake behavior in production-capable paths |
60| Misleading artifact | Docs, generated output, or API response advertises behavior that code does not implement |
61
62### Phase 4: Prove or dismiss each candidate
63
64For each candidate, answer:
65
661. Is it under production, test, generated, vendored, or deployment-only scope?
672. What imports, config, route, command, package manifest, or build rule makes it reachable?
683. What user-visible or system-visible behavior is fake, missing, or misleading?
694. What minimal change would remove the risk: delete, gate, rename, move to tests, fail closed, wire real implementation, or document as intentional?
70
71Dismiss candidates that are test-local, explicitly excluded from packages, behind a safe non-production gate, or clearly documented as an intentional simulation.
72
73## Output Specification
74
75Return a markdown report:
76
77```markdown
78## Findings
79
80| Severity | Class | Location | Evidence | Reachability | Impact | Recommended fix |
81|---|---|---|---|---|---|---|
82| High | Production mock/stub | path/file.ext:42 | FakeClient returned for default config | imported by app entrypoint | real API calls never happen | inject real client by default; keep fake in tests |
83
84## Needs Follow-up
85
86- path/file.ext:10: suspicious placeholder, but production reachability is not proven.
87
88## Excluded
89
90- tests/fixtures/user.json: fixture is test-local and not packaged.
91```
92
93Severity guide:
94
95- **Critical:** fake behavior can corrupt data, bypass security, report false success, or ship to customers by default.
96- **High:** production path uses mock/stub/fake behavior or silently drops required work.
97- **Medium:** production-capable config or branch can select fake behavior without strong guardrails.
98- **Low:** misleading placeholder with limited blast radius and clear remediation.
99
100## Quality Rubric
101
102- [ ] Each finding has `file:line`, class, evidence, reachability, impact, and fix.
103- [ ] Findings distinguish confirmed defects from weak signals.
104- [ ] Test-only code is not reported unless production reachability is proven.
105- [ ] Placeholder findings describe current behavior, not just a TODO string.
106- [ ] Fake paths/config findings identify where the value is loaded or shipped.
107- [ ] The report includes explicit exclusions for obvious false positives.
108- [ ] Recommendations are actionable and scoped to the smallest safe change.
109
110## Examples
111
112Confirmed finding:
113
114```text
115High | Test-only leak | src/client.ts:18 | imports "../tests/fixtures/users" | src/client.ts is exported by package.json "main" | production bundle returns fixture users | move fixture import behind test factory and fail closed when real source is absent
116```
117
118Not a finding:
119
120```text
121tests/payment.mock.ts is under tests/, only imported by *.test.ts, and excluded from package files.
122```
123
124## Troubleshooting
125
126| Problem | Cause | Response |
127|---|---|---|
128| Too many TODO hits | Text search is too broad | Only keep TODOs tied to runtime behavior, false success, or missing implementation |
129| Unsure if code ships | Packaging/build config unclear | Mark as needs follow-up and name the missing proof |
130| Mock is intentional | It is gated to tests or local dev | Exclude it and cite the gate |
131| Fake endpoint appears in docs | Docs may be examples | Report only if docs drive generated config or production defaults |
132| Empty return is idiomatic | Some APIs use empty collections safely | Report only if it masks required behavior or contradicts contract |
133
134## See Also / References
135
136- `review` skill for broader PR/code review.
137- `security` skill when fake credentials, bypasses, or auth placeholders affect access control.
138- `test` skill when findings need regression tests.
139- `refactor` or `implement` skills for follow-up fixes after findings are accepted.
majiayu000