Running Check Script

Running the check Script

This skill is the single source of truth for executing the package.json check script in lt-dev workflows. Every reviewer, rebaser, and orchestrator that needs to guarantee project runnability must follow this procedure verbatim — duplicating the rules across agents leads to drift.

Goal: A truly green check run (exit 0) is a non-negotiable prerequisite for any review or rebase to be considered complete. The only acceptable residual is an upstream dependency vulnerability where the full fix escalation ladder has been exhausted.

When to Use This Skill

Caller modes

The procedure is identical in both modes. Only what happens with an unresolved error differs, and that difference is the whole point of the distinction:

• Advisory — the caller's job is to report. A red check is surfaced as a Critical finding and the workflow continues, because the user still wants the rest of the review.
• Blocking — the caller's job is to ship. A red check stops the workflow. Nothing gets pushed, no MR/PR is created, nothing is merged.

Every path that leads to an MR/PR or a merge runs in Blocking mode. A red check reaching the base branch costs the whole team: CI goes red for everyone, the next take-ticket branches off broken code, and the person who eventually fixes it has no idea which change caused it. That is why the ladder in Step 3 has no iteration cap and why Step 5's residual list is as narrow as it is.

Procedure

Step 1 — Discover check scripts

Use the dedicated helper script (located in the lt-dev plugin):

bash "${CLAUDE_PLUGIN_ROOT}/scripts/discover-check-scripts.sh" "$(pwd)"

Output is TSV, one project per line:

<package.json path>\t<check script>\t<includes_tests:no|unit+api|yes>\t<package manager>

includes_tests values — callers decide what to do with each, the discovery itself prescribes no behavior:

The helper:

• Uses git ls-files "package.json" "**/package.json" so it respects .gitignore and skips node_modules
• Parses JSON via jq → node -e → grep+sed fallback chain (works even without jq installed)
• Resolves single-level composite scripts ("check": "pnpm run ci" → looks up ci)
• Detects the package manager per project via lockfile (pnpm-lock.yaml → pnpm, etc.; default pnpm)

If the helper returns nothing → no project defines check → skip the rest of this skill.

Step 2 — Run check per project

For each discovered project, cd into its directory and run the check script with the detected package manager:

cd "$(dirname <package.json path>)"
<package manager> run check

Capture stdout, stderr, and exit code. Track an iteration counter starting at 1.

Step 3 — Auto-fix loop (iterate until truly green)

If a project's check exits non-zero:

1. Parse all errors from the current run (typecheck, lint, build, missing imports, type mismatches, unused symbols, audit findings, etc.)
2. Fix every error at the root cause via Read + Edit. Pre-existing errors are fixed too — whether an error came from this diff or was already there makes no difference to whether the project runs, and a red check blocks the next person exactly as hard either way. Scope boundaries do not apply to runnability. The one question that decides anything here is "can this be fixed?", and while the answer is yes, it gets fixed.
3. Re-run check from scratch. Never trust partial state.
4. Continue iterating until one of these terminal conditions:
   • (a) GREEN — check exits 0 → done for this project
   • (b) STALLED — a full iteration produced no net reduction in error count
5. No hard iteration cap. As long as each iteration strictly reduces the error count, keep going. The goal is true green.

One narrow exception, and it is about collision rather than scope. Point 2 stands: a pre-existing error gets fixed, because a red check blocks the next person regardless of who wrote it. What point 2 does not cover is an error inside uncommitted work another live session is mid-slice on — reshaping that is not a scope question, it is two sessions editing one file, and the peer returns to a rewritten refactor having lost more time than the fix saved. Where the tree is dirty and bash ${CLAUDE_PLUGIN_ROOT}/scripts/change-provenance.sh attributes the failing path to a live peer: fix it if it is trivial and self-contained (a missing import, a format violation, a stray console.log), and otherwise report it as a blocker naming the author, plus one SOLVED carrying the diagnosis so they can fix their own work in one step. Nobody live, or nothing attributable: point 2 applies unchanged.

STALLED is a signal to change approach, not permission to stop. The count stops falling when the same fix is being retried, so the next iteration has to attack the error differently: read the failing file in full rather than patching around the reported line, check whether two errors share one cause, look at how a working sibling module solves the same thing, run the failing step alone to see output the aggregate run swallowed, check whether it is one of the known hazards in Steps 6.5 to 6.8. Only when a genuinely different approach has been tried and the count still holds does the residual classification in Step 5 apply — and in Blocking mode, everything except an exhausted-ladder dependency vulnerability still stops the workflow.

Step 4 — Audit findings: mandatory fix escalation ladder

When pnpm audit / npm audit / yarn audit (invoked by check) reports a vulnerability, you MUST exhaust this ladder before classifying the finding as Accepted. Re-run check after every step.

A finding may only be classified as Accepted after every applicable step has been tried and verified, with documented evidence that no patched version exists anywhere in the ecosystem.

CI parity — the local audit MUST match the CI security gate. A project's local check may run pnpm audit at a lower severity threshold (or narrower scope) than the CI gate — e.g. local pnpm audit --prod --audit-level=critical while a CI job runs pnpm audit --prod --audit-level=high (allow_failure: false). A green local check then hides findings that fail CI: the pipeline goes red on a "pre-existing" HIGH CVE the local loop never even surfaced. Before trusting a green local check, confirm its --audit-level and --prod/scope match the strictest audit gate in .gitlab-ci.yml / .github/workflows. If they diverge, raise the local check audit-level to match CI (so the local loop becomes the single source of truth that catches exactly what CI enforces), then run the ladder above on whatever new findings surface. An audit-level mismatch is a silent local↔CI parity bug — never an Accepted Residual.

Step 4a — Claim a cross-cutting finding before fixing it

An audit finding is not this branch's problem. It is the repository's, so every parallel session running check sees the same one and, left alone, every one of them walks the ladder above. That is wasted work in the best case; in the normal case it is four sessions rewriting the same lockfile and four rebase conflicts nobody needed. The same holds for any other finding that belongs to the repo rather than to the diff: a broken CI config, a corrupted lockfile, a broken shared test setup, an audit-level mismatch with CI.

So before starting the ladder on a finding of that kind:

1. Read the ledger. bash ${CLAUDE_PLUGIN_ROOT}/scripts/peer-ledger.sh read costs nothing and disturbs nobody. It also answers what a message cannot: whether a session that is no longer running already claimed this, and whether somebody already recorded the cause.
   • The finding shows as [held] → do not fix it. Record it as claimed in the report block and carry on with the rest of check.
   • It shows as [stale] → the claiming session is gone. Take it over.
   • A note already names the cause → start from it instead of diagnosing it again.
2. Claim it: peer-ledger.sh claim "audit:GHSA-xxxx" "<what it moves>". The command refuses a topic another live session holds and names that session, so the claim is decided rather than merely announced.
3. Then ListAgents. A live peer in this repository that is about to run check gets one CLAIM message on top (format in coordinating-peer-sessions). The ledger already covers everyone else, including sessions that start later.
4. Fix it, then peer-ledger.sh release naming the ticket or commit that carries the fix, so the topic frees up for good rather than only until this session closes.

If a CLAIM for the same finding arrives from a peer first, do not fix it. Classify it as claimed in the report block, name the peer, and carry on with the rest of check. Contest the claim only when you are already mid-ladder on it, and then one of the two backs off; two sessions on one lockfile is the outcome the claim exists to prevent. None of this reaches the user: who fixes a shared finding is coordination, and the ledger settles it.

This does not soften Step 5. A claimed finding is not an Accepted Residual and not a reason to call check green: it is still open, it is just open in a different session. If it blocks this branch's gate, the session waits for the peer (one ASK, or notify_when_idle) rather than fixing it in parallel.

The other direction is worth more than the claim. Most check failures are the repository's, not the diff's, so the diagnosis you just paid for is one a peer is about to pay for again. When you find the cause of something environmental or shared (a shared test database, a toolchain or SWC config, an audit-level mismatch with CI, a check-server-start port hazard), record it with peer-ledger.sh note "<topic>" "cause: … fix: …" and send one SOLVED to any peer live right now. The note is the part that lasts: it turns the next session's half hour into two lines, including a session that starts long after yours has closed. Record the cause, not the symptom.

Step 5 — Residual classification

Only after a project has STALLED (and, for audit findings, only after the escalation ladder is exhausted):

Step 6 — Bypass policy (hard rules)

Never use any of the following to silence errors:

• git commit --no-verify
• @ts-ignore, @ts-expect-error, @ts-nocheck
• eslint-disable, eslint-disable-next-line, eslint-disable-line
• /* istanbul ignore */
• Lint rule downgrades in config files
• Commenting out broken code
• Deleting failing tests
• it.skip(...), describe.skip(...), test.skip(...) in test files
• --passWithNoTests flags on the test command
• Adding oxlint-disable directives to suppress real findings

The ONLY permitted "non-fix" is an upstream dependency vulnerability where the escalation ladder has been fully exhausted. Everything else must be fixed at the root.

Step 6.5 — check-server-start.sh failure modes (Nitro/Nest port hazards)

The starter check pipeline ends with bash scripts/check-server-start.sh, which boots the production build and waits for the readiness log. Three known failure modes — all surface as the same symptom (ERR_SOCKET_BAD_PORT from node:net) but have different root causes:

1. Nitro PORT-string bug (App side): the script must use NITRO_PORT=$FREE_PORT, never PORT=$FREE_PORT. Some Nitro versions read process.env.PORT without parseInt and crash; NITRO_PORT is the documented Nitro-specific knob, goes through Nitro's own env loader, and is coerced to number reliably. Nest does not have this issue — NSC__PORT is fine on the API side.

2. lerna/nx ANSI-injection (BOTH api and app, only when check is invoked from a workspace runner): the runner wraps subprocess stdout and may inject ANSI color escape sequences (\x1b[33m...\x1b[39m) into command output. A naive FREE_PORT=$(node -e "...console.log(p)") captures the codes too. A naive tr -cd '0-9' makes it worse — the codes contain digits (33, 39) themselves, producing nonsense ports like 335454639. The only correct fix is to strip the ANSI sequence pattern explicitly with sed:

   FREE_PORT=$(node -e "..." | sed $'s/\x1b\\[[0-9;]*m//g' | tr -d '[:space:]')
   

3. Phantom Unix-domain-sockets named [33m12345[39m next to the package.json (mode srwx): leftover from earlier failed runs. When Nest's port-parser fell through "string with weird chars" → "treat as Unix socket path", it actually bound a socket file. cleanup() SIGTERM kills the process, the file stays. Delete with:

   rm -f $'\x1b[33m'*$'\x1b[39m'
   

   Then re-run check.

These hazards are documented in detail in the modernizing-toolchain skill (Phase 6). When a check run fails with ERR_SOCKET_BAD_PORT, the first triage step is to confirm the script in question already has both the NITRO_PORT and ANSI-strip fixes applied.

Step 6.6 — API tests fail only inside check? Suspect a shared test database

Symptom: check fails at api · test with one or two failures, but the very same suite is green when run on its own, and the failing spec moves between runs — a missing admin user in one run, a missing season in the next, always in a prepare / beforeAll step (Cannot read properties of null (reading '_id'), 404 on a record the setup just created).

This is not flakiness and not load. It is a second test run — from another working copy — dropping the database out from under this one. The API e2e global setup drops its database on start; if two checkouts resolve to the same database name, whoever starts second wipes the first one mid-run. Parallel checkouts are routine (lt ticket slots, a second clone, another agent session), so this hits often and reads as random.

Triage, in this order:

1. Run the failing suite alone. Green in isolation + wandering failure inside check ⇒ this bug, not a real defect. Do not "fix" the spec.
2. Check for a competing run: ps aux | grep [v]itest. A run in another directory of the same project is the smoking gun.
3. Confirm the database name is shared: it must contain a per-run (or at least per-working-copy) component. A constant like myproject-e2e is the bug.

The fix belongs in the project, not in the spec — and it very likely already exists upstream. nest-server / nest-server-starter give every test RUN its own database (<base>-e2e-run-<timestamp>-p<pid>, set in tests/global-setup.ts before the workers fork) plus a db-lifecycle.reporter.ts that drops it on green, keeps it on red for debugging, and collects the leftovers of crashed runs. A project still on a fixed database name has simply not adopted it — port the base-repo solution rather than inventing a local one (see the contributing-to-lt-framework skill).

Things to preserve when porting:

• The drop guard. A test setup that drops "whatever the URI points at" is dangerous, because a running lt dev session exports NSC__MONGOOSE__URI / MONGODB_URI pointing at the development database — a suite started in that shell would wipe the developer's data. Refuse to drop any database whose name does not look disposable (/(e2e|ci|test|acctest)/i).
• Derived databases. A spec that needs its own database must derive it from the run's database (deriveTestDbUri(...)), never invent a global name: a fixed name is shared by concurrent runs, and its leftovers are collected by nothing — an aborted run skips the spec's own cleanup, and the databases pile up (one real machine had accumulated ~140).
• The startup sweep + run governor (11.29.0+). tests/global-setup.ts additionally (a) sweeps stale leftover DBs of this project BEFORE the run (dead-PID/age guarded) — this is what survives SIGKILL and --reporter overrides, and (b) acquires a machine-wide e2e slot (tests/e2e-run-slots.ts, <tmpdir>/lt-e2e-run-slots) so concurrent check runs from parallel sessions queue instead of starving each other. A run printing [e2e-governor] waiting for a free e2e slot every 15s is QUEUED, not hung — do not kill it and do not misread it as a deadlock. The config also drops to low-resource mode (reduced forks) when another run is active.

Step 6.7 — api · test looks deadlocked (0% CPU, no output)? It is usually a retry grind

A spec file whose app/socket state broke (historically: resource starvation from overlapping runs) grinds through (1+retry) attempts × testTimeout × tests-per-file — with retry: 5 that was up to an hour at 0% CPU, which the watchdog kills as "workers idle". Base repos now ship retry: 2 and the run governor removes the starvation trigger. If you still see it: check pgrep -f vitest for a single surviving fork at 0% CPU, read which spec file last logged, and re-run that file alone. Never raise retry to paper over it.

Step 6.8 — check fails on a fully green test run? Console-intercept teardown race

Symptom: check fails at a test step while every single test passed:

EnvironmentTeardownError: [vitest-worker]: Closing rpc while "onUserConsoleLog" was pending
 Test Files  108 passed (108)   Tests  2300 passed   Errors  1 error

It is sporadic — roughly one run in ten — and a rerun is usually green, which makes it read as flakiness.

Cause. vitest replaces globalThis.console in every worker and ships each write to the main process as an onUserConsoleLog RPC call. On teardown, execute() does not await those calls, it rejects them. A console write from the last test in a file therefore races its own cleanup and loses. Structural, not load-dependent — which is why "rerun it" makes it disappear without fixing anything.

Only console.* can trigger this. vitest replaces globalThis.console and nothing else. A Nest Logger (and anything else writing straight to process.stdout/stderr) never becomes an onUserConsoleLog and cannot cause the race — so in an API suite the loudest lines are usually the irrelevant ones, and silencing the logger is the obvious move that does nothing. Hunt for console.* instead; in nest-server-starter the entire problem was one line, dist/core/common/helpers/config.helper.js:78.

The switch is disableConsoleIntercept: true in the vitest config: it skips setupConsoleLogSpy(), after which rpc.onUserConsoleLog has exactly one caller left — sendLog() in the very console that is no longer installed. Confirm it in the installed version rather than trusting the docs:

grep -rn "disableConsoleIntercept" node_modules/vitest/dist/chunks/base.*.js
# vitest 4.x → if (!config.disableConsoleIntercept) await setupConsoleLogSpy();

Never set it prophylactically. Measure first — and measure by diffing two runs, not by grepping one.

vitest run > /tmp/int-off.log 2>&1     # flag off
# flip disableConsoleIntercept in the CONFIG FILE — a CLI override does
# not win against a value set there — then run again:
vitest run > /tmp/int-on.log 2>&1      # flag on
diff /tmp/int-off.log /tmp/int-on.log

The difference between the two runs is the answer, for exposure and cost alike: everything the flag adds or removes shows up there and nowhere else. A diff that is empty bar timestamps and durations means there are no console writes at all — the flag is inert and the race cannot occur in that suite.

A grep is a quick pre-check, never the verdict:

vitest run 2>&1 | grep -cE '^(stdout|stderr) \| '     # lower bound only

It can only ever undercount, and it has already lied in two independent ways in this stack: the default reporter swallows passing tests' output in a piped run, and a content pattern tuned to one repo's wording (Configured for|No local config) reported 1 where 8 lines were waiting — seven of them phrased differently. Every repo words its logs its own way, so a pattern that catches everything in one catches a third of it in the next.

Check that the run executed tests at all before believing any number. A run that dies on startup — an unresolvable plugin, a config pointing at the wrong root — prints no console lines either and reports a clean 0. Read the test-file and test counts, not just the tail.

At this measurement, "nothing found" looks identical whether nothing was there or the instrument was not looking. Three variants of that have already cost time here: the reporter that swallows passing output, the crashed run that reports 0, and the pattern that was too narrow. All three say "nothing" where something is.

Exposure 0 on its own proves nothing — the grep measures what the reporter prints, not what the workers write. Only cost 0 establishes that no writes happen, and only then may a report claim the race is structurally impossible.

Cost runs in both directions, so it has to be measured per suite rather than reasoned about. Measured across the base repos:

In a unit config the flag tends to make output appear (without the intercept the reporter can no longer swallow the output of passing tests); in an e2e config it tends to make output shorter (the intercept prints an extra header line per write). Same flag, opposite balance. Record a range wherever the count moves between runs — three of nest-server's lines are the same rate-limit warning, and how many appear depends on how many counters a run happens to create. A single number invites the next reader to treat ordinary variance as a regression and go hunting for it.

The e2e row deliberately gives a direction instead of a count: its figures come from a file comment (vitest-e2e.config.ts:96) that nobody re-ran, and it is not even clear whether they covered the whole suite or a single spec. Where a number has not been measured, state the direction and stop. The direction is what carries that row anyway — an e2e config is where the balance inverts.

The nest-server-starter unit row is the cautionary one, and it is worth reading twice. It was first recorded as "exposure 0" because the grep said so — and that reading was the artefact this step exists to prevent: 114 console.info writes per run had been going over the RPC the whole time, invisible because the reporter swallows passing tests' output in a piped run. What settled it in the end was neither the flag nor any config line, but a single vi.spyOn(console, 'info') beside the one spec that produced them. Exposure and cost both read 0 there today because the writes are gone, not because they never existed.

Where the flag belongs: in the project that shows the symptom, never one level up in a starter or template. A config line in a starter travels into every generated project, where it costs attribution on day one — lines lose their stdout | <file> prefix and interleave under parallel runs — while paying off only in the few projects that actually have the symptom.

silent: true is not a middle ground. Without the intercept vitest no longer owns the console and has nothing left to suppress.

If the lines turn out to be unwanted, clean up the logs instead of setting the flag. A targeted vi.spyOn(console, 'debug').mockImplementation(() => {}) in the test setup silences only the noisy channels and leaves every other file its attribution — that is exactly why nuxt-base-template measures 0 on both numbers. Keep the flag only where the lines are wanted diagnostics (an e2e suite that gets read on failure) and the cost balance is favourable.

Step 7 — Test-duplication avoidance

Tests must not run twice if check already covered them on an unchanged working tree.

After each project completes Step 3 with a GREEN result, record a post-check baseline:

git -C <project-dir> rev-parse HEAD                 # commit baseline
git -C <project-dir> status --porcelain             # working-tree baseline

A subsequent test phase (e.g. branch-rebaser Phase 7, code-reviewer Phase 5, test-reviewer) may skip running tests for a project when ALL of the following hold:

1. The discovery output marked the project's check as covering the test category in question (see table below)
2. The project ended Step 3 in GREEN status (or YELLOW with only Accepted Residuals)
3. git rev-parse HEAD matches the baseline (no new commits)
4. git status --porcelain matches the baseline (no working-tree changes since)

Skip eligibility per includes_tests value:

If any condition fails → run tests as normal.

Step 8 — Report block

Every caller must include this block in its final report:

### Check Script Results
| Project | Script | Iterations | Initial Errors | Auto-Fixed | Accepted | Final Status |
|---------|--------|------------|----------------|------------|----------|--------------|
| projects/api | pnpm run check | 3 | 7 | 6 | 1 | ⚠️ (1 accepted) |
| projects/app | pnpm run check | 1 | 0 | 0 | 0 | ✅ |

**Fixes applied:**
- `projects/api/src/modules/user/user.service.ts:42` — Removed unused `LoggerService` import (pre-existing)
- `projects/api/src/modules/auth/auth.controller.ts:18` — Fixed implicit `any` on `req` parameter (introduced in diff)

**Check Script — Accepted Residuals** (escalation ladder exhausted):
- `projects/api`: `some-package@1.2.3` — GHSA-xxxx-yyyy-zzzz (Moderate)
  - Ladder steps tried: (1) `pnpm update` — no newer version, (2) `audit --fix` — no fix available, (3) `--force` — same, (4) next major doesn't exist, (5) no override target, (6) no maintained alternative
  - Evidence: registry shows latest = 1.2.3, no advisory fix listed

**Check Script — Unresolved** (Critical blockers):
- _(none)_

Step 9 — Gating

Both modes share the green case and split on the red one:

What a Blocking caller does on red: print which project failed, which errors remain, which fixes were attempted, and which approaches Step 3 already ruled out — then stop and hand it to the user. The branch stays local and intact, so nothing is lost and the work can resume the moment the cause is understood.

The green requirement covers every discovered project. A monorepo where projects/api is green and projects/app is red is red: CI runs both, so shipping on a partial green just moves the failure to the pipeline, where it costs a full round-trip instead of one more local iteration.

Skip Coordination Between Callers

When /lt-dev:review delegates a small diff to lt-dev:code-reviewer, the orchestrator has already executed this skill in Phase 1.5. To avoid a duplicate run, the orchestrator passes:

SKIP running-check-script — orchestrator already ran it. Pre-computed Check Script Results block: <paste full block from Step 8 verbatim>

The agent then skips Steps 1–7 and pastes the block verbatim into its report.

Related Elements