# Repo Checks > How the nine checks work and how to add one — why the runner is Python rather than shell, the seven times a mistyped construct reported clean for a check that never ran, and the fault-injection standard. Use when adding or changing a check, or when one fails. - Skill: `nginx/repo-checks` (Agent Skill) - Install (CLI): `npx skillmds@latest add nginx/repo-checks` - Raw SKILL.md: https://api.skillmd.com/api/skills/nginx/repo-checks/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - Author: nginx (https://skillmd.com/u/nginx) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/nginx/repo-checks --- # The checks, and how to add one One command, no dependencies: ```bash python3 .github/scripts/check-all.py # -v to see every check's full output ``` It runs all nine and prints **how many ran**, which is the number that matters. ## Why a runner, and never a shell line A mistyped shell construct reports "clean" for a check that never ran. That has now happened **seven** times in this repository: 1. A `$F` that expanded to one filename. 2. Broken `grep -c` arithmetic. 3. A zsh glob swallowing `--include`. 4. `for c in "python3 …"; do $c; done` — zsh treated each whole string as one command name, printed `exit=0` four times, and ran nothing. 5. `$b:` followed by a path — zsh parsed it as a history modifier, so `$b:assets/…` silently became `mainssets/…`. 6. `node --check assets/js/*.js`, which parses only the **first** glob match. That one sat in CI for two months, so `shared.js` had no syntax coverage. 7. `python3 … | head -20; echo "(exit $?)"` — `$?` is `head`'s status, not the script's, so a check that correctly exited 1 was reported as exit 0. If you must capture status through a pipe, zsh is `${pipestatus[1]}`, not `$?`. `check-all.py` is safe to trust for three specific reasons, and if you change it, keep all three: - It is **Python, not shell** — no globbing, word-splitting or history expansion happens between the declaration and the process. - Each check is a literal `(label, argv)` tuple, never a string to be parsed. - It **asserts it ran every check it declared** and prints the count, so "9 of 9 checks ran" is part of the output rather than something you infer from the absence of errors. A missing interpreter or script is a **failure**, never a skip (`FileNotFoundError` → exit 127). The same shape is why the CI test step names its glob and guards on `find`: `node --test` exits 0 when it matches no files, so a moved directory would turn it into a green no-op. ## The fault-injection standard **A new assertion is not finished until you have watched it fail.** Plant the defect it is meant to catch, confirm the check reports it and exits non-zero, then restore. Record what you planted and what it said in the commit body — "planted a misspelled transform; the run reported it and exited 1" is worth far more later than "fix check". Two traps, both hit repeatedly here: - **Injections that silently do not match.** `perl -pi -e` and blind `str.replace` happily change nothing and leave you verifying an unmodified file. Use Python with `assert new != original` before writing. - **Injections that are not actually defects.** Replacing *both* a sidebar label and its heading leaves them still agreeing, so the label check correctly stays green and you conclude, wrongly, that it is broken. Likewise a character you add to test font coverage must genuinely be outside the subset — ★ U+2605 and ∞ U+221E are; most punctuation is not. Interactive `cp`/`rm`/`mv` aliases will hang a non-interactive shell until it times out. Save and restore through Python, or call `/bin/cp` directly. ## What the nine can and cannot see | Check | Asserts | |---|---| | `check-syntax.py` | every script parses — one `node --check` per file, never a glob | | `check-tokens.py` | token invariants, retired colours/typefaces, undefined `var()`, inline `style=`, type pairing, the manifest, webfont coverage | | `check-contrast.py` | every pairing against WCAG 2.1 AA in both themes, plus pairings derived from the CSS, plus the measured-ratio comments | | `check-classes.py` | classes resolve to rules; load order, asset paths, sitemap and navigation labels hold | | `check-versions.py` | every version string agrees with its source of truth | | `check-markup.py` | tag balance, duplicate ids, anchors, JSON-LD | | `test-analyzer.js` | the migration analyzer under a DOM stub; regenerates `mapping-index.json` | | `node --test .github/test/` | page ↔ engine ↔ module wiring | | `where.py --self-test` | the locator's resolvers still match, by kind not line number | Five things to know: 1. **None of them can see the rendered page.** Every one is a static reader, so the entire class of visual defect — a stretched grid, a collapsed flex item, a truncated label, a card wrapping 3+1 — passes all of them green. A clean run means "nothing is structurally broken", not "it looks right". Use the `verify-visually` skill. 2. **`check-contrast.py` derives pairings from the stylesheets** as well as asserting a hand-written list, so a new coloured surface is measured without anyone remembering to add it. Two limits remain: it only sees pairs declared through tokens in the same rule (or a dark override of one), and it cannot know which text is large enough for the 3:1 bar, so it holds everything to 4.5:1. It is quiet on success — pass `-v` for all 58 measurements. 3. **`check-classes.py` matters most after a restyle.** A class that loses its rule does not error; the element just renders unstyled, which is invisible on a page with thousands of rows. It reports unused classes but never fails on them: there is dormant-by-design CSS here, listed by name in the script's own `DORMANT` set — the event banner and the ingress2gateway annotation grid, both built ahead of their content. Run `git log -S` before deleting anything on that list. 4. **The checks live under `.github/` because Pages publishes this branch.** A top-level `scripts/` was being served (`/scripts/check-tokens.py` returned 200); dot-directories 404, because Jekyll runs on this branch and skips dot-prefixed paths. **There is no `.nojekyll` and adding one would publish `.github/` wholesale** — it disables Jekyll rather than configuring it, so the dot-prefix exclusion goes with it. Anything else that must not be served goes under `.github/` too, which is why the test suite is at `.github/test/`. Each script and the test loader derive `ROOT` by walking up from their own path, so moving one means fixing that. 5. **`.github/workflows/tests.yml` runs the same nine**, one step each, on every push and pull request. Pushing to `main` is deploying and CI finishes at about the same time the deploy does, so a red run does not stop a bad commit reaching production. Verify before you push. ## Adding a check 1. Write it as a standalone script under `.github/scripts/` that exits non-zero on failure and derives `ROOT` from its own path. 2. Make it **quiet on success** — print a one-line summary that includes the counts it asserted, so "OK" can never mean "measured nothing". Put the detail behind `-v`. 3. Add a literal `(label, argv)` tuple to `CHECKS` in `check-all.py`. 4. Add a step to `.github/workflows/tests.yml`. 5. Fault-inject it, per the standard above, and write what you planted into the commit body. ## Adding a test file Not the same as adding a check — a new `.github/test/*.test.js` joins the existing "wiring suite" entry rather than becoming a tenth check. Register it in **both** places or it runs in only one of them: - `.github/workflows/tests.yml` globs `.github/test/*.test.js`, so CI picks a new file up on its own. - `check-all.py` lists the files by name, so a local run silently uses the smaller suite until the name is added there too. The asymmetry is deliberate — the glob is guarded against matching nothing, and a literal list is what keeps `check-all.py` honest about how many things ran — but it means the two can disagree without anything saying so.