# Nn Audit

> Audit the micro-NNs — is each model grounded in REAL data, or a synthetic copy of a hand-coded rule? Scans skills/botte_nn and reports, per model, the training data source (real/synthetic/unknown), whether the model file records provenance (trained_on/eval_accuracy), whether a test guards a real-world output, and a grounded/synthetic verdict. Deterministic, 0 cloud tokens. Use to tell which learned components are real vs placeholder, and which should be grounded or replaced by the rule they imitate.

- Skill: `zedarvates/nn-audit` (Agent Skill, multi-file: 5 files)
- Install (CLI): `npx skillmds@latest add zedarvates/nn-audit`
- Raw SKILL.md: https://api.skillmd.com/api/skills/zedarvates/nn-audit/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- Author: zedarvates (https://skillmd.com/u/zedarvates)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/zedarvates/nn-audit

---


# nn_audit — grounded NNs, or synthetic copies of rules?

A learned component only earns its place if it's trained on **real data a rule
can't capture**. A net trained on `np.random` + hand-coded labels just
approximates a deterministic function we already wrote — strictly worse than the
rule (adds error + opacity, learns nothing). This audits which micro-NNs are real.

```bash
python -m skills.nn_audit.cli                 # audit skills/botte_nn
python -m skills.nn_audit.cli <dir> --json
```

Per model it reports:
- **data_source** — `real` | `synthetic` | `unknown`, inferred from the training
  script's content (distill/corpus/labelled → real; `np.random` → synthetic).
- **wired** — does any *production* file (not tests/training/registry) consume it,
  and which (`usage`)? Tells **synthetic-but-driving-behaviour** (real risk) apart
  from **synthetic-but-orphan** (dead weight).
- **has_provenance** — does the `.json` record `trained_on` / `eval_accuracy` / `data`?
- **has_test_guard** — does a test assert a specific real-world output for it?
- **verdict** — `grounded` | `synthetic — drives behaviour: ground it` |
  `synthetic + orphan: delete or wire` | `unknown`; **`at_risk` = synthetic AND wired**.

The fix is decided by both axes: a **wired synthetic** net (e.g. `binary_router`
in the routing belt) must be **grounded** on real data (distillation / active-
learning, like `error_classifier`); an **orphan synthetic** net is dead weight to
**delete or wire**. Exposed via [[llm_mcp]] as `nn_audit`; pairs with [[botte_nn]].
Pure file inspection, 0 cloud tokens.