nn_audit — grounded NNs, or synthetic copies of rules?
A learned component only earns its place if it's trained on real data a rule
can't capture. A net trained on np.random + hand-coded labels just
approximates a deterministic function we already wrote — strictly worse than the
rule (adds error + opacity, learns nothing). This audits which micro-NNs are real.
python -m skills.nn_audit.cli # audit skills/botte_nn
python -m skills.nn_audit.cli <dir> --json
Per model it reports:
- data_source —
real|synthetic|unknown, inferred from the training script's content (distill/corpus/labelled → real;np.random→ synthetic). - wired — does any production file (not tests/training/registry) consume it,
and which (
usage)? Tells synthetic-but-driving-behaviour (real risk) apart from synthetic-but-orphan (dead weight). - has_provenance — does the
.jsonrecordtrained_on/eval_accuracy/data? - has_test_guard — does a test assert a specific real-world output for it?
- verdict —
grounded|synthetic — drives behaviour: ground it|synthetic + orphan: delete or wire|unknown;at_risk= synthetic AND wired.
The fix is decided by both axes: a wired synthetic net (e.g. binary_router
in the routing belt) must be grounded on real data (distillation / active-
learning, like error_classifier); an orphan synthetic net is dead weight to
delete or wire. Exposed via [[llm_mcp]] as nn_audit; pairs with [[botte_nn]].
Pure file inspection, 0 cloud tokens.