veegee82 Drift-Detection — outer loop (∂L/∂method), periodic upstream check
The Metaphor
The geologist, not the meteorologist. The meteorologist watches yesterday's weather. The geologist reads the erosion patterns that took decades to form — features no single rainstorm could have produced. Cumulative drift is geological: twenty reasonable commits, each individually innocent, erode the codebase's coherence. No per-commit gate can catch it. Looking for drift requires a different timescale and a different instrument — the aggregate, not the diff.
Overview
In Gradient Descent for Agents, this skill is the outer-loop sibling of method-evolution: method-evolution moves m = skills/rubrics in response to patterns; drift-detection surfaces the patterns worth moving against. Together they own the method axis. The complementary per-commit regularizer is docs-as-definition-of-done — it prevents drift being introduced; this skill finds drift that snuck through anyway.
Drift is divergence on a timescale longer than one work session. Twenty individually-reasonable commits can compose into a system whose mental model no longer matches the code. Per-commit gates (docs-as-definition-of-done) cannot catch it because no single commit violates a rule — the violation is the cumulative pattern.
Core principle: drift accumulates silently. Detecting it requires a periodic full-system scan, not per-edit vigilance. The scan produces a finite list of drift indicators; each becomes a candidate for immediate fix or a method-evolution step.
See ../../docs/ldd/convergence.md §4 for the drift taxonomy.
When to Use
Invoke on a periodic cadence, not per-commit:
- Weekly on active projects (bundled with release-candidate review)
- Before a major version bump or breaking-API release
- After onboarding/offboarding a contributor (someone leaves; check for orphaned mental models)
- After a long sprint of many small commits in one area
- Before a documentation overhaul (know what's wrong before you rewrite)
- After a refactor that touched >20 files
Do not use:
- On every commit (that's
docs-as-definition-of-done's job; this skill is slower and more expensive) - On small projects with few contributors (drift is a large-codebase / many-contributor pathology)
- During active development (it generates noise, you'll ignore it; run it at stable points)
The Seven Drift Indicators
Each is concrete and checkable. Score each as found / not found; the report is the union of all findings.
1. Identifier drift
Symptom: the same concept named two or three different ways in different modules.
Detection: grep for known concept synonyms: user_id|userId|uid|user; request_id|reqId|rid; total|sum|aggregate. Flag any where 2+ forms exist in the same codebase.
Example: user_id in the DB layer, userId in the API response types, uid in the logging code. Same thing, three names.
2. Contract drift
Symptom: the same logical interface has subtly different shapes at different call sites.
Detection: for a function/method called from ≥ 3 places, compare the argument lists / types / return handling. Flag inconsistencies.
Example: authenticate(user, password) in one module, authenticate(username=user, password=password, otp=None) in another, authenticate({...credentials}) in a third. Drift happened when optional params were added without backporting call sites.
3. Layer drift
Symptom: files that used to live in one architectural layer now import from two. The module-import graph has grown cycles or illegal cross-layer edges.
Detection: generate the import graph. Compare against the architecture doc's intended layer structure. Flag edges that cross documented layer boundaries or form cycles.
Example: domain/user.py started importing from persistence/postgres.py to avoid a conversion layer. Three weeks later, the domain layer depends on Postgres, silently.
4. Doc-model drift
Symptom: README architecture diagrams / concept descriptions / module listings describe a system that no longer exists.
Detection: re-generate the module dependency graph. Diff against the documented architecture. Flag any mismatch.
Example: README shows 4 services; the repo contains 7. README describes a message queue; there isn't one anymore. The 4-service / no-queue narrative was true 6 months ago.
5. Rubric drift
Symptom: evaluation rubrics / test assertions have been edited without a logged Δloss_method or justification.
Detection: git log on rubric-carrying files (evaluation.md, tests/fixtures/*/rubric.md, top-level test-expectation files). Every change should have a commit message explaining why AND a measured impact. Flag edits without.
Example: rubric.md used to require "all 5 layers named." It now requires "at least 3 layers named." No commit message explains the relaxation.
6. Test/spec drift
Symptom: tests (behavior) and specs/docs (promises) have diverged. Specs assert things tests never check, or tests enforce behaviors specs don't mention.
Detection: enumerate documented promises (spec-level invariants, API contract guarantees). For each, grep tests. Flag promises without tests and tests without spec-level justification.
Example: Spec says "idempotent-by-order-id." No test verifies idempotency. Tests exhaustively check rate-limiting that the spec never mentions.
7. Defaults drift
Symptom: the "default value" of a configuration key is stated differently in README, in code, in tests, in migrations.
Detection: for each documented default, grep all surfaces (README, source, tests, migrations, example configs). Flag any where values differ.
Example: README says default timeout is 30s. config.py has DEFAULT_TIMEOUT = 60. Test fixture uses 45. Migration sets 30. Four answers.
The scan — what it produces
A drift scan produces one artifact: a drift report with one row per indicator found. Not a "health score." Not a green checkmark. A list of specific, fixable issues.
# Drift scan — 2026-05-04
## Identifier drift
- `user_id` (38 occurrences) vs `userId` (12 occurrences). Modules using `userId`: api/, frontend/.
## Contract drift
- `parse_date(s)` in utils/dates.py vs `parse_date(s, fmt=None, tz=None)` in integrations/google/ — second form unused by utils callers.
## Layer drift
- `domain/user.py:12` imports `persistence.postgres.Session` (crosses documented boundary).
## Doc-model drift
- README describes 4-service architecture (§2); repo contains 7 services (services/).
## Rubric drift
- `evaluation.md` commit 0ed546c edited Δloss_bundle target from 3.0 to 2.0 without recorded reason.
## Test/spec drift
- `spec/api.md` promises idempotency on POST /charges; no test covers a duplicate charge.
## Defaults drift
- `DEFAULT_TIMEOUT`: README=30, code=60, test=45, migration=30.
Each row is either (a) immediately fixable — handle like any bug, invoke root-cause-by-layer and fix; or (b) a pattern suggesting the method itself allowed the drift — candidate for method-evolution.
Red Flags — STOP, the scan isn't working
- "The scan returned nothing" → either your project is tiny, your indicators are wrong, or you're scanning stale state
- "We'll fix these later" → unfixed drift compounds; schedule the fixes
- "The scan takes an hour, skip this cycle" → you saved an hour and lost the drift signal; next week's drift will include more
- "One indicator is too strict" → raise it in
method-evolution, don't silently drop the indicator - Reading the scan without acting on any row → you've turned the scan into theater
Automation surface (optional, Claude-Code-targeted)
scripts/drift-scan.py in the scripts/ directory runs the seven indicators as a single pass. Requires git, grep, and a Python 3.10+ environment. Not required for the skill itself — you can run the seven indicators manually. Provided for Claude-Code users who want to cron it.
Relation to other skills
docs-as-definition-of-doneprevents drift per-commit. This skill detects drift that accumulated anyway.root-cause-by-layerfixes each drift indicator (each indicator is a bug).method-evolutionhandles patterns in the scan — if indicators 3, 5, 6 all fire repeatedly, the method that allows this is the target of evolution.loop-driven-engineeringdispatches this skill at release-candidate time or on a scheduled cadence.
How to Apply — checklist
- Pick the scan cadence. Weekly is aggressive; biweekly is realistic for most projects.
- Run the seven indicators. Manually or via
scripts/drift-scan.py. - Read every row of the report. Not just the count.
- Triage. Immediate fix (single instance) vs method-evolution (pattern across multiple indicators).
- Schedule the fixes. Small issues: in the current sprint. Large issues: next sprint, with
loop-driven-engineeringdiscipline. - Log the scan. Even if the report is short. Scan history is outer-loop data.
Common Rationalizations
| Excuse | Reality |
|---|---|
| "We don't have drift, we review carefully" | Careful review prevents new drift; it does not detect existing drift. |
| "The scan is too slow" | Then automate a subset on every CI run; do the full scan monthly. |
| "Most findings are cosmetic" | Cosmetic drift is how non-cosmetic drift hides. |
| "Small team, we know the codebase" | Today you do. The next contributor / the next LLM session does not. |
| "The scan finds too much to fix" | Prioritize. A finding is not a commitment to fix today; it's an entry in the queue. |
Real-World Cue
If your project has >3 months of active development and you've never run a drift scan, your first scan will find things. That is not an indictment — it is the starting state. The scan's value is the trend over time: does the drift backlog shrink, stabilize, or grow? Growing drift is a method-evolution signal.