Validation Authority
When integrating external validators (PSScriptAnalyzer, markdownlint, ESLint, etc.), respect upstream defaults. Modify local configuration to match upstream behavior. Do not modify upstream tool code.
Triggers
Say validation failed unexpectedly, should I change the validator, or
is this validator authoritative. Activate when:
- Validation fails unexpectedly
- Before modifying validator behavior or configuration
- When tempted to change upstream tool source code
- When adding a new external validator to the project
- When suppressing validator warnings without rationale
Decision Tree
Validation failure occurred
|
v
Is the validator upstream (external tool)?
| |
YES NO (local/custom)
| |
v v
Modify LOCAL config Modify tool as needed
to align with tool (you own the code)
|
v
Document override rationale
in config comments
Process
- Classify the validator: Determine if the tool is upstream (external) or local (project-owned).
- Identify the conflict: Find the specific rule or default causing the failure.
- Check upstream defaults: Read the tool's documentation for the default behavior.
- Align local config: Update project configuration files to match or explicitly override upstream defaults.
- Document overrides: Add comments explaining why any override differs from the upstream default.
Trigger Table
| Scenario |
Action |
Example |
| PSScriptAnalyzer rule fails |
Update .psscriptanalyzerrc.psd1 |
Suppress PSAvoidUsingWriteHost with rationale |
| markdownlint rule fails |
Update .markdownlint.yaml |
Disable MD013 line-length for generated docs |
| ESLint rule conflicts |
Update .eslintrc |
Override no-console for CLI tools |
| Upstream tool has a bug |
Flag the bug to the owner with a repro, add workaround in config |
Pin tool version, suppress specific rule |
| Tool default changed after upgrade |
Review and align local config to new default |
Update config after major version bump |
Adjunct Mode: Build Phase 2b
/build Phase 2b invokes this skill after analysis-provenance classifies
every path validation_trigger.py flagged as a validation target. This mode
writes the decision record; it does not replace the general decision-tree
guidance above, which still applies whenever a human asks how to treat an
upstream validator.
- For each target, combine
analysis-provenance's category, owner, and
evidence with a diagnosis of what actually happened:
implementation-defect (the local check has a real bug), local-config-defect
(only the project's own config needs to change), stale-generated-output
(a generated mirror drifted from its template or source), baseline-update
(a ratchet or baseline needs new entries), upstream-defect (the vendored
or upstream tool itself is wrong), relocation-no-semantic-change (a test
file moved or split under tests/ with the same collected tests; see
rule 9 below), or unknown (evidence is insufficient).
- Name the one
permitted_change_location Phase 3 may edit. GENERATED
always permits only the canonical source, never the mirror. VENDOR and
UPSTREAM never permit the target itself; point at the local override or
config instead. unknown diagnosis or UNKNOWN category permits nothing:
the record blocks further semantic edits until ownership evidence exists.
- For
baseline-update, cite the existing policy that authorizes the
refresh (baseline_justification.policy_source, a path that exists on
disk), state the reason, and justify every added entry individually.
- Write the record to
<record-path> (see "Validation Change Record" below
for where that resolves) and run
python3 "${COPILOT_PLUGIN_ROOT:-${CLAUDE_PLUGIN_ROOT:-.claude}}/skills/validation-authority/scripts/validation_record.py" --record <record-path> --changed-path <path>
for every changed path. Exit 0 clears Phase 3 to edit the permitted
locations; exit 1 names the blocking targets; exit 2 is a
configuration error in the record or the invocation.
Scripts
| Script |
Purpose |
Exit codes |
validation_record.py (this skill's script directory) |
Checks the Phase 2b decision record: field shape, provenance/authority rules, and changed-path coverage. Emits a JSON summary of target count, counts by category, defects, and checks (whether the trigger recheck ran, and how many targets real collection checked). |
0 record passed, 1 the record has defects, 2 config error (unreadable file, invalid JSON, the sibling build trigger could not be loaded or could not read its inputs when --changed-path needed it, or pytest is not installed) |
Validation Change Record
<record-path> above means: resolve a scratch directory the way
paths.artifact_dir does, take its scratch/ subdirectory, then
validation-authority-record.json inside it. That directory is agent PR
scratch space in the consumer's own workspace (git-ignored in this repo's own
checkout); the PR body carries the record's summary instead of a commit.
{
"trigger": { "...": "the validation_trigger.py output, carried through unchanged" },
"targets": [
{
"target": "<path>",
"component": "<human label for the tool or config>",
"provenance": {
"category": "LOCAL | GENERATED | VENDOR | UPSTREAM | UNKNOWN",
"owner": "<non-empty>",
"evidence": "<non-empty>",
"canonical_source": "<required, and equal to permitted_change_location, when category is GENERATED>"
},
"authority": {
"contract": "<non-empty; the validator's own exit-code or pass/fail contract>",
"diagnosis": "implementation-defect | local-config-defect | stale-generated-output | baseline-update | upstream-defect | relocation-no-semantic-change | unknown",
"permitted_change_location": "<non-empty; never the target itself for VENDOR or UPSTREAM>",
"escalation": "<required, non-empty, when diagnosis is upstream-defect>"
},
"baseline_justification": {
"policy_source": "<repository-relative path to an existing file; required when diagnosis is baseline-update>",
"reason": "<non-empty>",
"added_entries": [{ "justification": "<non-empty per entry>" }]
},
"relocation_evidence": {
"collect_command": "<required when diagnosis is relocation-no-semantic-change; the command that listed the tests>",
"moved_from": ["<test module path the tests left, non-empty list>"],
"moved_to": ["<test module path the tests landed in, non-empty list; includes this target>"],
"tests_before": ["<collected test id at the base, from the moved_from files only>"],
"tests_after": ["<collected test id after the move, from the moved_to files only>"]
}
}
]
}
Rules validation_record.py enforces:
- Every target needs
target, component, provenance, and authority.
trigger holds the trigger's JSON object, and its decision is activate
or skip. A skip that lists effects or targets is a contradiction. When
the trigger activated, the record needs at least one target, and each path
the trigger named needs a record target.
An activated trigger must carry an effects list; a record without one is a
defect. The validator cannot recompute effects from paths: they are
diff-verified declarations, and an empty list is trusted. With
--changed-path, the validator reruns the trigger on those paths and
refuses a skip the trigger would activate.
UNKNOWN category or unknown diagnosis is always a blocking defect: stop
semantic edits and request ownership evidence.
GENERATED needs provenance.canonical_source, and
authority.permitted_change_location must equal it.
VENDOR or UPSTREAM: authority.permitted_change_location must not
equal target. upstream-defect needs a non-empty authority.escalation.
baseline-update needs baseline_justification with a policy_source
that is a relative path to an existing file inside the repository, a non-empty reason, and a non-empty justification on
every entry in added_entries.
- With
--changed-path supplied: every path the sibling build trigger's
path cues alone would flag needs a record target; a VENDOR or UPSTREAM
target must not itself be a changed path; a changed GENERATED target
needs its canonical source changed too, never edited as a standalone
mirror.
- Paths compare after normalization:
./a/b.py, a\\b.py, and a/b.py are
the same target.
- One run reports every defect: field defects and coverage defects together.
relocation-no-semantic-change needs LOCAL provenance, a test module
target under tests/ (test_*.py or *_test.py, so conftest.py and
other support files are refused), and a relocation_evidence block with
moved_from and moved_to path lists. moved_to lists the target, and
the before and after ids must name exactly the moved_from and moved_to
files. List the collected test ids (for example pytest --collect-only -q)
at the base and after the move. The ids compare without their file path, so
the part after the first :: must form equal multisets (order does not
matter, counts do). A dropped, added, or renamed test is a defect: use a real diagnosis. The names do not show an edited
assertion, so declare --effect fixture-redefines-case (or the matching
effect) to the trigger when an assertion changed. A record that pairs
this diagnosis with any trigger effect is a defect.
Every moved_from and moved_to path follows the same rule as the target:
under tests/, no .., a test module name. The CLI also runs a fixed
pytest --collect-only -q over the moved_to files and compares the real
ids with tests_after. It never runs the record's own collect_command.
A collection that fails or finds no tests is a defect.
Trust model: collection imports each moved_to module, every
conftest.py from the pytest rootdir down to it (the repository root one
included), and every installed pytest plugin. The path checks bound which
test modules load, not what code runs. Run the CLI only on a checkout you
trust, as you would run its test suite.
Anti-Patterns
| Avoid |
Why |
Instead |
| Forking upstream tools to change behavior |
Creates maintenance burden, diverges from community |
Configure locally, file upstream issues |
| Patching tool source to suppress warnings |
Hides real issues, breaks on updates |
Use tool's suppression mechanism |
| Disabling entire rule categories |
Loses protection the rules provide |
Suppress specific rules with rationale |
| Suppressing without rationale |
Future maintainers cannot evaluate the override |
Always document why in config comments |
| Copying tool defaults into local config |
Config drift when upstream updates |
Only override what you need to change |
Verification
After applying this skill:
Examples
Example 1: PSScriptAnalyzer Rule Failure
Problem: PSAvoidUsingWriteHost fails on a CLI script that intentionally uses Write-Host.
Wrong approach: Modify PSScriptAnalyzer source or remove the rule globally.
Correct approach:
# .psscriptanalyzerrc.psd1
# Rationale: CLI scripts use Write-Host for user-facing output
@{
Rules = @{
PSAvoidUsingWriteHost = @{
Enable = $false
}
}
}
Example 2: markdownlint Conflict
Problem: MD013 (line length) fails on auto-generated documentation.
Wrong approach: Fork markdownlint to increase default line length.
Correct approach:
# .markdownlint.yaml
# Rationale: Generated docs have long lines from tool output
MD013:
line_length: 200
tables: false
Example 3: New Tool Integration
Problem: Adding ruff to a Python project. Several existing files fail.
Wrong approach: Disable all failing rules immediately.
Correct approach:
- Run
ruff check with defaults to see all violations.
- Fix violations that align with project standards.
- Suppress remaining rules per-file with
# noqa and rationale.
- Document any project-wide overrides in
pyproject.toml.
Related Skills
Timelessness: 9/10
External tool integration is a universal software engineering concern. The principle of respecting upstream authority applies to any validator, linter, or static analysis tool regardless of language or framework.
1---2name: validation-authority-33description: Treat upstream validators as authoritative. Align local config to them. Use when validation fails unexpectedly, before modifying validator behavior, or when tempted to change upstream tool code.4metadata:5 source: https://skillmd.com/skills/rjmurillo/validation-authority-36---78# Validation Authority910When integrating external validators (PSScriptAnalyzer, markdownlint, ESLint, etc.), respect upstream defaults. Modify local configuration to match upstream behavior. Do not modify upstream tool code.1112## Triggers1314Say `validation failed unexpectedly`, `should I change the validator`, or15`is this validator authoritative`. Activate when:1617- Validation fails unexpectedly18- Before modifying validator behavior or configuration19- When tempted to change upstream tool source code20- When adding a new external validator to the project21- When suppressing validator warnings without rationale2223## Decision Tree2425```26Validation failure occurred27 |28 v29Is the validator upstream (external tool)?30 | |31 YES NO (local/custom)32 | |33 v v34Modify LOCAL config Modify tool as needed35to align with tool (you own the code)36 |37 v38Document override rationale39in config comments40```4142## Process43441. **Classify the validator**: Determine if the tool is upstream (external) or local (project-owned).452. **Identify the conflict**: Find the specific rule or default causing the failure.463. **Check upstream defaults**: Read the tool's documentation for the default behavior.474. **Align local config**: Update project configuration files to match or explicitly override upstream defaults.485. **Document overrides**: Add comments explaining why any override differs from the upstream default.4950## Trigger Table5152| Scenario | Action | Example |53|----------|--------|---------|54| PSScriptAnalyzer rule fails | Update `.psscriptanalyzerrc.psd1` | Suppress `PSAvoidUsingWriteHost` with rationale |55| markdownlint rule fails | Update `.markdownlint.yaml` | Disable `MD013` line-length for generated docs |56| ESLint rule conflicts | Update `.eslintrc` | Override `no-console` for CLI tools |57| Upstream tool has a bug | Flag the bug to the owner with a repro, add workaround in config | Pin tool version, suppress specific rule |58| Tool default changed after upgrade | Review and align local config to new default | Update config after major version bump |5960## Adjunct Mode: Build Phase 2b6162`/build` Phase 2b invokes this skill after `analysis-provenance` classifies63every path `validation_trigger.py` flagged as a validation target. This mode64writes the decision record; it does not replace the general decision-tree65guidance above, which still applies whenever a human asks how to treat an66upstream validator.67681. For each target, combine `analysis-provenance`'s category, owner, and69 evidence with a diagnosis of what actually happened:70 `implementation-defect` (the local check has a real bug), `local-config-defect`71 (only the project's own config needs to change), `stale-generated-output`72 (a generated mirror drifted from its template or source), `baseline-update`73 (a ratchet or baseline needs new entries), `upstream-defect` (the vendored74 or upstream tool itself is wrong), `relocation-no-semantic-change` (a test75 file moved or split under `tests/` with the same collected tests; see76 rule 9 below), or `unknown` (evidence is insufficient).772. Name the one `permitted_change_location` Phase 3 may edit. `GENERATED`78 always permits only the canonical source, never the mirror. `VENDOR` and79 `UPSTREAM` never permit the target itself; point at the local override or80 config instead. `unknown` diagnosis or `UNKNOWN` category permits nothing:81 the record blocks further semantic edits until ownership evidence exists.823. For `baseline-update`, cite the existing policy that authorizes the83 refresh (`baseline_justification.policy_source`, a path that exists on84 disk), state the reason, and justify every added entry individually.854. Write the record to `<record-path>` (see "Validation Change Record" below86 for where that resolves) and run87 `python3 "${COPILOT_PLUGIN_ROOT:-${CLAUDE_PLUGIN_ROOT:-.claude}}/skills/validation-authority/scripts/validation_record.py" --record <record-path> --changed-path <path>`88 for every changed path. Exit `0` clears Phase 3 to edit the permitted89 locations; exit `1` names the blocking targets; exit `2` is a90 configuration error in the record or the invocation.9192## Scripts9394| Script | Purpose | Exit codes |95|--------|---------|------------|96| `validation_record.py` (this skill's script directory) | Checks the Phase 2b decision record: field shape, provenance/authority rules, and changed-path coverage. Emits a JSON summary of target count, counts by category, defects, and `checks` (whether the trigger recheck ran, and how many targets real collection checked). | `0` record passed, `1` the record has defects, `2` config error (unreadable file, invalid JSON, the sibling `build` trigger could not be loaded or could not read its inputs when `--changed-path` needed it, or pytest is not installed) |9798## Validation Change Record99100`<record-path>` above means: resolve a scratch directory the way101`paths.artifact_dir` does, take its `scratch/` subdirectory, then102`validation-authority-record.json` inside it. That directory is agent PR103scratch space in the consumer's own workspace (git-ignored in this repo's own104checkout); the PR body carries the record's summary instead of a commit.105106```json107{108 "trigger": { "...": "the validation_trigger.py output, carried through unchanged" },109 "targets": [110 {111 "target": "<path>",112 "component": "<human label for the tool or config>",113 "provenance": {114 "category": "LOCAL | GENERATED | VENDOR | UPSTREAM | UNKNOWN",115 "owner": "<non-empty>",116 "evidence": "<non-empty>",117 "canonical_source": "<required, and equal to permitted_change_location, when category is GENERATED>"118 },119 "authority": {120 "contract": "<non-empty; the validator's own exit-code or pass/fail contract>",121 "diagnosis": "implementation-defect | local-config-defect | stale-generated-output | baseline-update | upstream-defect | relocation-no-semantic-change | unknown",122 "permitted_change_location": "<non-empty; never the target itself for VENDOR or UPSTREAM>",123 "escalation": "<required, non-empty, when diagnosis is upstream-defect>"124 },125 "baseline_justification": {126 "policy_source": "<repository-relative path to an existing file; required when diagnosis is baseline-update>",127 "reason": "<non-empty>",128 "added_entries": [{ "justification": "<non-empty per entry>" }]129 },130 "relocation_evidence": {131 "collect_command": "<required when diagnosis is relocation-no-semantic-change; the command that listed the tests>",132 "moved_from": ["<test module path the tests left, non-empty list>"],133 "moved_to": ["<test module path the tests landed in, non-empty list; includes this target>"],134 "tests_before": ["<collected test id at the base, from the moved_from files only>"],135 "tests_after": ["<collected test id after the move, from the moved_to files only>"]136 }137 }138 ]139}140```141142Rules `validation_record.py` enforces:1431441. Every target needs `target`, `component`, `provenance`, and `authority`.145 `trigger` holds the trigger's JSON object, and its `decision` is `activate`146 or `skip`. A `skip` that lists effects or targets is a contradiction. When147 the trigger activated, the record needs at least one target, and each path148 the trigger named needs a record target.149 An activated trigger must carry an `effects` list; a record without one is a150 defect. The validator cannot recompute `effects` from paths: they are151 diff-verified declarations, and an empty list is trusted. With152 `--changed-path`, the validator reruns the trigger on those paths and153 refuses a `skip` the trigger would activate.1542. `UNKNOWN` category or `unknown` diagnosis is always a blocking defect: stop155 semantic edits and request ownership evidence.1563. `GENERATED` needs `provenance.canonical_source`, and157 `authority.permitted_change_location` must equal it.1584. `VENDOR` or `UPSTREAM`: `authority.permitted_change_location` must not159 equal `target`. `upstream-defect` needs a non-empty `authority.escalation`.1605. `baseline-update` needs `baseline_justification` with a `policy_source`161 that is a relative path to an existing file inside the repository, a non-empty `reason`, and a non-empty `justification` on162 every entry in `added_entries`.1636. With `--changed-path` supplied: every path the sibling `build` trigger's164 path cues alone would flag needs a record target; a `VENDOR` or `UPSTREAM`165 target must not itself be a changed path; a changed `GENERATED` target166 needs its canonical source changed too, never edited as a standalone167 mirror.1687. Paths compare after normalization: `./a/b.py`, `a\\b.py`, and `a/b.py` are169 the same target.1708. One run reports every defect: field defects and coverage defects together.1719. `relocation-no-semantic-change` needs `LOCAL` provenance, a test module172 target under `tests/` (`test_*.py` or `*_test.py`, so `conftest.py` and173 other support files are refused), and a `relocation_evidence` block with174 `moved_from` and `moved_to` path lists. `moved_to` lists the target, and175 the before and after ids must name exactly the `moved_from` and `moved_to`176 files. List the collected test ids (for example `pytest --collect-only -q`)177 at the base and after the move. The ids compare without their file path, so178 the part after the first `::` must form equal multisets (order does not179 matter, counts do). A dropped, added, or renamed test is a defect: use a real diagnosis. The names do not show an edited180 assertion, so declare `--effect fixture-redefines-case` (or the matching181 effect) to the trigger when an assertion changed. A record that pairs182 this diagnosis with any trigger effect is a defect.183 Every `moved_from` and `moved_to` path follows the same rule as the target:184 under `tests/`, no `..`, a test module name. The CLI also runs a fixed185 `pytest --collect-only -q` over the `moved_to` files and compares the real186 ids with `tests_after`. It never runs the record's own `collect_command`.187 A collection that fails or finds no tests is a defect.188 Trust model: collection imports each `moved_to` module, every189 `conftest.py` from the pytest rootdir down to it (the repository root one190 included), and every installed pytest plugin. The path checks bound which191 test modules load, not what code runs. Run the CLI only on a checkout you192 trust, as you would run its test suite.193194## Anti-Patterns195196| Avoid | Why | Instead |197|-------|-----|---------|198| Forking upstream tools to change behavior | Creates maintenance burden, diverges from community | Configure locally, file upstream issues |199| Patching tool source to suppress warnings | Hides real issues, breaks on updates | Use tool's suppression mechanism |200| Disabling entire rule categories | Loses protection the rules provide | Suppress specific rules with rationale |201| Suppressing without rationale | Future maintainers cannot evaluate the override | Always document why in config comments |202| Copying tool defaults into local config | Config drift when upstream updates | Only override what you need to change |203204## Verification205206After applying this skill:207208- [ ] Validator is classified as upstream or local209- [ ] Local config aligns with upstream defaults210- [ ] Any overrides include documented rationale211- [ ] No upstream tool source code was modified212- [ ] Reconciliation: paste the validator's exit status line (e.g., `echo "exit=$?"` output after running the validator command) to confirm the updated config passes; a claim of "validation passes" without pasted output does not count213214## Examples215216### Example 1: PSScriptAnalyzer Rule Failure217218**Problem**: `PSAvoidUsingWriteHost` fails on a CLI script that intentionally uses `Write-Host`.219220**Wrong approach**: Modify PSScriptAnalyzer source or remove the rule globally.221222**Correct approach**:223224```powershell225# .psscriptanalyzerrc.psd1226# Rationale: CLI scripts use Write-Host for user-facing output227@{228 Rules = @{229 PSAvoidUsingWriteHost = @{230 Enable = $false231 }232 }233}234```235236### Example 2: markdownlint Conflict237238**Problem**: `MD013` (line length) fails on auto-generated documentation.239240**Wrong approach**: Fork markdownlint to increase default line length.241242**Correct approach**:243244```yaml245# .markdownlint.yaml246# Rationale: Generated docs have long lines from tool output247MD013:248 line_length: 200249 tables: false250```251252### Example 3: New Tool Integration253254**Problem**: Adding `ruff` to a Python project. Several existing files fail.255256**Wrong approach**: Disable all failing rules immediately.257258**Correct approach**:2592601. Run `ruff check` with defaults to see all violations.2612. Fix violations that align with project standards.2623. Suppress remaining rules per-file with `# noqa` and rationale.2634. Document any project-wide overrides in `pyproject.toml`.264265## Related Skills266267| Skill | Relationship |268|-------|--------------|269| [style-enforcement](../style-enforcement/SKILL.md) | Enforces style rules that this skill governs |270| [code-qualities-assessment](../code-qualities-assessment/SKILL.md) | Quality assessment, not validator config |271| [doc-accuracy](../doc-accuracy/SKILL.md) | Detects contradictions between docs/config and behavior |272273## Timelessness: 9/10274275External tool integration is a universal software engineering concern. The principle of respecting upstream authority applies to any validator, linter, or static analysis tool regardless of language or framework.