# Refresh Provider Docs

> Retrieve, normalize, compare, and cite authoritative AI provider documentation to verify what agentconfig.org publishes. Use when checking whether provider config locations, hook events, skills, MCP, custom agents, or precedence claims are still accurate, when adding a provider to the compatibility index, or when running the scheduled documentation refresh.

- Skill: `jonmagic/refresh-provider-docs` (Agent Skill, multi-file: 37 files)
- Install (CLI): `npx skillmds@latest add jonmagic/refresh-provider-docs`
- Raw SKILL.md: https://api.skillmd.com/api/skills/jonmagic/refresh-provider-docs/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: jonmagic (https://skillmd.com/u/jonmagic)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/jonmagic/refresh-provider-docs

---


# Refresh Provider Documentation

Verify what agentconfig.org publishes against what providers actually document, and cite a primary source for every claim.

The site's credibility rests on being right about file paths, lifecycle events, and precedence rules that change without notice. This skill exists so that no provider panel, comparison row, or tutorial is expanded on memory.

## The split this skill enforces

Reading documentation is a judgment task. Deciding whether the site is wrong is not.

| Step | Owner | Why |
| --- | --- | --- |
| Retrieve registered sources | Script | Reproducible, cited, and offline afterwards |
| Read the retrieved documents | Agent | Prose is not parseable by regex |
| Write normalized claims | Agent | Extraction is judgment, so it must be explicit and citable |
| Validate, compare, and report | Script | A verdict must be deterministic and reviewable |

Never skip the middle two steps by guessing, and never hand-write a finding. If a claim cannot be traced to a retrieved snapshot, it does not get recorded.

## When to Use

Use this skill when:

- A provider's configuration location, hook events, permissions, or precedence rules need checking against current documentation.
- A new provider is being considered for the compatibility index.
- The scheduled refresh runs and needs an evidence report.
- Someone asks which primary source backs a published claim.

Do not use this skill to edit content. It produces evidence and findings; the edits belong to `add-primitive`, `add-provider`, and `generate-llms`.

## Workflow

### 1. Retrieve the sources

```bash
bun .github/skills/refresh-provider-docs/scripts/provider-docs.ts fetch --allow-network
bun .github/skills/refresh-provider-docs/scripts/provider-docs.ts fetch --allow-network --provider claude
```

Retrieval requires `--allow-network` every time. Nothing in this skill runs during an ordinary site build, and the site never depends on network access to render.

Snapshots and a `manifest.json` are written to a fresh run-specific directory outside the repository, printed at the end of the run. Each run gets its own directory so a partial or provider-scoped retrieval can never leave older snapshots beside the new manifest, where they would be read back as current evidence. Each snapshot carries its source id, canonical URL, requested URL, final URL, content type, authority, and retrieval timestamp in a header comment. Responses are limited to 512 KB, non-document content types fail closed, and unexpected redirect targets are rejected. An unknown `--provider` value fails closed rather than reporting an empty run as a success. Scheduled runs add `--tracked-only` so candidate providers cannot silently enter the compatibility refresh. If any source fails, the command exits non-zero: fix or retire the registry entry rather than publishing a claim you could not retrieve.

### 2. Read the snapshots and write claims

Read the retrieved files. For each thing the site asserts, or should assert, record a claim in a JSON array:

```json
{
  "id": "copilot.hooks.location",
  "provider": "copilot",
  "primitive": "hooks",
  "aspect": "location",
  "value": [".github/hooks/*.json", "~/.copilot/hooks/*.json"],
  "sourceId": "copilot.about-hooks",
  "sourceUrl": "https://docs.github.com/en/copilot/concepts/agents/hooks",
  "sourceAuthority": "primary",
  "retrievedAt": "2026-08-25",
  "notes": "Repository hooks apply to any Copilot agent; personal hooks apply to Copilot CLI."
}
```

Rules that keep claims honest:

- Record what the document says, not what the site already says. Reading the site first biases extraction.
- One claim per provider, primitive, and aspect. If two official pages disagree, record both and let the comparison mark it ambiguous.
- Quote paths exactly, including globs and extensions. `.mdc` and `.md` are different answers.
- Use `support: "none"` only when documentation states the capability is unavailable, never when documentation is merely silent.
- Leave the claim out when the documentation is unclear. Silence is a finding; a guess is a defect.

See `references/NORMALIZATION.md` for the aspect vocabulary and worked examples.

### 3. Validate, compare, and report

```bash
S=.github/skills/refresh-provider-docs
bun $S/scripts/provider-docs.ts validate claims.json --manifest manifest.json --baseline baseline-claims.json
bun $S/scripts/provider-docs.ts compare claims.json --manifest manifest.json --baseline baseline-claims.json --json findings.json
bun $S/scripts/provider-docs.ts report claims.json --manifest manifest.json --baseline baseline-claims.json --out report.md
```

The manifest and baseline flags are blocking evidence gates. Every claim source must be present and successful in the exact fetch manifest, its retrieval date must match that manifest entry, and every committed baseline claim ID must remain in the new claim set. Missing or truncated evidence exits `1`; it can never look like a clean refresh.

Each finding carries one of four statuses:

| Status | Meaning | What happens next |
| --- | --- | --- |
| `confirmed` | The published value matches its primary source | Nothing, and that is a result worth keeping |
| `changed` | Documentation and the site disagree, or the site cannot express the fact | Edit the data, or design the missing concept |
| `ambiguous` | The comparison cannot be settled deterministically | A person reads the cited sources and decides |
| `unsupported` | Documentation states the capability is unavailable | Do not publish a capability claim |

Exit codes: `0` clean, `1` fail-closed error, `2` findings require action. Exit `2` means any finding carries an action other than `none`, including an `unsupported` finding that still requires a site edit, so a scheduled refresh cannot miss one. The scheduled refresh keys off exit code `2`.

### 4. Act on the findings

Apply `update-site-data` findings to `site/src/data/primitives.ts` only, then regenerate published files with the `generate-llms` skill. `site/src/data/comparison.ts` and `site/src/data/providerProfiles.ts` compute their values from `primitives.ts` at import time, so hand-editing them there would duplicate the same fact in two places and reintroduce the drift this derivation was built to eliminate — the edit belongs in `primitives.ts` alone.

Treat `extend-site-model` findings as content design rather than a data edit. They mean the documented behavior has no home in the current taxonomy.

Re-run the comparison after editing so the report reflects the change.

The scheduled workflow is evidence-only. It may update only the weekly claims, findings, and report files and may open only a draft pull request. It cannot edit `primitives.ts`, generated content, the registry, skills, or workflows. A person reviews the evidence and applies factual site changes in a separate change.

## Fail-Closed Rules

The comparison refuses to guess. It returns `ambiguous` and asks for a person when:

- Two official sources disagree about the same fact.
- The only support for a claim is a secondary source.
- The cited source is not in the registry, belongs to another provider, or does not match the registered URL.
- The evidence is older than the registry's freshness limit.
- The site contradicts itself between what it publishes and any other indexed site data source (a defense against drift, even though `comparison.ts` is now derived from `primitives.ts` and should not diverge in normal operation).

An empty claim set, a missing baseline claim ID, a claim not bound to the exact successful manifest entry, or a mismatched retrieval date is treated as a failed retrieval, not a clean run.

## The Source Registry

`data/sources.json` holds every documentation entry point, its topic, and its authority. Primary means the provider's own documentation or source repository. Secondary means anything else, must carry a `note` justifying its use, and can never confirm a claim on its own.

```bash
bun .github/skills/refresh-provider-docs/scripts/provider-docs.ts sources
bun .github/skills/refresh-provider-docs/scripts/provider-docs.ts sources --check-urls --allow-network
```

Run the URL check before trusting a refresh; a silently moved documentation page is the most common way published claims go stale. See `references/SOURCE-REGISTRY.md` for the authority policy and how to add a provider.

## Validation

```bash
cd .github/skills/refresh-provider-docs && bun test
```

`bun test` skips hidden directories when invoked from the repository root, so run it from the skill directory.

The suite covers schema validation, every fail-closed path, report integrity, registry integrity, the network opt-in, and activation cases. `evals/trigger-queries.json` records when this skill should and should not activate.

## Worked Example

`reports/2026-08-25-claims.json` and `reports/2026-08-25-report.md` are a real run against the live site. Read them before your first refresh: they show the claim format, the four statuses, and the kind of drift this process is meant to catch.

## Related Skills

- `add-provider` adds a verified provider to the comparison system.
- `add-primitive` edits primitive data once a finding says what to change.
- `generate-llms` regenerates published machine-readable files after data edits.