# Interface Facts > Establish a fact about an external interface - a CLI flag, an API field, a model id, a price, a limit, a version - by fetching the vendor's current documentation instead of recalling it. Use before writing code, a design note, or any claim that depends on how a third-party tool behaves, and whenever a repo document already asserts such a behaviour. - Skill: `niksavis/interface-facts` (Agent Skill) - Install (CLI): `npx skillmds@latest add niksavis/interface-facts` - Raw SKILL.md: https://api.skillmd.com/api/skills/niksavis/interface-facts/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - Author: niksavis (https://skillmd.com/u/niksavis) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/niksavis/interface-facts --- # Interface Facts A model's knowledge of an external interface is a snapshot of whenever it was trained, and vendors ship. So **an external interface fact is unverified until it is fetched**. The cost is asymmetric: a wrong fact about our own code fails a gate, a wrong fact about someone else's interface gets committed as prose and is believed for months. Covers CLI flags and output formats, API fields and endpoints, model ids, context windows, token accounting, pricing, rate limits, config schemas, and version-gated behaviour. ## The order of evidence 1. **Our own code that already touches the interface.** If an adapter, parser or hook reads the thing, that code is the evidence — it was written against a real probe and it is under test. 2. **The installed binary, interrogated.** `--help`, `--version`, a `--dry-run`, an invalid flag whose error names the valid set. This outranks documentation when the question is what *this* version accepts: the docs describe a release, the binary on the path is the one the command will run. 3. **The vendor's current documentation**, fetched now. 4. **The vendor's changelog**, which dates the change. 5. **The vendor's source or issue tracker**, where a documented feature that does not work gets contradicted. 6. **Third-party writing** — weakest, never sufficient alone. A landing page or feature grid is marketing. A README is a claim. ## Prefer our own code over our own prose This is the rule that gets skipped and the one that has cost the most. A design document's *summary* of an interface is a claim about a claim. Read the implementation before quoting the document, and read the section a summary cites before trusting the summary. Measured instance: a design note carried "copilot has no counts" in a components table while section 7.5 immediately below documented the extraction mechanism and `runner.py` implemented it — mapping copilot's `session.shutdown` `modelMetrics` onto our usage split, with summation semantics verified against 15 local stores. The stale summary was read, it matched a prior assumption, and it was repeated to the owner as fact. ## Pin the version and the date Record what you verified against; the answer has a shelf life. This repo already does it where it does it at all — `copilot 1.0.77`, `codex 0.146.0`, "re-probed 1.0.75". An unpinned capability claim cannot be re-checked, only re-argued. ## How to fetch Scope the search to the vendor so third-party summaries cannot outrank the source, then fetch the specific page and ask a **narrow** question — exact flag names, exact field names, whether a machine-readable output mode exists — rather than "tell me about X". Use browser tools only when a page needs JavaScript to render. ## Machine-readable routes, reached only after the binary **Run the installed binary first.** This table makes rung 3 cheap, and cheap is how it goes wrong. Recorded incident, 2026-08-19 on this repo's own toolchain before it was updated: `uv` was installed at 0.11.28 while 0.12.5 was released, and uv's changelog gives 0.12.0 "Define build systems by default with `uv init`" — so the current documentation described a `uv init` the installed binary did not have. `uv init --help` on 0.11.28 already listed both `--package` and `--no-package`, so even a flag probe missed it and only running the command found the default. A route below answers "what does this release document", never "what will this command do". Every row was fetched, not recalled, on 2026-08-19. Follow redirects, and read an unexpected hop or a 404 as route rot rather than as absence of the fact: `docs.claude.com/llms.txt` 301s to `platform.claude.com/llms.txt`, and `developers.openai.com/codex/llms.txt` 308s to `learn.chatgpt.com/docs/llms.txt` while the OpenAI index still advertises the old path. Probe the row before you quote it. | dependency | route, fetched 2026-08-19 | | --- | --- | | uv | `docs.astral.sh/uv/llms.txt`; per page `docs.astral.sh/uv//index.md` | | ruff | `docs.astral.sh/ruff/llms.txt`; per page `docs.astral.sh/ruff//index.md` | | Claude Code | `code.claude.com/docs/llms.txt`; per page `code.claude.com/docs/en/.md` | | Anthropic API | `platform.claude.com/llms.txt`; per page `platform.claude.com/docs/en/.md`. `llms-full.txt` exists and is 32 MB — never fetch it whole | | Codex | `developers.openai.com/codex/llms.txt` → `learn.chatgpt.com/docs/llms.txt`; per page `learn.chatgpt.com/docs/.md` | | GitHub Docs, and so Copilot CLI and Actions | `docs.github.com/llms.txt`; any page as markdown by appending `.md` (`text/markdown`); or the APIs below | | Python | no `llms.txt` (404); Sphinx `docs.python.org/3/_sources/.rst.txt` and `docs.python.org/3/objects.inv` | | pytest | no `llms.txt` (404); `docs.pytest.org/en/stable/_sources/.rst.txt` and `docs.pytest.org/en/stable/objects.inv` | | git | none served (404) — rung 2, `git --help` on the installed binary | | pre-commit | none served (404) — rung 2, `pre-commit --help` on the installed binary | | any other dependency the repo declares (in basicly: jinja2, jsonschema, pyyaml, rich, ruamel.yaml, pyright, bandit, vulture, import-linter, pip-audit, djlint) | probe `/llms.txt`, then the Sphinx pair `objects.inv` + `_sources/.rst.txt` (both answered 200 for jinja2 and rich), else rung 2 | GitHub Docs names its Article, Article Body, Page List and Search APIs the preferred route for automated tools, and **the Search API requires an undocumented `client_name` parameter**: the example GitHub prints in its own `llms.txt`, `/api/search/v1?query=actions&language=en&version=free-pro-team@latest`, answers 400 `Missing required parameter 'client_name' for external requests`, and the same URL with `&client_name=` answers 200. `/api/article` and `/api/article/body` answered 200 without it — send it anyway rather than depending on that asymmetry holding. Three refusals go with the table. Never cache or commit fetched documentation: a cache lands in a consumer tree and the upstream licence is not ours to redistribute. Never cite an aggregator, mirror or docs-summarising site as the source of an interface fact — only the vendor's own host counts. Never record a fact a route gave you without the version it describes and the date you fetched it. ## Distinguish "the capability exists" from "we can reach it" Separate questions; conflating them produces a confident wrong answer in either direction. Copilot reports per-model token usage interactively via `/usage`, and every plan moved to token-based billing on 2026-06-01 — yet the docs show no flag or JSON mode that emits it, and the mechanism our adapter uses is out of band: the dispatch supplies the session UUID with `--session-id`, which makes the store path knowable before the store exists. "It is documented" does not mean "our dispatch can read it", and "not on stdout" does not mean unmeterable. ## State the claim at the precision the evidence supports A correct-but-vague note invites a future reader to "fix" it. Copilot genuinely has no config-file *deny* — but a config file does exist, `~/.copilot/permissions-config.json`, which stores approvals. Written as "has no config file", the next reader finds that file, reads the note as stale, and removes the invocation-time injection it justifies. ## Record it where the next reader will hit it A fact established and not written down gets re-derived or re-guessed. Put it on the bead (`basicly tracker write -- comments add "verified against : ..."`), and **correct the document that carried the stale version** rather than adding the new claim beside it — two claims that disagree cost more than one that is wrong. ## When a repo document and the vendor disagree The vendor wins on what the interface *does*; the repo wins on what we *decided*. Correct the document in place and name what refuted it — a fresher number pasted over an older one with no provenance is the same unchecked declaration one generation on. Where a value we depend on can be checked against data we already record, wire the comparison as a gate: `runner.window_violations` exists because a declared context window of 200000 sat against a recorded occupancy of 223221 for months, mechanically detectable the whole time.