bilgecheck: agent supply chain check
job
check the files and configs that tell an agent what to do and which tools it has, because those are now part of the attack surface.
gap it targets
- evidence (public record): few open tools check the agent layer at all, and lookalike package names on npm have already caused confusion in this space. bilgecheck is meant to ship under one name on PyPI and npm and to take contributions.
- hypothesis: a hash baseline of tool descriptions catches silent changes to a trusted MCP server that a one-time scan misses.
inputs
- paths to check. defaults:
.mcp.json,.cursor/mcp.json,~/.cursor/mcp.json,~/.claude.jsonmcp section,.claude/settings*.jsonhooks,.claude/skills/,~/.claude/skills/,.codex/,AGENTS.md,CLAUDE.md,.cursorrules,.cursor/rules/. - only check the user's home folder paths if the user asked for their machine, not only the repo.
method
- inventory: list every MCP server (command or URL), every skill, every hook, every rules file.
- servers: flag
npxoruvxwith no version pin, remote URLs over plainhttp, servers that get broad env vars (cloud keys, tokens), and commands that run shell scripts from the internet. - text files (skills, rules, AGENTS.md): flag
- lines addressed to the model that change its priorities ("ignore previous", "do not tell the user", "always run", "send", "upload")
- requests to read
~/.ssh,~/.aws,.env, browser profiles, or to post data to a URL - invisible or direction-changing unicode (
U+200BtoU+200F,U+202AtoU+202E,U+2066toU+2069, tag charactersU+E0000toU+E007F) - base64 or hex blobs over 200 chars inside prose
- hooks: flag hooks that run network commands or write outside the repo.
- tool descriptions: if a server is running and the user agrees, list its tools and store a sha256 of each description in
.keelsec/bilge-baseline.json. on later runs, report any change. - self-check: also scan the keelsec skills folder itself.
what counts as a finding
- any of the flags above. severity high for exfiltration asks and secret-file reads, medium for hidden unicode and unpinned servers, low for style issues.
output
{"command":"bilgecheck","rule_id":"ks.agent.hidden-unicode","kind":"agent-supply","severity":"medium","confidence":"high","location":{"path":".claude/skills/pdf-tools/SKILL.md","start_line":12},"evidence":"U+202E inside instruction text","sources":["bilgecheck"],"status":"open"}
today vs later
- today:
rg -Pfor the unicode ranges and phrases,jqover config JSON, and reading each file. - once the keelsec engine exists:
keelsec bilgecheck --format json, published on PyPI and npm under our own name.
stop and scope
- offline by default. never start an MCP server just to inspect it without the user's yes.
- report, do not edit or delete anyone's config.