all publishers

RyanAlberts

@ryanalberts source repo

30 published skills

  1. ▌
    Regression Finder · ryanalberts bundle
    Regression check for coding agents: shows how the agent behaved before and after each harness update, model switch, or week in the user's own Claude Code or Codex history, and finds the point where it changed. Use when the user says the agent's quality dropped or it got worse, dumber, lazier, or degraded since an update or since they upgraded; asks whether a new Claude Code or Codex version or model made it worse than the old one; wants to know which release, version, or week it regressed in; or wants numbers to report a regression, such as reads before edits, interruptions, and corrections per version. Runs locally and reads transcripts only; nothing goes over the network.
    9 repo stars
  2. ▌
    Harness Test Drive · ryanalberts bundle
    Test-drives coding agents (Claude Code, Codex, Gemini CLI) on tasks mined from the user's own git history: each agent gets a past commit message in a fresh copy of the repo, and the repo's own tests score how many tasks it completes, dollars per task, minutes, and change size. Use when the user asks which coding agent or harness works best on their codebase; wants to compare or benchmark agents on their own repository instead of trusting leaderboards such as SWE-bench; wants to trial one agent against another before switching; or asks what each agent costs per fixed bug. Spends money: the harness CLIs call their model providers, so a dollar cap is required. The scripts make no network calls.
    9 repo stars
  3. ▌
    Tool Design Checker · ryanalberts bundle
    Grades MCP tool definitions the way a model reads them: a clear purpose, described parameters, safe annotations such as readOnlyHint, and the token size of every tool. Use when an author wants to lint, grade, or review an MCP server's tools, tool descriptions, or input schemas before release; when a user asks which MCP servers are installed, or how many tools and tokens they add to the context window in Claude Code, Codex, Cursor, Gemini CLI, or OpenCode; when tools overlap or collide across servers and the model picks the wrong one; or when checking an OpenAI or Anthropic function list. Runs locally; starts servers only with --launch and contacts remote servers only with --remote.
    9 repo stars
  4. ▌
    Session Waste Report · ryanalberts bundle
    Waste report for coding-agent sessions in Claude Code, Codex, Gemini CLI, and OpenCode: finds where tokens, money, and time were wasted, and the fix for each kind of waste. Use when the user asks where tokens or money went or why the bill is so high; about habits that burn tokens, such as re-reading a file that has not changed, huge tool outputs, polling, or cache misses from pauses; about the subagent share of cost; or for failure patterns across sessions: tool errors, permission denials, interrupts, corrections, and repeated calls. Runs locally and reads transcripts only; nothing goes over the network.
    9 repo stars
  5. ▌
    Claim Check · ryanalberts bundle
    Claim checker that audits a coding agent's statements that tests pass or a build is clean against its own session transcripts: whether a matching run happened before the claim, whether it passed, and whether code changed after it. Use when the user asks whether the agent really ran the tests, how often it said tests passed without proof, or whether "all tests pass" was true; wants to catch false or stale success claims; asks whether the agent deleted, skipped, or xfailed failing tests or loosened assertions in the current diff; or wants a Stop hook that sends the agent back to rerun tests before finishing. Runs locally: reads Claude Code, Codex, Gemini CLI, and OpenCode transcripts and the git diff; sends nothing.
    9 repo stars
  6. ▌
    Runaway Guard · ryanalberts bundle
    Runaway guard: a hook that stops a live Claude Code or Codex session when the agent loops on the same tool call, keeps failing, or exceeds a dollar cap. Use when the user wants a spending cap, budget limit, or cost ceiling for interactive agent sessions (the built-in --max-budget-usd works only in print mode); wants to halt an agent that repeats the same command, is stuck in a loop, or runs up a bill unattended overnight; wants a circuit breaker for several failed tool calls in a row; or asks how much the current session has spent against its cap, why a call was blocked, or how to raise or reset the cap. Runs locally: reads the session transcripts before each tool call and sends nothing.
    9 repo stars
  7. ▌
    Sandbox Check · ryanalberts bundle
    Sandbox audit that probes, from inside a live coding-agent session, what the agent can really reach: secret files it can open, secret-like environment variables, files that git, the shell, the editor, or the harness later run (git hooks, shell startup files, harness settings), writes outside the project, the Docker socket, the SSH agent, and passwordless sudo. Use when the user asks what the agent can access or write on this machine, whether the sandbox actually works, how big the blast radius is if the agent gets prompt-injected, whether SSH keys, cloud credentials, or .env files are exposed, or whether it could escape through Docker or sudo. Runs locally and edits no existing file: it creates and deletes one empty test file per folder it checks, and only the optional --network flag makes DNS lookups and TCP connections.
    9 repo stars
  8. ▌
    Rules To Guards · ryanalberts bundle
    Rule enforcer that finds which written rules in AGENTS.md, CLAUDE.md, and GEMINI.md a coding agent keeps breaking, counts every violation in recent Claude Code, Codex, Gemini CLI, and OpenCode sessions, and turns each broken rule into a hook that blocks it, tested by replaying the past violations. Use when the user says the agent keeps breaking or disobeying a rule (such as "use pnpm, never npm"); asks how often agents violate their rules; wants a rule enforced by a hook instead of repeated in text; or wants to convert rules into hooks or permission rules. Runs locally: reads AGENTS.md-style rules and session transcripts, and changes settings only with --write after the user agrees.
    9 repo stars
  9. ▌
    Guardrail Tester · ryanalberts bundle
    Guardrail tester that checks whether the permission rules and PreToolUse hooks already set up in Claude Code, Codex, Gemini CLI, OpenCode, or Cursor stop a battery of dangerous commands, including wrapped, reordered, and full-path forms that slip past prefix rules, and measures prompt friction on the latest tool calls. Use when the user asks whether deny rules or hooks block force pushes, rm -rf, secret reads, or downloads piped into a shell; wants to test or audit guardrails, or find a bypass in permission settings or exec policy; asks how many prompts or blocks the rules cause; or just installed a guard hook and wants proof it holds. Local only, no network; executes the user's own hook commands only with their yes.
    9 repo stars
  10. ▌
    Agents Md Checker · ryanalberts bundle
    Checks which instruction files (AGENTS.md, CLAUDE.md, GEMINI.md, Cursor rules, Copilot instructions) each coding agent loads from a repo, what gets cut or skipped, and whether the commands those files document still work. Use when the user asks whether Claude Code, Codex, Gemini CLI, Cursor, OpenCode, Copilot, or Aider reads their AGENTS.md or CLAUDE.md; why an agent ignores or truncates part of a context file; whether the build, test, or lint commands and paths in AGENTS.md are stale; or wants to audit context files across agents. Reads local files and makes no network calls; runs documented test, lint, and build commands only with --run after the user agrees.
    9 repo stars
  11. ▌
    Pmstack Prd · ryanalberts
    Turn a raw customer signal (quote, support ticket, feature request, exec ask) into a structured Product Requirements Document. Use when a PM mentions a customer quote and wants a spec, when "let's write a PRD" comes up, when a vague feature request needs scoping, or when the user asks for a problem statement, MoSCoW prioritization, success metrics, or PRD template.
    9 repo stars
  12. ▌
    Pmstack Voc · ryanalberts
    Synthesize many raw customer signals (support tickets, interview snippets, churn reasons, sales notes, NPS verbatims, app reviews) into a small set of ranked, PRD-ready problems. Use when a PM pastes or attaches a pile of feedback and asks "what's the real problem / what should we build first / what are the themes", when there are too many signals to read one by one, or when someone needs to turn voice-of-customer data into prioritized problem statements before writing a PRD. This is the front-of-funnel step that precedes pmstack-prd.
    9 repo stars
  13. ▌
    Pmstack Eval · ryanalberts
    pmstack /eval — Anthropic's eval framework, PM-runnable
    9 repo stars
  14. ▌
    Pmstack Lint · ryanalberts
    Walks the pmstack skill graph against the outputs/ directory to find graph gaps (missing downstream artifacts), cross-artifact drift (PRD-metrics-eval misalignment), and stale candidates (superseded files). Each finding includes a one-line "do this" recommendation. Trigger when the user says "lint my workspace," "graph check," "find drift," "tidy up outputs," "what's stale," or asks for a workspace audit. Run weekly on schedule, or before any review.
    9 repo stars
  15. ▌
    Pmstack Brief · ryanalberts
    Draft a stakeholder brief on a topic, tailored to executive, engineering, customer, or board audience. Use when a PM asks for a "brief", "update", "readout", "exec summary", "eng comms", "customer announcement", or "board update", or when they need to communicate a status / decision / change to a specific audience and need help adapting the framing.
    9 repo stars
  16. ▌
    Pmstack Sprint · ryanalberts
    Full PM sprint orchestrator — chains pmstack-prd, pmstack-metrics, pmstack-eval, and pmstack-brief in sequence with explicit user confirmation between each step. Use when the user has a customer signal or feature ask and wants to go end-to-end (problem → spec → metrics → eval design → stakeholder comms) in one guided pass with checkpoints.
    9 repo stars
  17. ▌
    Pmstack Weekly · ryanalberts
    A Monday-morning self-snapshot for PMs. Three sections only — decisions made, open loops aging out, and one required "thing I changed my mind about." Anti-vanity by design. Trigger when the user says "weekly memo," "weekly recap," "Monday status," "what changed this week," "weekly self-check," or asks to summarize last week's PM activity. Run weekly on schedule, or on demand to catch up.
    9 repo stars
  18. ▌
    Pmstack Compare · ryanalberts
    Feature-by-feature comparison of two or more products with built-in eval design and runnable execution. Use when the PM wants to compare specific features across products (e.g., "compare GitHub Copilot vs Cursor", "how does our pricing stack up against X and Y", "evaluate these two AI coding tools head-to-head"). Use pmstack-competitive instead for market-positioning analysis (audience, value prop, white space).
    9 repo stars
  19. ▌
    Pmstack Metrics · ryanalberts
    Design a measurement framework for an AI product or feature with North Star, supporting metrics, counter-metrics, and AI-specific quality / latency / cost metrics. Use when a PM asks "how do we measure this", "what's the North Star for X", "how would we know X is winning", or mentions instrumentation, OKRs, KPIs, or success criteria for an AI feature.
    9 repo stars
  20. ▌
    Pmstack Run Eval · ryanalberts
    Execute an evaluation YAML produced by pmstack-eval against a real target system, running n_trials per task and reporting both pass@k and pass^k. Implements the running side of Anthropic's eval framework. HARD-STOPS with educative errors if no target is configured, no tasks are present, or required env vars are missing — never simulates results. Use when the user asks to "run", "execute", or "score" an eval, or wants real metrics from an existing eval design. Requires a runtime that can execute Python and the `claude` CLI (Claude Code, sandboxed code-exec on web, or local terminal).
    9 repo stars
  21. ▌
    Pmstack Eval Self · ryanalberts
    Run the pmstack self-eval suite to score every pmstack skill against canonical scenarios, with regression check vs the locked golden baseline. Use when the user wants to test pmstack itself, check skill quality after a model upgrade or prompt change, or verify pmstack still works before recommending it to others.
    9 repo stars
  22. ▌
    Pmstack Premortem · ryanalberts
    Run a Klein-style pre-mortem on a draft PRD before launch — simulates 3 plausible failure stories 6 months in the future, names leading indicators and mitigations, then offers to mutate the PRD's Risks section. Trigger when the user says "pre-mortem," "what could go wrong," "risk pass," "imagine this fails," "stress test the spec," or asks to harden a PRD before review. Use after /prd, before launch.
    9 repo stars
  23. ▌
    Pmstack Vibe Test · ryanalberts
    pmstack /vibe-test — read the transcripts before you write the eval
    9 repo stars
  24. ▌
    Pmstack Eval Drift · ryanalberts
    Re-runs an existing eval suite, diffs against the prior baseline, and writes a release-blocker memo if anything regressed. Use weekly on a schedule, or on demand before a release. Trigger when the user mentions "did my AI feature get worse," "eval drift," "model regression," "eval over time," "weekly eval," "release-blocker check," "is this still working," or asks to re-run /eval-self / /run-eval and compare to last time. Designed for repeated runs — produces durable, dated artifacts.
    9 repo stars
  25. ▌
    Pmstack Eval Grade · ryanalberts
    pmstack /eval-grade — is your eval any good?
    9 repo stars
  26. ▌
    Pmstack Onboarding · ryanalberts
    An interactive 9-step tutorial that walks a new user through every pmstack capability using a realistic AI-code-review scenario. Includes Anthropic's vibe-test (read raw transcripts before formalizing an eval) and transcript-review (diagnose failed trials as model / grader / task errors) rituals. Works in claude.ai web, desktop, and mobile (no terminal needed). Trigger when the user says "I just installed pmstack," "how do I use this," "tutorial," "walk me through pmstack," "getting started," or "show me what pmstack does."
    9 repo stars
  27. ▌
    Pmstack Competitive · ryanalberts
    Generate a structured competitive landscape analysis for a market or product category. Use when a PM mentions "competitive analysis", "market positioning", "white space", "where do we wedge in", asks who else is in a space, or wants to size up rivals before strategy/launch. Focus is on market positioning (audience, value prop, white space) — for feature-by-feature parity, use pmstack-compare instead.
    9 repo stars
  28. ▌
    Pmstack Eval Report · ryanalberts
    pmstack /eval-report — make your eval run shareable
    9 repo stars
  29. ▌
    Pmstack Launch Readiness · ryanalberts
    Pre-launch verifier. Aggregates PRD, metrics, eval design, eval run, premortem, eval-drift, and brief evidence into a single GO/NO-GO/CONDITIONAL verdict with each item showing pass/fail/missing plus the file that proves it. Trigger when the user says "are we ready to ship," "launch readiness," "release check," "pre-launch review," "ship it?" or asks to gate a release. Run before any launch.
    9 repo stars
  30. ▌
    Pmstack Transcript Review · ryanalberts
    pmstack /transcript-review — model mistake, grader mistake, or task error?
    9 repo stars