Profile My skills Saved Collections Edit profile Submit a skill

← all publishers

trapstreet

@trapstreet source repo

3 published skills

  1. ▌
    Trapstreet Setup · trapstreet
    Install and authorize the trapstreet CLI (tp) -- install uv, install trap-cli, run tp auth login, verify the pairing. Use when the user says "set up trapstreet", "install tp", "install the CLI", "auth failed", "tp isn't working", or trapstreet-solution-scaffold/trapstreet-task-scaffold discover tp is missing or unauthenticated.
    0
    installs
  2. ▌
    Trapstreet Task Scaffold · trapstreet bundle
    Design and scaffold a new trapstreet.run task to evaluate a given agent/skill/tool -- the reverse of trapstreet-solution-scaffold (a solution for an existing task). Generates the mechanical parts (traptask.yaml, judge.py/grader.py on the TRAPTASK_MANIFEST contract, build_cases.py's validate-then-render pipeline) and guides the judgment-heavy parts through a structured interview (what the tool actually does, what counts as correct, what makes it hard, where ground truth comes from, how scoring resists gaming), plus the calibration protocol that says whether the task discriminates and a checklist of exploits found the hard way. Use whenever the user wants to build a new evaluation task, turn an agent/skill into a benchmark, design test cases for a tool, fix a task that everything passes or fails, or asks things like "can we make a task out of this", "how do I evaluate my agent on trapstreet", "why is my task too easy", "design a benchmark for X" -- even if they don't say "task" or "trapstreet-tasks" by name.
    2
    installs
  3. ▌
    Trapstreet Solution Scaffold · trapstreet bundle
    Scaffold a submission-ready trap-cli solution for a trapstreet.run task -- generates trap.yaml (and, optionally, solution.py) with the correct current schema, sets up credentials, and gets the repo into leaderboard-eligible shape. Covers writing one from scratch, wrapping an existing solution.py, and adapting someone else's existing repo. Use whenever the user wants to build a new solution for a trapstreet task, wrap an existing script so it can be evaluated, port a project they found elsewhere into this task-eval format, add another model/provider to compare against an existing task, or is troubleshooting why a solution won't submit or isn't showing up on the leaderboard -- even if they don't say "trap.yaml" or "trap-cli" by name, e.g. "I want to test claude opus against this task", "make a solution for X", "I found this agent repo, can we test it against our task", "why isn't my run showing up", "add gpt-5 as another comparison point".
    0
    installs