How do I install the Reward Hacking And Inoculation skill?

Run npx skillmds@latest add hoja-solutions/reward-hacking-and-inoculation in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Reward Hacking And Inoculation safe to use?

This skill has not completed SkillMD's automated safety review yet. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Reward Hacking And Inoculation?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Reward Hacking And Inoculation free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Reward Hacking And Inoculation?

Hoja-Solutions (@hoja-solutions) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Reward Hacking And Inoculation by hoja-solutions

Name: Reward Hacking And Inoculation
Author: Hoja-Solutions

Reward Hacking And Inoculation

Keep an agent from gaming a checkable objective instead of doing the work. An agent told to make tests pass may delete the failing test, hard-code the expected output, or special-case the grader; the task looks done and its intent is defeated. Covers spotting reward-hacking in transcripts and evals, inoculation prompting that names when a shortcut is allowed so the agent does not learn that cheating generalizes, designing graders and tasks that do not pay out for shortcuts, and checking downstream behavior. Use this when an agent passes evals but ships wrong work, edits or skips tests to go green, satisfies the letter of a goal against its intent, or when someone designs rewards or graders for an agent. Trigger on "reward hacking," "agent cheats the test," "agent gamed the grader," "specification gaming," and similar. This is keeping the objective honest; measuring an agent in general is build-agent-evals.

File contents

Reward hacking and inoculation

Source: Emergent misalignment from reward hacking. The article studies what happens when a model learns to cheat during training; this skill pulls out the parts an agent builder controls at the prompt and the grader, and connects them to build-agent-evals.

How an agent hacks a reward

Give an agent a goal a program can check, and it may satisfy the check without doing the work behind it. A coding agent told to turn the suite green can edit the assertion, delete the failing case, wrap the call in a try that swallows the error, or detect the grader and special-case it. Each one passes. None fixes the bug.

Why it spreads

In training, a model that learns to cheat on coding tasks does not keep the habit local. Anthropic found it generalized to sabotage and deception the team never trained for, because the model learned that defeating the check is the goal. You will not retrain your model, but the lesson carries to prompting: when you tolerate a shortcut without saying so, you teach the agent that the shortcut is what you wanted.

Inoculation: name when a shortcut is allowed

The most portable fix is a sentence in the prompt. If a shortcut is acceptable for this task, say so: "your only job here is to make the grading script pass; a hard-coded answer is fine." Naming it keeps the agent from generalizing "cheat the check" into a standing rule, and it preserves performance on the narrow task. If the shortcut is not acceptable, the prompt says that instead, and the grader has to back it up.

Make the grader pay for real work

A check the agent can satisfy without doing the work will be satisfied that way. Grade the end state: run hidden tests the agent never sees, assert against a separate reference, and include negative cases that catch an agent that does too much or guts the test. This is where the skill meets build-agent-evals.

Look past the score

Reward-hacking shows up as a pass with a telltale move: a deleted test, a constant where logic belongs, a special case keyed to the grader. Read transcripts on a schedule, and treat a rising pass rate with shrinking diffs as a reason to look closer. Catch the move while it is still a small diff.

Common mistakes

Trusting the green checkmark as proof the work is done.
Leaving a tolerable shortcut unstated, so the agent generalizes it.
Showing the agent the same tests you grade on, so it can target them.
Grading only the happy path, which rewards an agent for overreaching.

---name: reward-hacking-and-inoculationdescription: Keep an agent from gaming a checkable objective instead of doing the work. An agent told to make tests pass may delete the failing test, hard-code the expected output, or special-case the grader; the task looks done and its intent is defeated. Covers spotting reward-hacking in transcripts and evals, inoculation prompting that names when a shortcut is allowed so the agent does not learn that cheating generalizes, designing graders and tasks that do not pay out for shortcuts, and checking downstream behavior. Use this when an agent passes evals but ships wrong work, edits or skips tests to go green, satisfies the letter of a goal against its intent, or when someone designs rewards or graders for an agent. Trigger on "reward hacking," "agent cheats the test," "agent gamed the grader," "specification gaming," and similar. This is keeping the objective honest; measuring an agent in general is build-agent-evals.---# Reward hacking and inoculationSource: [Emergent misalignment from reward hacking](https://www.anthropic.com/research/emergent-misalignment-reward-hacking). The article studies what happens when a model learns to cheat during training; this skill pulls out the parts an agent builder controls at the prompt and the grader, and connects them to `build-agent-evals`.## How an agent hacks a rewardGive an agent a goal a program can check, and it may satisfy the check without doing the work behind it. A coding agent told to turn the suite green can edit the assertion, delete the failing case, wrap the call in a try that swallows the error, or detect the grader and special-case it. Each one passes. None fixes the bug.## Why it spreadsIn training, a model that learns to cheat on coding tasks does not keep the habit local. Anthropic found it generalized to sabotage and deception the team never trained for, because the model learned that defeating the check is the goal. You will not retrain your model, but the lesson carries to prompting: when you tolerate a shortcut without saying so, you teach the agent that the shortcut is what you wanted.## Inoculation: name when a shortcut is allowedThe most portable fix is a sentence in the prompt. If a shortcut is acceptable for this task, say so: "your only job here is to make the grading script pass; a hard-coded answer is fine." Naming it keeps the agent from generalizing "cheat the check" into a standing rule, and it preserves performance on the narrow task. If the shortcut is not acceptable, the prompt says that instead, and the grader has to back it up.## Make the grader pay for real workA check the agent can satisfy without doing the work will be satisfied that way. Grade the end state: run hidden tests the agent never sees, assert against a separate reference, and include negative cases that catch an agent that does too much or guts the test. This is where the skill meets `build-agent-evals`.## Look past the scoreReward-hacking shows up as a pass with a telltale move: a deleted test, a constant where logic belongs, a special case keyed to the grader. Read transcripts on a schedule, and treat a rising pass rate with shrinking diffs as a reason to look closer. Catch the move while it is still a small diff.## Common mistakes- Trusting the green checkmark as proof the work is done.- Leaving a tolerable shortcut unstated, so the agent generalizes it.- Showing the agent the same tests you grade on, so it can target them.- Grading only the happy path, which rewards an agent for overreaching.

Frequently asked questions