Aegis

Aegis — the self-harnessing methodology you apply yourself

THIS IS NOT A DELEGATION SKILL. There is no subprocess, no second LLM, no Python runtime to call. YOU — the assistant reading this — apply the 5 stages yourself, in this conversation, using your existing tools. Everything you need to apply the methodology is in this single markdown file: the procedure, the risk catalog, the harness template, and the verification checklist.

When to invoke this skill

Apply Aegis when the user's request has any of these properties:

• Hallucination cost is high — citations, URLs, named entities that must exist, quotes attributed to people, numeric facts pulled from memory.
• The output flows into an action — code that will be committed, recommendations a user will act on, JSON consumed by downstream code, trades, refunds, deletions.
• Arithmetic / data integrity matters — computations from CSVs, financial calculations, counts of items that must match a requested N.
• The user explicitly says "verify", "be careful", "double-check", "show your work", "make sure", "before I commit", "is it safe to".

Do NOT apply Aegis for plain chat, casual formatting, single-word answers, or anywhere the user accepts a rough estimate. The 5-stage discipline is overhead with no benefit if the answer is going to be a paragraph of explanation.

The procedure (do all five, in order, in this same conversation)

When you decide to apply Aegis, write the user a single response structured like this:

## Stage 1 — Analyze
[your decomposition of the goal]

## Stage 2 — Assess (risks)
[the 3-7 risks from the catalog below that apply to THIS goal]

## Stage 3 — Synthesize (the harness)
[a single Python code block defining Output schema, ALLOWED_TOOLS, and verify()]

## Stage 4 — Execute
[the actual work, using your own tools, conforming to the schema above]

## Stage 5 — Verify
[run the verifier from stage 3 against your stage-4 output; report pass/fail]

## Result
[the verified answer, or an honest refusal if verify failed]

The user sees every stage. They can stop you between stages. The result they trust is what survives stage 5.

Stage 1 — Analyze

Write 2-4 lines decomposing the goal:

• Deliverable: what shape must the final answer take? (a single number, a list of N items, JSON with these keys, a code patch, …)
• Tools needed: which of your existing tools (Bash, Read, Edit, Write, WebFetch, WebSearch, …) will you actually need?
• Open questions: what's ambiguous about the goal that you're going to commit to?

Be terse. This is a header, not an essay.

Stage 2 — Assess (apply the risk catalog)

Read through the catalog below. Pick the 3-7 entries that genuinely apply to THIS goal. For each, write one line:

[HIGH] citation-hallucination — research goal with cited sources, model may invent URLs
[MED]  ranking-ambiguity      — "top 5" without a stated metric

If a risk is borderline, include it. False positives are cheap; false negatives ship bugs.

Risk catalog (consult during stage 2)

Hallucination / fabrication

Arithmetic / data integrity

Time / freshness

Tool / sandbox safety

Output shape

Code-task specific

Multi-step / planning

Ambiguity / framing

PII / safety

Format / encoding

Determinism

Stage 3 — Synthesize (write the harness as a Python code block)

Write a SINGLE Python code block that defines:

• A Pydantic Output schema capturing the answer shape
• An ALLOWED_TOOLS list (yours, the assistant's, e.g. ["Read", "Bash", "WebFetch"])
• A verify(output) function returning failure strings (empty = pass)
• Optionally: a SYSTEM_PROMPT string you're going to internally adhere to
• Optionally: a MAX_STEPS, MAX_REPAIRS

The user reads this code. They see exactly what defenses you committed to. This is the audit artifact for the task.

Template — fill this in per task

# AUTO-WRITTEN HARNESS for: <copy the user's goal verbatim>
# Defenses (from stage 2):
#   <risk-id> -> <one-line description of how the verifier catches it>
#   ...

from typing import Literal
from pydantic import BaseModel, Field

SYSTEM_PROMPT = (
    "You are a <domain> agent. <2-3 sentences of task-specific rules: what to
    verify before answering, what counts as 'done', what evidence the answer
    must rest on, when to refuse>."
)
MAX_STEPS = <3-15 based on task complexity>
MAX_REPAIRS = <0-3 based on cost of being wrong>

class Output(BaseModel):
    <field1>: <type> = Field(<constraints>)
    <field2>: <type> = Field(<constraints>)
    # always add when uncertain:
    confidence: Literal["low", "medium", "high"]
    caveats: list[str] = Field(default_factory=list)

ALLOWED_TOOLS = [<smallest subset of your tools that COULD do the task>]

def verify(output) -> list[str]:
    """Returns list of failure strings. Empty list = pass."""
    failures = []
    # Concrete checks: re-fetch URLs, recompute numbers, validate ranges, etc.
    # Be conservative — catch failures, don't generate them.
    return failures

Worked example for a research goal

User: "Find 3 OSS Python web frameworks with their GitHub URLs."

# AUTO-WRITTEN HARNESS for: "Find 3 OSS Python web frameworks with their GitHub URLs"
# Defenses:
#   citation-hallucination -> regex on github URL + verifier re-fetches each URL
#   entity-fabrication     -> regex requires github.com/{user}/{repo} shape
#   truncated-list         -> Field(min_length=3, max_length=3)
#   ranking-ambiguity      -> required `ranking_criterion` field

from pydantic import BaseModel, Field
from typing import Literal

SYSTEM_PROMPT = (
    "Research agent. Every framework MUST come from a tool call to WebSearch "
    "or WebFetch in THIS run. Never cite from prior knowledge. If you cannot "
    "find 3 well-sourced frameworks, say so in caveats rather than padding."
)
MAX_STEPS = 8
MAX_REPAIRS = 2

class Framework(BaseModel):
    name: str = Field(min_length=2)
    github_url: str = Field(pattern=r"^https://github\.com/[^/]+/[^/]+$")
    one_liner: str = Field(max_length=200)

class Output(BaseModel):
    frameworks: list[Framework] = Field(min_length=3, max_length=3)
    ranking_criterion: str = Field(min_length=4)
    confidence: Literal["low", "medium", "high"]
    caveats: list[str] = Field(default_factory=list)

ALLOWED_TOOLS = ["WebSearch", "WebFetch"]

def verify(output) -> list[str]:
    failures = []
    for fw in output.frameworks:
        # Run a HEAD or GET against each cited URL; in your stage-5 run,
        # you'll actually invoke the WebFetch tool to do this.
        # For now, just declare the check.
        pass  # → in stage 5 you call WebFetch(fw.github_url) and check status
    return failures

Stage 4 — Execute (use your own tools, conform to the schema)

Now do the actual task. Use your real tools (Bash / Read / Edit / Write / WebFetch / WebSearch / etc.). Stay within ALLOWED_TOOLS from stage 3 — if a stage-4 step needs a tool you didn't list, STOP and go back to stage 3 to update.

When you have an answer, write it as a JSON object matching Output. If your output doesn't validate against the schema, repair it (don't loosen the schema).

Stage 5 — Verify (run the verifier from stage 3)

Now execute verify(output) against your stage-4 answer. Concretely:

1. For URL/citation checks → actually call WebFetch on each URL and check status.
2. For arithmetic checks → actually run python -c "..." via Bash and compare.
3. For test-running checks → actually run pytest / npm test / whatever applies.
4. For schema checks → mentally walk the Pydantic constraints over the output.

Then report:

## Stage 5 — Verify
Ran the verifier from stage 3. Results:
  ✓ all 3 github URLs resolved (HTTP 200)
  ✓ output validates against Output schema
  ✓ ranking_criterion is non-empty
  ✓ confidence = "high" matches evidence

Verifier PASSED. Returning the result.

If anything fails:

## Stage 5 — Verify
Ran the verifier. Results:
  ✗ github.com/some-org/fake-repo returned 404
  ✓ output validates against schema

Verifier FAILED. Repairing: dropping fake-repo, searching for a replacement.
[then do another stage-4 cycle, with one fewer attempt left]

If MAX_REPAIRS exhausted and still failing:
  ✗ Could not produce 3 well-sourced frameworks. Refusing.

A refusal is a valid answer. Tell the user what you tried and what failed.

Stage 5b — The Result block

## Result
<the verified JSON OR an honest refusal with reasoning>

The user trusts what's in this block because they saw every stage above it.

Doing this efficiently

You don't have to write the headers literally for trivial tasks. For a task where two stages are obviously low-content ("analyze: user wants X. assess: risk=arithmetic-drift only"), collapse them into one paragraph. Don't skip stages, but do scale the depth to the task.

For high-stakes tasks (financial, medical, code shipping to prod), be verbose — the audit trail is the value.

What this skill does NOT do

• It does not call out to another LLM or subprocess.
• It does not need an API key.
• It does not need the self-harness Python package installed.
• It does not magically prevent hallucination — you still have to actually run the verifier in stage 5 with real tool calls, not mentally rubber-stamp.

If you find yourself thinking "the verifier passes" without having run an actual tool call to check it, you skipped stage 5. Go back.

When you want the Python runtime instead

If you're not Claude — or you want to expose this methodology as a tool other LLMs can call programmatically — the self-harness Python package implements all 5 stages as code. See https://github.com/jcaiagent7143-ui/aegis for the runtime, MCP server, and OpenAI-compatible HTTP proxy. But for direct use inside Claude (Chat / Cowork / Code), you don't need any of that — this markdown is the whole skill.