Deterministic Core, LLM Judgment
An agent where the LLM is the backbone fails when the LLM hallucinates, drifts, is unavailable, or costs too much. An agent where the deterministic duties are the backbone always runs and the LLM only adds judgment on top. If the LLM is off, the work still happens; if it's on, the work is better. This is the robust agent architecture.
The split
- Deterministic core (load-bearing, always executes): the duties that must happen every run regardless of LLM availability — fetching the queue, snapshotting state, computing metrics, writing the run log, surfacing blockers. Pure code, no LLM calls. Testable, auditable, no hallucination.
- LLM judgment layer (additive, optional): the parts where judgment helps — ranking the next experiments, curating new sources, summarizing for a digest, flagging collapse alerts. The LLM is called after the deterministic core has gathered the facts, and its output is used to enhance, not to gate.
The principle
Deterministic duties are load-bearing and always execute; the LLM adds judgment only.
Consequences:
- LLM off → agent still useful. The deterministic tick still runs, still logs, still surfaces. You ship the agent before you have API access.
- LLM hallucination → bounded damage. The LLM can't break the load-bearing work; it can only make a bad ranking or a wrong summary. The deterministic state is still correct.
- LLM cost → optional. You can run the deterministic core every tick and the LLM layer only when the budget allows.
- Auditability → intact. The deterministic transcript shows what actually happened; the LLM transcript shows what it suggested.
How to build it
- List the duties. What must happen every run? Those are deterministic — pure code, no LLM.
- List the judgment calls. Where would ranking/curation/synthesis help? Those are the LLM layer — called after the facts are gathered.
- Gate the LLM layer behind a flag + a key.
if LLM_ENABLED: ... — the agent runs either way. The key is the one user-only input.
- Keep the LLM additive-only. The deterministic duties don't read LLM output to decide whether to run. The LLM reads deterministic output to decide what to suggest.
- Test the deterministic core offline. The eval gate runs fully offline (no network, no LLM) — that's the proof the core is load-bearing. Wire it into CI.
Anti-patterns
- LLM-as-backbone. The agent's core work is an LLM call. When the LLM is wrong, the work is wrong; when it's down, the work stops.
- LLM gates the deterministic work. "If the LLM says it's worth doing, do the tick." The LLM is now a single point of failure for the load-bearing work.
- No offline mode. The agent can't run without an API key → can't ship, can't test in CI, can't run in a budget-constrained env.
- Untestable core. The deterministic duties are tangled with LLM calls so you can't verify them without the LLM.
Pair with
agent-guardrails — the deterministic core is what makes bounded loops safe (the agent does its work and stops, doesn't freewheel).validate-gate — the offline eval gate is the proof the core is load-bearing.fill-the-wait — the deterministic tick is a perfect fill task (no LLM, no network, always useful).
1---2name: deterministic-core-llm-judgment3description: Build agents where the deterministic duties are the load-bearing backbone (always execute, reliable, auditable) and the LLM only adds an optional judgment layer on top — ranking, curation, synthesis. The agent must do its core work with the LLM layer off; the LLM only makes the work better. Use when designing or building an autonomous agent, a recurring agent loop, or any system where reliability matters more than cleverness.4---56# Deterministic Core, LLM Judgment78An agent where the LLM is the backbone fails when the LLM hallucinates, drifts, is unavailable, or costs too much. An agent where the deterministic duties are the backbone *always runs* and the LLM only adds judgment on top. If the LLM is off, the work still happens; if it's on, the work is better. This is the robust agent architecture.910## The split1112- **Deterministic core (load-bearing, always executes):** the duties that must happen every run regardless of LLM availability — fetching the queue, snapshotting state, computing metrics, writing the run log, surfacing blockers. Pure code, no LLM calls. Testable, auditable, no hallucination.13- **LLM judgment layer (additive, optional):** the parts where judgment helps — ranking the next experiments, curating new sources, summarizing for a digest, flagging collapse alerts. The LLM is called *after* the deterministic core has gathered the facts, and its output is used to enhance, not to gate.1415## The principle1617> Deterministic duties are load-bearing and always execute; the LLM adds judgment only.1819Consequences:20- **LLM off → agent still useful.** The deterministic tick still runs, still logs, still surfaces. You ship the agent before you have API access.21- **LLM hallucination → bounded damage.** The LLM can't break the load-bearing work; it can only make a bad ranking or a wrong summary. The deterministic state is still correct.22- **LLM cost → optional.** You can run the deterministic core every tick and the LLM layer only when the budget allows.23- **Auditability → intact.** The deterministic transcript shows what actually happened; the LLM transcript shows what it suggested.2425## How to build it26271. **List the duties.** What must happen every run? Those are deterministic — pure code, no LLM.282. **List the judgment calls.** Where would ranking/curation/synthesis help? Those are the LLM layer — called after the facts are gathered.293. **Gate the LLM layer behind a flag + a key.** `if LLM_ENABLED: ...` — the agent runs either way. The key is the one user-only input.304. **Keep the LLM additive-only.** The deterministic duties don't read LLM output to decide whether to run. The LLM reads deterministic output to decide what to suggest.315. **Test the deterministic core offline.** The eval gate runs fully offline (no network, no LLM) — that's the proof the core is load-bearing. Wire it into CI.3233## Anti-patterns3435- **LLM-as-backbone.** The agent's core work is an LLM call. When the LLM is wrong, the work is wrong; when it's down, the work stops.36- **LLM gates the deterministic work.** "If the LLM says it's worth doing, do the tick." The LLM is now a single point of failure for the load-bearing work.37- **No offline mode.** The agent can't run without an API key → can't ship, can't test in CI, can't run in a budget-constrained env.38- **Untestable core.** The deterministic duties are tangled with LLM calls so you can't verify them without the LLM.3940## Pair with4142- `agent-guardrails` — the deterministic core is what makes bounded loops safe (the agent does its work and stops, doesn't freewheel).43- `validate-gate` — the offline eval gate is the proof the core is load-bearing.44- `fill-the-wait` — the deterministic tick is a perfect fill task (no LLM, no network, always useful).
jcdavis131