Dspy Rlm Hermes Runtime Auth

DSPy RLM + Hermes Runtime Auth

Companion skill to dspy-hermes-runtime-auth. Use this skill when the question is specifically: “Should I use dspy.RLM here, and if so, how do I wire it to Hermes runtime auth?”

What RLM is

dspy.RLM is not just “more context” and not just “an agent with tools.”

Per the original RLM paper, the core idea is:

• treat the long prompt or corpus as an external environment
• let the model programmatically inspect, decompose, and transform that environment
• let it make recursive sub-calls over constructed slices of the context instead of forcing one giant prompt through a single context window

In practice, that means a persistent REPL-style environment is often a better fit than a monolithic prompt when the model needs to:

• inspect many parts of the input
• compute over them
• compare them
• aggregate them
• recurse on subsets

When to use this skill

Use this skill when both are true:

• you want dspy.RLM to reuse the same provider / base URL / auth Hermes is already using
• the task is a genuine RLM-shaped problem, not just a normal prompt that happens to be long

High-signal triggers

Reach for dspy.RLM when one or more of these are true:

• the context is too large, too fragile, or too distributed for one-shot prompting
• the answer depends on many dispersed details, not one or two passages
• the task requires dense access throughout the prompt and would be damaged by aggressive summarization/compaction
• the model benefits from programmatic operations like grep/filter/chunk/count/map/reduce/compare before answering
• the problem naturally decomposes into recursive subquestions over slices, files, documents, or candidate pairs
• the input is a large codebase, document corpus, transcript set, or web/document bundle where understanding comes from inspecting many local pieces and combining them

Canonical use cases from the paper and surrounding ecosystem

These are the strongest shareable use cases surfaced by the paper, author docs/blog, ecosystem writeups, and X/Twitter discussion:

• deep research over large corpora — many documents, many candidate sources, answer assembled across the corpus rather than retrieved from one snippet
• information aggregation — the system must synthesize many pieces scattered across documents or long prompts
• code repository understanding — reasoning across many files/functions/modules rather than a single local excerpt
• long-context tasks where the answer depends on almost every line or many pairwise comparisons — the task is not “find one fact,” but “understand structure across the whole thing”
• massive-context browsing / comparison tasks — especially when context compaction or naïve retrieval starts dropping relevant structure

When not to use this skill

Prefer a simpler path when:

• the prompt already fits comfortably and a normal dspy.Predict / ChainOfThought call is enough
• standard retrieval can surface the few passages that matter
• the task is mostly summarization or generation, with little need for computation over the input
• the challenge is primarily external tool interaction, not recursive understanding of a large context object
• the problem is a normal DSPy task and you mainly need Hermes auth portability — in that case use dspy-hermes-runtime-auth

RLM vs nearby alternatives

Use plain DSPy (Predict, ChainOfThought) when

• the prompt fits
• the answer path is short
• decomposition is unnecessary

Use retrieval / RAG when

• only a small number of passages matter
• good retrieval is likely to surface them
• you do not need broad computation over the corpus

Use compaction / summarization when

• losing some early detail is acceptable
• the task does not require dense access across the original material

Use dspy.RLM when

• you do not want the system to pre-commit to a shallow decomposition
• the model should decide how to inspect and recurse over the context
• symbolic/programmatic interaction with the context is a feature, not a workaround

What this skill does

It gives you an RLM-oriented runtime pattern:

1. resolve Hermes runtime credentials with hermes_cli.runtime_provider.resolve_runtime_provider()
2. build a DSPy-compatible LM that follows Hermes runtime routing
3. pass that LM into dspy.RLM(..., sub_lm=lm)

That keeps RLM code portable across Hermes users even if they change:

• provider
• model
• base URL
• auth mode

Verified shape

Verified locally as a small working example against one Hermes installation:

• DSPy 3.1.3
• dspy.RLM available
• deno installed
• helper works for a tiny dspy.RLM(...) run
• the live verification environment happened to resolve to openai-codex, which is why the helper includes a narrow Codex compatibility branch

Interpret this as:

• generic design across Hermes providers via resolve_runtime_provider()
• concrete runtime verification on one live install

Important version pitfall

Do not assume the target repo's own environment has a new enough DSPy.

A project can pin an older DSPy release even when its normal install flow succeeds. In one live verification case, the repo's standard sync path installed DSPy 3.0.4, and that build did not expose dspy.RLM. The workaround was to run RLM in a separate temporary environment with DSPy 3.1.3 while keeping Hermes runtime auth importable through PYTHONPATH.

Practical rule:

• first verify hasattr(dspy, "RLM")
• if false, do not keep debugging the repo env as if RLM should be there
• instead, spin up an isolated uv run --with dspy-ai==3.1.3 --with openai ... environment and point PYTHONPATH at your Hermes agent checkout so the runtime helper imports cleanly

Important limits

• Do not run billable benchmark/eval loops without the user’s approval.
• Do not print tokens, .env, or auth-store contents.
• dspy.RLM needs Deno/Pyodide available on the machine.
• Some providers may need provider-specific adaptation even when runtime auth resolution succeeds.
• Today, the explicitly verified special-case branch is the Codex/Responses path; other providers follow the standard DSPy/OpenAI-compatible route.

Recommended path

Use the helper template in templates/hermes_dspy_runtime.py.

Steps

1. Build the LM from Hermes runtime

from hermes_dspy_runtime import load_hermes_dspy_lm

lm = load_hermes_dspy_lm()

You can also override provider/model while still using Hermes auth/config:

lm = load_hermes_dspy_lm(requested_provider="anthropic")
lm = load_hermes_dspy_lm(model="claude-sonnet-4-5-20250929")
lm = load_hermes_dspy_lm(
    requested_provider="openrouter",
    model="google/gemini-3-flash-preview",
)

2. Configure DSPy and create the RLM

import dspy

lm = load_hermes_dspy_lm()
dspy.configure(lm=lm)

rlm = dspy.RLM(
    "context, query -> answer",
    sub_lm=lm,
    max_iterations=20,
)

3. Use RLM on an RLM-shaped task

result = rlm(
    context=long_text_or_corpus,
    query="Answer the question by programmatically exploring the context."
)
print(result.answer)

Files

• templates/hermes_dspy_runtime.py — provider-aware DSPy loader
• templates/hermes_dspy_rlm_web_search_experiment.py — additive experimental template that exposes Hermes web_search inside dspy.RLM
• references/when-to-use-rlm.md — background and invocation guidance synthesized from paper/blog/web/X research
• references/version-pitfall-repo-pinned-dspy.md — guidance for repos that pin an older DSPy without exposing dspy.RLM

Codex-specific note

When Hermes resolves to openai-codex, the helper does not use bare dspy.LM(..., model_type="responses"). It instead routes through Hermes's Codex auxiliary shim because the ChatGPT Codex endpoint expected additional request semantics in live testing.

Anti-patterns

Do not:

• use RLM just because the prompt is “kind of long”
• use RLM when retrieval alone is enough
• assume a fixed provider/model combination instead of reading Hermes runtime state
• dump resolved runtime secrets to stdout/logs
• confuse RLM with generic sub-agents or generic tool use

Output expectation

This skill is for RLM-specific invocation judgment plus runtime wiring. Use it when the bottleneck is: “decide whether this is truly an RLM problem, and if it is, run dspy.RLM on Hermes auth cleanly.”