Goal
Produce a complete, timestamped audit trail of a Self-RAG reasoning session that captures every retrieval decision, relevance assessment, support evaluation, and claim-to-source mapping. This trace serves as both a quality assurance artifactΓÇöallowing human reviewers to independently verify every evidence-backed claimΓÇöand a feedback mechanism for improving future retrieval strategies by identifying patterns in relevance failures, support gaps, and retrieval timing errors.
The trace is designed to be machine-parseable (JSON-structured within markdown) so it can feed into automated testing pipelines, regression suites for strategy output quality, and continuous improvement loops where retrieval-pattern analytics surface which query formulations and source types most reliably produce [SUPPORTED] outcomes.
Implementation Details
The Self-RAG framework, introduced by Asai et al. (2023) in "Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection," centers on a single language model fine-tuned to emit special reflection tokens that govern retrieval and critique behavior at inference time. These tokensΓÇö[RETRIEVE], [RELEVANT], [IRRELEVANT], [SUPPORTED], [PARTIALLY], and [UNSUPPORTED]ΓÇöform a compact control language that the model uses to decide on-demand whether to retrieve external knowledge for a given generation step. The critical departure from standard RAG is that retrieval is conditional: the model learns to recognize when its parametric knowledge is sufficient and when external context would improve output quality. This adaptive mechanism is what enables Self-RAG to outperform both ChatGPT and retrieval-augmented Llama-2-chat on open-domain QA (Asai et al., 2023, Table 2) and fact verification benchmarks (Asai et al., 2023, Table 3), because it avoids the well-documented degradation that occurs when irrelevant passages are injected into the context window.
The reflection token taxonomy serves distinct functions within the generation pipeline. The [RETRIEVE] token is a binary decision emitted before each generation segment: when present, the system queries an external retriever (typically a sparse or dense index over a document corpus, or a live web search endpoint) and returns the top-k passages. Each retrieved passage is then evaluated for relevance to the current query using [RELEVANT] or [IRRELEVANT] tokens; passages marked [IRRELEVANT] are discarded immediately to prevent context pollution. The surviving passages are concatenated to the generation context, and the model produces output annotated with [SUPPORTED] (the generated text is fully grounded in a retrieved passage), [PARTIALLY] (the generated text draws partial support from retrieval, with gaps filled by parametric knowledge), or [UNSUPPORTED] (the generated text has no evidential basis in the retrieved set and derives entirely from the model's internal knowledge). This taxonomy enables fine-grained factuality auditing: a human reviewer can trace every [SUPPORTED] segment to its source passage and independently verify accuracy, while [UNSUPPORTED] segments are explicitly flagged as model-generated and warrant additional scrutiny.
The on-demand retrieval decision is the framework's most impactful design choice. In standard RAG pipelines, retrieval is a fixed preprocessing step that runs before any generation beginsΓÇöevery query triggers retrieval regardless of whether the model actually needs external knowledge. This produces two failure modes: retrieval of irrelevant passages that distract the model and degrade output quality, and wasted computational cost for queries the model could answer correctly from parametric knowledge alone. Self-RAG's adaptive mechanism eliminates both failures by conditioning retrieval on the specific reasoning step. The model is trained (via instruction fine-tuning on data annotated with reflection tokens by a critic model) to recognize knowledge boundaries and request retrieval only when a gap exists. For long-form generation tasks, this produces measurable gains in both factuality (as evaluated by automated fact-checking metrics) and citation accuracy (the precision with which generated claims can be attributed to specific source passages), because the model is not overwhelmed by irrelevant context and can focus its attention on the passages that matter.
The practical implementation for this trace system maps the Self-RAG framework onto Vidbyte's strategy execution workflow. Each trace entry records: the step identifier within the algorithm (matching the eight-step Algorithm section of the self-rag-reasoning skill), a millisecond-precision timestamp, the retrieval decision (retrieve or skip) with rationale, the query string sent to the retriever, the number of passages returned, the relevance filter outcome per passage (accepted/discarded with reason), the generated output segment, and the support annotation per claim with a pointer to the source passage or a null marker for model-generated claims. The trace concludes with an aggregate support summary: the ratio of [SUPPORTED] to [UNSUPPORTED] claims, the average retrieval latency, the relevance precision (accepted passages / total retrieved), and a per-section confidence score derived from the support density of claims within that section. This structured output enables both immediate human audit and downstream programmatic analysisΓÇöfor example, identifying which query formulations most reliably surface relevant passages for a given strategy domain, or detecting strategy steps where the model consistently overclaims support.
Intent
We run this trace because retrieval-augmented generation is only as trustworthy as its retrieval decisions are auditable. Standard RAG pipelines produce answers that cite sources, but without a record of what was not retrieved, what was retrieved and discarded as irrelevant, and which claims genuinely drew support from which passages, the consumer cannot distinguish well-grounded answers from answers that cherry-pick supportive passages while ignoring contradictory ones. Self-RAG's reflection token mechanism—the [RETRIEVE], [RELEVANT], [IRRELEVANT], [SUPPORTED], [PARTIALLY], and [UNSUPPORTED] tags—creates a structured decision record at every step of the generation pipeline, and the trace's purpose is to surface that record in a human-auditable and machine-parseable format. Every retrieval decision, relevance filter, and support annotation becomes part of the public record, so a reviewer can independently verify the evidential basis of every claim.
The trace also serves as a feedback mechanism for improving retrieval strategy. By recording not just what was retrieved and used, but also what was retrieved and discarded, the trace enables retrospective analysis of retrieval quality: which query formulations produce high relevance precision, which source types yield the highest ratio of [SUPPORTED] to [UNSUPPORTED] claims, and which strategy steps trigger unnecessary retrievals that add latency without improving output quality. Over multiple sessions, this data accumulates into a retrieval performance profile that can inform query template optimization, retriever selection, and threshold tuning for the relevance filter—turning each trace from a one-time audit artifact into a building block for continuous improvement of the Self-RAG pipeline.
Algorithm for the Output Document
Restate the strategy question, constraints, evidence standard, and domain classification. Identify the information needs of the question—which claims will require external evidence and which can be answered from parametric knowledge alone—and establish the retrieval sources available (web search, knowledge base, document corpus).
For each reasoning step in the Self-RAG algorithm, emit the retrieval decision: either [RETRIEVE], triggering a query to the configured retriever, or skip, relying on parametric knowledge. Record the decision timestamp, the rationale for retrieving or skipping, and if retrieving, the exact query string sent to the retriever. If multiple query formulations are attempted for the same step, record each attempt and its result count.
For each retrieval execution, record the number of passages returned, the retrieval latency in milliseconds, and the source identifier (search endpoint URL, knowledge base index name, or document corpus identifier). Log each retrieved passage in full or as a content excerpt with its source URL or document identifier, so the passage can be independently verified by a human reviewer.
For each retrieved passage, apply the relevance filter: emit [RELEVANT] or [IRRELEVANT] with a one-sentence rationale for the determination. Passages marked [IRRELEVANT] are recorded in the trace with their discard reason rather than silently dropped, enabling downstream analysis of relevance precision and the identification of query formulations that surface too many irrelevant passages. Passages marked [RELEVANT] are concatenated to the generation context.
Generate the output for the current reasoning step using the surviving relevant passages as context. Produce the generated text with inline support annotations: every factual claim receives [SUPPORTED] with a pointer to the specific source passage that grounds it, [PARTIALLY] with a note on which portion is supported and which is parametric, or [UNSUPPORTED] for claims with no evidential basis in the retrieved set.
Iterate through all reasoning steps, repeating the retrieve-filter-generate cycle for each step where the model determines that external knowledge is needed. Record each cycle as a distinct trace entry with its own timestamp, retrieval decision, query, relevance determinations, and generation output. The trace must preserve the sequential order of steps so a reviewer can follow the evidence flow chronologically.
After all reasoning steps are complete, produce the aggregate support summary. Compute and report: the total number of claims made, the count and percentage of [SUPPORTED] claims, the count and percentage of [PARTIALLY] claims, the count and percentage of [UNSUPPORTED] claims, the relevance precision (accepted passages divided by total retrieved), the average retrieval latency per query, and a per-section confidence score derived from the support density of claims within that section.
Assemble the complete Self-RAG trace document. Structure it as valid markdown with JSON-structured trace entries for machine parseability, with sections for Retrieval Configuration (sources and parameters), Step-by-Step Execution (each step with its retrieval decision, query, passage log, relevance determinations, generation output, and support annotations), Support Summary (aggregate statistics), and Confidence Assessment (per-section scores with interpretation). The trace must enable a human reviewer to independently verify every [SUPPORTED] claim against its source passage without access to the original LLM session.
Output Information
Write the Self-RAG trace as structured markdown with JSON-parseable trace entries in root memory/{question_name}.md before responding to the user. Include this scale note near the top of the file: "Scale: the trace length scales with the number of retrieval decisions and the volume of retrieved passages—a session with 3 retrievals and 12 passages produces a shorter trace than one with 12 retrievals and 50 passages. Every retrieval decision, passage, relevance determination, and support annotation receives a full entry." Use numbered entries for each step in the execution sequence so a reviewer can cross-reference retrieval decisions with their downstream effects on generation quality. Keep the trace public, inspectable, and complete—no passage is silently discarded, no support annotation is omitted, and no retrieval decision is summarized instead of recorded. After writing the file, respond with the path, the support summary statistics, the relevance precision, and a brief statement of overall evidential confidence.
Things Not to Do
- Do not silently discard irrelevant passages. Every retrieved passage—whether accepted or rejected—must appear in the trace with its content excerpt, source identifier, and relevance determination rationale. Discarding passages from the trace makes retrieval-quality analysis impossible and hides the retriever's failure modes.
- Do not fabricate support annotations where evidence is missing. If a generated claim has no grounding in any retrieved passage, it must be marked [UNSUPPORTED]—not silently presented as [SUPPORTED] by stretching the interpretation of a tangentially related passage. Overclaiming support is worse than acknowledging parametric generation because it misleads the reviewer about the answer's evidential basis.
- Do not treat [PARTIALLY] as a dumping ground for ambiguous claims. Every [PARTIALLY] annotation must specify which portion of the claim is supported by which passage and which portion is parametric. An annotation that says "partially supported" without decomposing the claim into supported and unsupported segments provides no audit value.
- Do not force retrieval when parametric knowledge is sufficient. The Self-RAG framework's central innovation is on-demand retrieval—the model should skip retrieval for steps where its internal knowledge is adequate. Forcing retrieval on every step wastes latency, adds irrelevant context, and produces a trace that misrepresents the model's actual retrieval needs.
- Do not summarize or paraphrase retrieved passages in the trace. The trace must record the actual passage content returned by the retriever, not the model's summary of what it thought the passage said. Summaries introduce an interpretation layer between the evidence and the reviewer, defeating the purpose of source-level auditability.
- Do not omit retrieval latency from the trace. Retrieval timing is critical for pipeline optimization—a step that retrieves relevant passages but takes 4 seconds may need a different retriever configuration than one that takes 200 milliseconds. Without latency data, the trace cannot inform retrieval performance optimization.
- Do not conflate retrieval relevance with passage correctness. A passage can be relevant to the query (it addresses the same topic) but factually incorrect or outdated. The relevance filter only addresses topical match; the [SUPPORTED] annotation addresses evidential grounding. Do not mark a passage as [RELEVANT] because it supports a desired conclusion if it is topically unrelated to the query.
- Do not present aggregate support ratios as a proxy for overall answer correctness. A trace where 90% of claims are [SUPPORTED] does not mean the answer is 90% correct—it means 90% of claims can be traced to passages, but those passages may themselves be incorrect, biased, or incomplete. The support ratio is an evidential grounding metric, not a truth metric.
Success Criteria
- Every retrieval decision is recorded with a timestamp, the query string used, the number of results returned, and the retrieval source (web, knowledge base, document corpus).
- Each retrieved passage is individually logged with its content excerpt, source URL or identifier, and the relevance determination ([RELEVANT] or [IRRELEVANT]) with a one-sentence rationale.
- Every generated claim in the strategy output is mapped to a support annotation and, if [SUPPORTED], a pointer to the specific source passage that grounds it.
- The support summary provides aggregate statistics: total claims, [SUPPORTED] count, [PARTIALLY] count, [UNSUPPORTED] count, and the support ratio.
- The trace is valid JSON and parseable by automated validation scripts without manual cleanup.
- All timestamps use ISO-8601 format with millisecond precision for precise latency analysis.
- Irrelevant passages are recorded in the trace (with discard reason) rather than silently dropped, enabling retrieval-quality analysis.
- The trace includes a per-section confidence score that correlates with the density of [SUPPORTED] claims in that section.
- No [SUPPORTED] claim in the trace points to a source that is inaccessible or unreproducible.
Input
The output of a self-rag-reasoning session: the full strategy output with inline reflection tokens, the retrieval log (query strings, result counts, passage contents), and the support annotations per claim. The trace generator consumes this structured output and produces the formatted audit trail. No additional user input is required beyond the raw session output.
1---2name: self-rag-trace3description: Standalone execution trace that records every retrieval decision, relevance filter, and support annotation across an entire Self-RAG reasoning session for auditability and continuous improvement.4---56## Goal78Produce a complete, timestamped audit trail of a Self-RAG reasoning session that captures every retrieval decision, relevance assessment, support evaluation, and claim-to-source mapping. This trace serves as both a quality assurance artifactΓÇöallowing human reviewers to independently verify every evidence-backed claimΓÇöand a feedback mechanism for improving future retrieval strategies by identifying patterns in relevance failures, support gaps, and retrieval timing errors.910The trace is designed to be machine-parseable (JSON-structured within markdown) so it can feed into automated testing pipelines, regression suites for strategy output quality, and continuous improvement loops where retrieval-pattern analytics surface which query formulations and source types most reliably produce [SUPPORTED] outcomes.1112## Implementation Details1314The Self-RAG framework, introduced by Asai et al. (2023) in "Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection," centers on a single language model fine-tuned to emit special reflection tokens that govern retrieval and critique behavior at inference time. These tokensΓÇö[RETRIEVE], [RELEVANT], [IRRELEVANT], [SUPPORTED], [PARTIALLY], and [UNSUPPORTED]ΓÇöform a compact control language that the model uses to decide on-demand whether to retrieve external knowledge for a given generation step. The critical departure from standard RAG is that retrieval is conditional: the model learns to recognize when its parametric knowledge is sufficient and when external context would improve output quality. This adaptive mechanism is what enables Self-RAG to outperform both ChatGPT and retrieval-augmented Llama-2-chat on open-domain QA (Asai et al., 2023, Table 2) and fact verification benchmarks (Asai et al., 2023, Table 3), because it avoids the well-documented degradation that occurs when irrelevant passages are injected into the context window.1516The reflection token taxonomy serves distinct functions within the generation pipeline. The [RETRIEVE] token is a binary decision emitted before each generation segment: when present, the system queries an external retriever (typically a sparse or dense index over a document corpus, or a live web search endpoint) and returns the top-k passages. Each retrieved passage is then evaluated for relevance to the current query using [RELEVANT] or [IRRELEVANT] tokens; passages marked [IRRELEVANT] are discarded immediately to prevent context pollution. The surviving passages are concatenated to the generation context, and the model produces output annotated with [SUPPORTED] (the generated text is fully grounded in a retrieved passage), [PARTIALLY] (the generated text draws partial support from retrieval, with gaps filled by parametric knowledge), or [UNSUPPORTED] (the generated text has no evidential basis in the retrieved set and derives entirely from the model's internal knowledge). This taxonomy enables fine-grained factuality auditing: a human reviewer can trace every [SUPPORTED] segment to its source passage and independently verify accuracy, while [UNSUPPORTED] segments are explicitly flagged as model-generated and warrant additional scrutiny.1718The on-demand retrieval decision is the framework's most impactful design choice. In standard RAG pipelines, retrieval is a fixed preprocessing step that runs before any generation beginsΓÇöevery query triggers retrieval regardless of whether the model actually needs external knowledge. This produces two failure modes: retrieval of irrelevant passages that distract the model and degrade output quality, and wasted computational cost for queries the model could answer correctly from parametric knowledge alone. Self-RAG's adaptive mechanism eliminates both failures by conditioning retrieval on the specific reasoning step. The model is trained (via instruction fine-tuning on data annotated with reflection tokens by a critic model) to recognize knowledge boundaries and request retrieval only when a gap exists. For long-form generation tasks, this produces measurable gains in both factuality (as evaluated by automated fact-checking metrics) and citation accuracy (the precision with which generated claims can be attributed to specific source passages), because the model is not overwhelmed by irrelevant context and can focus its attention on the passages that matter.1920The practical implementation for this trace system maps the Self-RAG framework onto Vidbyte's strategy execution workflow. Each trace entry records: the step identifier within the algorithm (matching the eight-step Algorithm section of the self-rag-reasoning skill), a millisecond-precision timestamp, the retrieval decision (retrieve or skip) with rationale, the query string sent to the retriever, the number of passages returned, the relevance filter outcome per passage (accepted/discarded with reason), the generated output segment, and the support annotation per claim with a pointer to the source passage or a null marker for model-generated claims. The trace concludes with an aggregate support summary: the ratio of [SUPPORTED] to [UNSUPPORTED] claims, the average retrieval latency, the relevance precision (accepted passages / total retrieved), and a per-section confidence score derived from the support density of claims within that section. This structured output enables both immediate human audit and downstream programmatic analysisΓÇöfor example, identifying which query formulations most reliably surface relevant passages for a given strategy domain, or detecting strategy steps where the model consistently overclaims support.2122## Intent2324We run this trace because retrieval-augmented generation is only as trustworthy as its retrieval decisions are auditable. Standard RAG pipelines produce answers that cite sources, but without a record of what was not retrieved, what was retrieved and discarded as irrelevant, and which claims genuinely drew support from which passages, the consumer cannot distinguish well-grounded answers from answers that cherry-pick supportive passages while ignoring contradictory ones. Self-RAG's reflection token mechanism—the [RETRIEVE], [RELEVANT], [IRRELEVANT], [SUPPORTED], [PARTIALLY], and [UNSUPPORTED] tags—creates a structured decision record at every step of the generation pipeline, and the trace's purpose is to surface that record in a human-auditable and machine-parseable format. Every retrieval decision, relevance filter, and support annotation becomes part of the public record, so a reviewer can independently verify the evidential basis of every claim.2526The trace also serves as a feedback mechanism for improving retrieval strategy. By recording not just what was retrieved and used, but also what was retrieved and discarded, the trace enables retrospective analysis of retrieval quality: which query formulations produce high relevance precision, which source types yield the highest ratio of [SUPPORTED] to [UNSUPPORTED] claims, and which strategy steps trigger unnecessary retrievals that add latency without improving output quality. Over multiple sessions, this data accumulates into a retrieval performance profile that can inform query template optimization, retriever selection, and threshold tuning for the relevance filter—turning each trace from a one-time audit artifact into a building block for continuous improvement of the Self-RAG pipeline.2728## Algorithm for the Output Document29301. Restate the strategy question, constraints, evidence standard, and domain classification. Identify the information needs of the question—which claims will require external evidence and which can be answered from parametric knowledge alone—and establish the retrieval sources available (web search, knowledge base, document corpus).31322. For each reasoning step in the Self-RAG algorithm, emit the retrieval decision: either [RETRIEVE], triggering a query to the configured retriever, or skip, relying on parametric knowledge. Record the decision timestamp, the rationale for retrieving or skipping, and if retrieving, the exact query string sent to the retriever. If multiple query formulations are attempted for the same step, record each attempt and its result count.33343. For each retrieval execution, record the number of passages returned, the retrieval latency in milliseconds, and the source identifier (search endpoint URL, knowledge base index name, or document corpus identifier). Log each retrieved passage in full or as a content excerpt with its source URL or document identifier, so the passage can be independently verified by a human reviewer.35364. For each retrieved passage, apply the relevance filter: emit [RELEVANT] or [IRRELEVANT] with a one-sentence rationale for the determination. Passages marked [IRRELEVANT] are recorded in the trace with their discard reason rather than silently dropped, enabling downstream analysis of relevance precision and the identification of query formulations that surface too many irrelevant passages. Passages marked [RELEVANT] are concatenated to the generation context.37385. Generate the output for the current reasoning step using the surviving relevant passages as context. Produce the generated text with inline support annotations: every factual claim receives [SUPPORTED] with a pointer to the specific source passage that grounds it, [PARTIALLY] with a note on which portion is supported and which is parametric, or [UNSUPPORTED] for claims with no evidential basis in the retrieved set.39406. Iterate through all reasoning steps, repeating the retrieve-filter-generate cycle for each step where the model determines that external knowledge is needed. Record each cycle as a distinct trace entry with its own timestamp, retrieval decision, query, relevance determinations, and generation output. The trace must preserve the sequential order of steps so a reviewer can follow the evidence flow chronologically.41427. After all reasoning steps are complete, produce the aggregate support summary. Compute and report: the total number of claims made, the count and percentage of [SUPPORTED] claims, the count and percentage of [PARTIALLY] claims, the count and percentage of [UNSUPPORTED] claims, the relevance precision (accepted passages divided by total retrieved), the average retrieval latency per query, and a per-section confidence score derived from the support density of claims within that section.43448. Assemble the complete Self-RAG trace document. Structure it as valid markdown with JSON-structured trace entries for machine parseability, with sections for Retrieval Configuration (sources and parameters), Step-by-Step Execution (each step with its retrieval decision, query, passage log, relevance determinations, generation output, and support annotations), Support Summary (aggregate statistics), and Confidence Assessment (per-section scores with interpretation). The trace must enable a human reviewer to independently verify every [SUPPORTED] claim against its source passage without access to the original LLM session.4546## Output Information4748Write the Self-RAG trace as structured markdown with JSON-parseable trace entries in root memory/{question_name}.md before responding to the user. Include this scale note near the top of the file: "Scale: the trace length scales with the number of retrieval decisions and the volume of retrieved passages—a session with 3 retrievals and 12 passages produces a shorter trace than one with 12 retrievals and 50 passages. Every retrieval decision, passage, relevance determination, and support annotation receives a full entry." Use numbered entries for each step in the execution sequence so a reviewer can cross-reference retrieval decisions with their downstream effects on generation quality. Keep the trace public, inspectable, and complete—no passage is silently discarded, no support annotation is omitted, and no retrieval decision is summarized instead of recorded. After writing the file, respond with the path, the support summary statistics, the relevance precision, and a brief statement of overall evidential confidence.4950## Things Not to Do5152- Do not silently discard irrelevant passages. Every retrieved passage—whether accepted or rejected—must appear in the trace with its content excerpt, source identifier, and relevance determination rationale. Discarding passages from the trace makes retrieval-quality analysis impossible and hides the retriever's failure modes.53- Do not fabricate support annotations where evidence is missing. If a generated claim has no grounding in any retrieved passage, it must be marked [UNSUPPORTED]—not silently presented as [SUPPORTED] by stretching the interpretation of a tangentially related passage. Overclaiming support is worse than acknowledging parametric generation because it misleads the reviewer about the answer's evidential basis.54- Do not treat [PARTIALLY] as a dumping ground for ambiguous claims. Every [PARTIALLY] annotation must specify which portion of the claim is supported by which passage and which portion is parametric. An annotation that says "partially supported" without decomposing the claim into supported and unsupported segments provides no audit value.55- Do not force retrieval when parametric knowledge is sufficient. The Self-RAG framework's central innovation is on-demand retrieval—the model should skip retrieval for steps where its internal knowledge is adequate. Forcing retrieval on every step wastes latency, adds irrelevant context, and produces a trace that misrepresents the model's actual retrieval needs.56- Do not summarize or paraphrase retrieved passages in the trace. The trace must record the actual passage content returned by the retriever, not the model's summary of what it thought the passage said. Summaries introduce an interpretation layer between the evidence and the reviewer, defeating the purpose of source-level auditability.57- Do not omit retrieval latency from the trace. Retrieval timing is critical for pipeline optimization—a step that retrieves relevant passages but takes 4 seconds may need a different retriever configuration than one that takes 200 milliseconds. Without latency data, the trace cannot inform retrieval performance optimization.58- Do not conflate retrieval relevance with passage correctness. A passage can be relevant to the query (it addresses the same topic) but factually incorrect or outdated. The relevance filter only addresses topical match; the [SUPPORTED] annotation addresses evidential grounding. Do not mark a passage as [RELEVANT] because it supports a desired conclusion if it is topically unrelated to the query.59- Do not present aggregate support ratios as a proxy for overall answer correctness. A trace where 90% of claims are [SUPPORTED] does not mean the answer is 90% correct—it means 90% of claims can be traced to passages, but those passages may themselves be incorrect, biased, or incomplete. The support ratio is an evidential grounding metric, not a truth metric.6061## Success Criteria6263- Every retrieval decision is recorded with a timestamp, the query string used, the number of results returned, and the retrieval source (web, knowledge base, document corpus).64- Each retrieved passage is individually logged with its content excerpt, source URL or identifier, and the relevance determination ([RELEVANT] or [IRRELEVANT]) with a one-sentence rationale.65- Every generated claim in the strategy output is mapped to a support annotation and, if [SUPPORTED], a pointer to the specific source passage that grounds it.66- The support summary provides aggregate statistics: total claims, [SUPPORTED] count, [PARTIALLY] count, [UNSUPPORTED] count, and the support ratio.67- The trace is valid JSON and parseable by automated validation scripts without manual cleanup.68- All timestamps use ISO-8601 format with millisecond precision for precise latency analysis.69- Irrelevant passages are recorded in the trace (with discard reason) rather than silently dropped, enabling retrieval-quality analysis.70- The trace includes a per-section confidence score that correlates with the density of [SUPPORTED] claims in that section.71- No [SUPPORTED] claim in the trace points to a source that is inaccessible or unreproducible.7273## Input7475The output of a self-rag-reasoning session: the full strategy output with inline reflection tokens, the retrieval log (query strings, result counts, passage contents), and the support annotations per claim. The trace generator consumes this structured output and produces the formatted audit trail. No additional user input is required beyond the raw session output.
cerredz