tomevault-io Analysis Methodology
A systematic approach to converting unknowns to knowns through structured investigation.
Core Principle
Objective: Every analysis session should reduce uncertainty. If an unknown cannot be resolved, document why and what is needed to close it.
Confidence Levels
Classify every finding by its evidence strength:
| Level | Label | Meaning | Example |
|---|---|---|---|
| 1 | Proven | Verified by code execution, POC, or reproducible test. | "Running npm test -- --filter=X confirms the error." |
| 2 | Observed | Seen in logs, monitoring, or direct inspection, but not isolated. | "The error appears in production logs at 3am." |
| 3 | Inferred | Derived from documentation, patterns, or logical deduction. | "The API docs suggest this should return 404." |
Rule: Findings at Level 3 (Inferred) should be flagged for upgrade to Level 1 (Proven) before being used for decisions.
Gap Tracking Template
Use this structure to surface remaining unknowns:
## Remaining Gaps
| # | Unknown | Blocker | Required Action | Owner |
|---|---------|---------|-----------------|-------|
| 1 | Why does X fail under load? | Cannot reproduce locally. | Need staging access or load test harness. | [TBD] |
| 2 | Does API Y support pagination? | Docs unclear. | Contact vendor or run POC against sandbox. | [TBD] |
Behaviors:
- Populate this table during investigation, not just at the end.
- Surface the table to the user in chat before handoff.
- Mark items as "Resolved" (with link to finding) or "Deferred" (with rationale).
Investigation Techniques
Use these patterns to move unknowns to knowns:
Log Tracing
- Add targeted logging to isolate behavior.
- Compare expected vs actual log sequences.
Component Isolation
- Reproduce behavior with minimal dependencies.
- Use mocks/stubs to eliminate variables.
Binary Search Debugging
- Narrow the failure window by bisecting commits, config changes, or code paths.
- Halve the search space with each step.
POC Execution
- Write minimal, runnable code to prove or disprove a behavior.
- POCs must be reproducible by others (check in or share).
Upstream Tracing
- Follow data/control flow backward to find the root cause.
- Ask: "Where did this value come from?"
Analysis Document Structure
Recommended sections for analysis outputs:
- Changelog — Date, handoff context, outcome summary.
- Value Statement & Objective — Why this analysis matters.
- Context — Background, scope, constraints.
- Methodology — What techniques were used (reference above).
- Findings — Factual results, classified by Confidence Level.
- Gap Tracking Table — (see template above).
- Analysis Recommendations — Next steps to deepen inquiry (not solutions).
- Open Questions — Unresolved items requiring user/agent input.
Handoff Protocol
Before handing off to another agent or user:
- List resolved unknowns — What was determined and at what confidence level.
- List remaining gaps — Use the Gap Tracking Table.
- State blockers explicitly — What prevents further progress.
- Communicate in chat — Do not rely solely on the document; surface gaps directly.
Source: groupzer0/vs-code-agents — distributed by TomeVault.