Research Deep Dive

Research Deep Dive

Purpose

Turn a vague research question into an evidence-oriented technical assessment.

This skill is for:

• papers
• method families
• modeling directions
• algorithmic choices
• technical paradigms
• "should we even try this?" questions

The goal is not to retell the source material. The goal is to determine:

• what works
• under what assumptions
• where it fails
• what is transferable to practice
• what still needs experimental verification

Use when

Use this skill when the user wants to:

• understand a paper beyond a summary
• compare a method to neighboring approaches
• evaluate whether a direction is promising
• inspect assumptions and hidden constraints
• determine applicability to a concrete task
• turn literature understanding into next steps

Typical trigger phrases:

• "разбери статью глубоко"
• "хочу понять, что тут реально работает"
• "стоит ли брать это в работу"
• "разложи направление по полочкам"
• "сравни этот подход с соседними"
• "what actually works here"
• "analyze the assumptions and failure modes"
• "is this direction worth trying"

Do not use when

Do not use this skill for:

• short summaries
• translation
• beginner-friendly explainers
• bibliography collection without analysis
• local codebase understanding
• product market research
• generic brainstorming with no technical object

Inputs

Expected inputs:

• paper, papers, method, topic, or direction
• user goal or downstream task
• optional domain constraints
• optional target setting where applicability matters

Optional inputs:

• benchmark context
• compute or latency constraints
• data regime
• deployment constraints
• code or implementation artifacts related to the method

Outputs

Always produce:

1. problem formalization
2. taxonomy of approaches
3. assumptions
4. failure modes
5. comparison with adjacent methods
6. claims that should not be trusted without verification
7. experimental checks to run
8. final conclusion in the format:
   • take into work
   • do not take into work
   • remains unknown

Core principles

• Do not confuse reported results with robust evidence.
• Do not treat benchmark success as universal validity.
• Do not accept claims without checking assumptions, data regime, and evaluation setup.
• Separate theory, evidence, and speculation.
• Make uncertainty explicit.

Constraints

• Start from the user's actual downstream goal when it is known.
• Do not analyze a method in isolation when meaningful adjacent baselines exist.
• Prefer primary sources and author-provided evaluation details when available.
• Treat sweeping claims about generality, robustness, sample efficiency, or transfer as suspect until checked.
• End with a real judgment, not a neutral restatement.

Procedure

1. Formalize the actual problem. Before discussing methods, define:

   • task
   • inputs and outputs
   • objective
   • relevant constraints
   • the setting in which the method is supposed to work

   If the user's downstream use case is known, specialize the analysis to that setting.

2. Identify the object of analysis. Determine whether the request is about:

   • one paper
   • a cluster of papers
   • a broader method family
   • a research direction
   • a modeling paradigm

3. Build a taxonomy. Organize the relevant approaches into a compact structure. The taxonomy should reflect meaningful distinctions, such as:

   • modeling assumptions
   • supervision regime
   • inference pattern
   • optimization structure
   • data requirements
   • architectural bias
   • online vs offline setting
   • generative vs discriminative use

   The taxonomy must help explain why methods differ, not just list names.

4. Extract assumptions. For each main approach, identify:

   • data assumptions
   • distribution assumptions
   • observability assumptions
   • stationarity assumptions
   • independence or factorization assumptions
   • optimization assumptions
   • compute or memory assumptions
   • annotation or labeling assumptions
   • evaluation assumptions

   Mark which assumptions are essential and which are merely convenient.

5. Identify what actually drives success. Separate:

   • core idea
   • implementation detail
   • benchmark-specific trick
   • scaling effect
   • data advantage
   • training recipe advantage

   Ask whether the apparent gain is due to the method itself or to surrounding setup.

6. Enumerate failure modes. For each relevant approach, identify:

   • where it breaks
   • what data regimes hurt it
   • what misspecification hurts it
   • what deployment constraints hurt it
   • what optimization pathologies appear
   • what robustness or generalization gaps are likely

   Prefer concrete failure modes over vague caveats.

7. Compare with adjacent methods. Explicitly compare the target approach with neighboring alternatives. For each comparison, state:

   • where the target method is stronger
   • where it is weaker
   • what trade-off is being paid
   • whether the difference is theoretical, empirical, or purely practical

8. Audit claims. For important claims, classify them as:

   • well supported
   • plausible but under-validated
   • benchmark-specific
   • likely confounded
   • not credible without stronger evidence

   Be especially suspicious of claims about:

   • generality
   • robustness
   • sample efficiency
   • scalability
   • transferability
   • causal interpretation
   • real-world applicability

9. Translate to experiments. Propose the smallest useful set of experiments needed to validate applicability. Prefer experiments that test:

   • the key claimed advantage
   • the central assumption
   • the likely failure mode
   • the transfer to the user's actual setting

   Make the experiments diagnostic, not decorative.

10. Produce the final decision. End with:

• what to take into work
• what not to take into work
• what remains unknown
• the next cheapest clarifying experiment or reading step

Subagent use

Use subagents only when the topic is broad enough that bounded exploration helps:

• one subagent for literature or official-source collection
• one subagent for focused comparison of adjacent methods
• one subagent for extracting assumptions or evaluation caveats

Each subagent must return a compact memo. The main thread should synthesize the memos into one judgment.

Do not use subagents for a narrow single-paper read unless the topic is unusually dense.

If a reusable custom subagent is available, prefer research-scout for bounded evidence collection.

Decision rules

Evidence rule

Do not equate a paper claim with reality. Treat every major claim as conditional on assumptions, setup, and evaluation quality.

Applicability rule

Always translate the method into the user's likely deployment or experiment setting. A method can be strong in general and still wrong for the actual use case.

Adjacency rule

Never analyze a method in isolation if meaningful neighboring baselines or alternatives exist.

Fragility rule

If performance appears to depend heavily on scaling, data curation, tricks, or evaluation choices, say that explicitly instead of attributing the result to the core idea.

Experiment rule

Prefer 2-4 diagnostic experiments that answer real uncertainties over a broad but shallow experiment list.

References

Use the supporting references when needed:

• references/deep-dive-template.md
• references/claims-audit-checklist.md
• references/adjacent-method-comparison-template.md
• references/diagnostic-experiments-template.md

Definition of done

• The problem is formalized in the user's actual setting.
• A meaningful taxonomy or method structure exists.
• Assumptions and failure modes are explicit.
• Claims are audited rather than repeated.
• Adjacent methods are compared where relevant.
• The output ends with an actionable judgment, not just a summary.

Final response format

Return the final result in this structure:

• Problem formalization
• Taxonomy of approaches
• Target method / direction
• Assumptions
• What seems to actually work
• Failure modes
• Comparison to adjacent methods
• Claims not to trust without verification
• Experiments to run
• Conclusion:
  • Take into work
  • Do not take into work
  • Remains unknown

Positive examples

Use this skill for:

• "разбери paper глубоко"
• "хочу понять, работает ли этот подход"
• "сравни это направление с соседними"
• "какие у метода assumptions и где он ломается"
• "стоит ли пробовать это в нашем сеттинге"
• "go beyond summary and tell me what actually works"

Negative examples

Do not use this skill for:

• "переведи abstract"
• "сделай краткое summary"
• "объясни простыми словами"
• "найди функцию в этом репозитории"
• "собери список ссылок без анализа"