majiayu000 Critique Dimensions for Research Review
Load when reviewing research documents. Apply each dimension systematically.
Dimension 1: Source Selection Bias
Check: contradictory viewpoints included? | Multiple organizations/authors/perspectives? | Geographic/temporal diversity? | Sources truly independent (not circular)?
Flags: 60%+ from single org/author -> critical | All supporting same conclusion without counterpoint -> critical | Single geographic region -> medium | Clustered publication dates -> medium
Dimension 2: Evidence Quality
Check: every major claim cited | sources reputable (peer-reviewed, official, established) | primary over secondary | technical sources recent (5 years) | confidence matches evidence
Flags: uncited claim -> high | blog/forum for factual claim -> high | all secondary sources -> medium | sources >5 years for tech -> medium | high confidence with 1-2 sources -> high
Dimension 3: Replicability
Check: search strategy documented | source selection criteria explicit | methodology transparent | confidence levels with rationale
Flags: no methodology section -> high | vague methodology ("searched the web") -> medium | no confidence ratings -> medium
Dimension 4: Priority Validation
For research driving architectural/strategic decisions.
Q1: Is this the largest bottleneck? (timing/measurement data?) | Q2: Simpler alternatives considered and rejected with evidence? | Q3: Constraint prioritization correct? (>50% solution for <30% problem = flag) | Q4: Key decision data-justified?
Flags: secondary concern addressed while larger exists -> critical | no measurement data for performance -> high | alternatives not documented -> high | prioritization not explicit -> medium
Output template:
priority_validation:
q1_largest_bottleneck:
evidence: "{timing data or 'NOT PROVIDED'}"
assessment: "YES|NO|UNCLEAR"
q2_simple_alternatives:
assessment: "ADEQUATE|INADEQUATE|MISSING"
q3_constraint_prioritization:
minority_constraint_dominating: "YES|NO"
assessment: "CORRECT|INVERTED|NOT_ANALYZED"
q4_data_justified:
assessment: "JUSTIFIED|UNJUSTIFIED|NO_DATA"
verdict: "PASS|FAIL"
Dimension 5: Completeness
Check: knowledge gaps documented (what searched, why insufficient) | conflicting info acknowledged with credibility analysis | all required sections present (summary, findings, sources, gaps, citations) | research metadata included
Flags: missing gaps section when gaps exist -> critical | conflicting sources unacknowledged -> high | missing required sections -> high | no metadata -> medium
Review Output Template
review_id: "research_rev_{timestamp}"
reviewer: "nw-researcher-reviewer (Scholar)"
issues_identified:
source_bias:
- issue: "{specific description with numbers}"
severity: "critical|high|medium"
recommendation: "{actionable fix}"
evidence_quality:
- issue: "{specific claim or location}"
severity: "critical|high|medium"
recommendation: "{actionable fix}"
replicability:
- issue: "{what is missing}"
severity: "critical|high|medium"
recommendation: "{actionable fix}"
priority_validation:
- issue: "{mismatch description}"
severity: "critical|high|medium"
recommendation: "{actionable fix}"
completeness:
- issue: "{missing element}"
severity: "critical|high|medium"
recommendation: "{actionable fix}"
quality_scores:
source_bias: 0.00
evidence_quality: 0.00
replicability: 0.00
completeness: 0.00
priority_validation: 0.00
approval_status: "approved|rejected_pending_revisions"
blocking_issues:
- "{critical issue 1}"
iteration: 1
max_iterations: 2