Protocol & Preregistration (revedres-proposal-and-commissioning)
When to trigger
- Topic selection cleared the four RER fit tests and it is time to lock the plan
- You are about to search but have not written down inclusion/exclusion rules
- You want a PROSPERO / OSF registration so the search is demonstrably a-priori
- A reviewer is likely to ask "were these inclusion rules set before or after you saw the results?"
RER is SUBMITTED, not commissioned — the protocol replaces the commission
Annual Reviews invite their authors; RER reviews are submitted and peer-reviewed, so no editor pre-blesses your scope. The credibility that an invitation would have signaled must instead come from a transparent, a-priori protocol. For a systematic review or meta-analysis this is not optional polish — it is the difference between a synthesis and a fishing expedition. (RER occasionally publishes solicited analytic reviews of special topics, but the default and the bar is a strong unsolicited submission — 检索于 2026-06；以官网为准.)
The protocol: decisions to fix BEFORE you search
Write these down first; a reviewer who senses they were chosen after seeing results will distrust the whole review.
- Question in PICO/PICOS form. Population (learners/settings), Intervention/exposure or construct, Comparison, Outcomes, Study designs. This is the scope.
- Eligibility criteria. Inclusion and exclusion on population, constructs, designs, outcomes, language, publication type (peer-reviewed? grey literature? dissertations?), and date range — each with a reason.
- Information sources & search strategy. Databases (ERIC, PsycINFO, Education Source, Web of Science, Scopus, ProQuest Dissertations), grey-literature sources, and at least one full search string to be reported verbatim.
- Screening procedure. Two independent screeners at title/abstract and full-text; how disagreements are resolved; the reliability statistic you will report.
- Data extraction & coding scheme. What you will code from each study (design, sample, measures, effect sizes, moderators, risk-of-bias) and the codebook.
- Synthesis plan. Meta-analytic model (fixed/random/multilevel/robust variance), effect-size metric, planned moderator and sensitivity analyses, and publication-bias diagnostics — or the explicit qualitative/narrative synthesis method for a non-poolable literature.
- Risk-of-bias / quality appraisal tool chosen in advance (e.g. a design-appropriate instrument).
Preregistration: register the plan, then deviate transparently
- Systematic reviews / meta-analyses: register the protocol on PROSPERO (health-adjacent) or OSF before screening; cite the registration in the manuscript. Use PRISMA-P as the protocol-reporting checklist.
- A registration is a commitment, not a cage — you may change the plan, but you must flag and justify every deviation in the paper (planned vs. exploratory analyses kept distinct).
- Purely conceptual/critical syntheses are not "registered" the same way, but they still need an explicit, written scope and search logic that the paper reports.
Why a-priori discipline pays off at RER specifically
A protocol is not bureaucratic overhead — it is what lets a submitted review be trusted the way an invited one is. Three concrete payoffs:
- It pre-empts the "fishing expedition" objection. A reviewer who sees that inclusion rules, the model, and the moderators were fixed before screening cannot claim they were chosen to produce a clean result.
- It makes the PRISMA flow honest. Counts flow from rules set in advance, so the diagram reports what the search found rather than what survived after-the-fact pruning.
- It separates confirmatory from exploratory. Anything not in the protocol is labeled exploratory in the paper — which protects the headline result and still lets you report interesting surprises.
A common failure is treating the protocol as a draft of the paper. It is not prose; it is a set of decision rules a stranger could execute and reach your corpus. Write it as rules, register it, then let the search run against it.
Checklist
Anti-patterns
- Searching first, then writing inclusion rules to fit the studies you liked
- "Random-effects, REML" chosen after seeing heterogeneity rather than in advance
- No codebook — coders improvise, and reliability cannot be defended later
- Treating registration as a formality, then silently changing the question or outcomes
- Promising an exhaustive search but planning only one database
Output format
【Question (PICOS)】<population / intervention-construct / comparison / outcomes / designs>
【Eligibility】<key inclusion + exclusion rules, each with a reason>
【Sources + search】<databases + grey lit; one full search string drafted? Y/N>
【Screening】dual independent? Y/N; reliability stat = <kappa/agreement>
【Coding scheme】codebook drafted before extraction? Y/N
【Synthesis plan】<meta-analytic model + moderators + bias diagnostics | narrative method>
【Risk-of-bias tool】<instrument, chosen a-priori>
【Registration】PROSPERO/OSF id or "conceptual synthesis — scope written"
【Next step】→ revedres-literature-synthesis (run the search to a PRISMA flow)
Source: brycewang-stanford/Awesome-Journal-Skills → Review-of-Educational-Research-Skills/skills/revedres-proposal-and-commissioning/SKILL.md
1---2name: revedres-proposal-and-commissioning3description: Use when writing the protocol, fixing scope and eligibility, and (where applicable) preregistering a systematic review or meta-analysis before searching, for a Review of Educational Research (RER) manuscript. Produces the a-priori plan; it does not run the search (revedres-literature-synthesis) or impose the conceptual spine (revedres-organizing-framework).4---5
6
7# Protocol & Preregistration (revedres-proposal-and-commissioning)
8
9## When to trigger
10
11- Topic selection cleared the four RER fit tests and it is time to lock the plan
12- You are about to search but have not written down inclusion/exclusion rules
13- You want a PROSPERO / OSF registration so the search is demonstrably a-priori
14- A reviewer is likely to ask "were these inclusion rules set before or after you saw the results?"
15
16## RER is SUBMITTED, not commissioned — the protocol replaces the commission
17
18Annual Reviews *invite* their authors; **RER reviews are submitted and peer-reviewed**, so no editor pre-blesses your scope. The credibility that an invitation would have signaled must instead come from a **transparent, a-priori protocol**. For a systematic review or meta-analysis this is not optional polish — it is the difference between a synthesis and a fishing expedition. (RER occasionally publishes *solicited* analytic reviews of special topics, but the default and the bar is a strong unsolicited submission — 检索于 2026-06；以官网为准.)
19
20## The protocol: decisions to fix BEFORE you search
21
22Write these down first; a reviewer who senses they were chosen after seeing results will distrust the whole review.
23
241. **Question in PICO/PICOS form.** Population (learners/settings), Intervention/exposure or construct, Comparison, Outcomes, Study designs. This *is* the scope.
252. **Eligibility criteria.** Inclusion and exclusion on population, constructs, designs, outcomes, language, publication type (peer-reviewed? grey literature? dissertations?), and date range — each with a reason.
263. **Information sources & search strategy.** Databases (ERIC, PsycINFO, Education Source, Web of Science, Scopus, ProQuest Dissertations), grey-literature sources, and at least one full search string to be reported verbatim.
274. **Screening procedure.** Two independent screeners at title/abstract and full-text; how disagreements are resolved; the reliability statistic you will report.
285. **Data extraction & coding scheme.** What you will code from each study (design, sample, measures, effect sizes, moderators, risk-of-bias) and the codebook.
296. **Synthesis plan.** Meta-analytic model (fixed/random/multilevel/robust variance), effect-size metric, planned moderator and sensitivity analyses, and publication-bias diagnostics — *or* the explicit qualitative/narrative synthesis method for a non-poolable literature.
307. **Risk-of-bias / quality appraisal tool** chosen in advance (e.g. a design-appropriate instrument).
31
32## Preregistration: register the plan, then deviate transparently
33
34- **Systematic reviews / meta-analyses:** register the protocol on **PROSPERO** (health-adjacent) or **OSF** before screening; cite the registration in the manuscript. Use **PRISMA-P** as the protocol-reporting checklist.
35- A registration is a commitment, not a cage — you *may* change the plan, but you must **flag and justify every deviation** in the paper (planned vs. exploratory analyses kept distinct).
36- Purely conceptual/critical syntheses are not "registered" the same way, but they still need an explicit, written scope and search logic that the paper reports.
37
38## Why a-priori discipline pays off at RER specifically
39
40A protocol is not bureaucratic overhead — it is what lets a *submitted* review be trusted the way an *invited* one is. Three concrete payoffs:
41
42- **It pre-empts the "fishing expedition" objection.** A reviewer who sees that inclusion rules, the model, and the moderators were fixed before screening cannot claim they were chosen to produce a clean result.
43- **It makes the PRISMA flow honest.** Counts flow from rules set in advance, so the diagram reports what the search found rather than what survived after-the-fact pruning.
44- **It separates confirmatory from exploratory.** Anything not in the protocol is labeled exploratory in the paper — which protects the headline result and still lets you report interesting surprises.
45
46A common failure is treating the protocol as a draft of the paper. It is not prose; it is a set of **decision rules** a stranger could execute and reach your corpus. Write it as rules, register it, then let the search run against it.
47
48## Checklist
49
50- [ ] Question in PICO/PICOS; scope bounded the same way the search will be
51- [ ] Eligibility criteria written with a reason for each inclusion/exclusion rule
52- [ ] Databases + grey-literature sources listed; at least one full search string drafted
53- [ ] Dual independent screening defined; reliability statistic chosen
54- [ ] Coding scheme / codebook drafted before extraction
55- [ ] Synthesis plan fixed (meta-analytic model + moderators + bias diagnostics, or narrative method)
56- [ ] Risk-of-bias tool chosen a-priori
57- [ ] Registered on PROSPERO/OSF (or scope/search logic written for a conceptual synthesis)
58
59## Anti-patterns
60
61- Searching first, then writing inclusion rules to fit the studies you liked
62- "Random-effects, REML" chosen after seeing heterogeneity rather than in advance
63- No codebook — coders improvise, and reliability cannot be defended later
64- Treating registration as a formality, then silently changing the question or outcomes
65- Promising an exhaustive search but planning only one database
66
67## Output format
68
69```
70【Question (PICOS)】<population / intervention-construct / comparison / outcomes / designs>
71【Eligibility】<key inclusion + exclusion rules, each with a reason>
72【Sources + search】<databases + grey lit; one full search string drafted? Y/N>
73【Screening】dual independent? Y/N; reliability stat = <kappa/agreement>
74【Coding scheme】codebook drafted before extraction? Y/N
75【Synthesis plan】<meta-analytic model + moderators + bias diagnostics | narrative method>
76【Risk-of-bias tool】<instrument, chosen a-priori>
77【Registration】PROSPERO/OSF id or "conceptual synthesis — scope written"
78【Next step】→ revedres-literature-synthesis (run the search to a PRISMA flow)
79```
80
81---
82
83**Source:** [`brycewang-stanford/Awesome-Journal-Skills`](https://github.com/brycewang-stanford/Awesome-Journal-Skills) → `Review-of-Educational-Research-Skills/skills/revedres-proposal-and-commissioning/SKILL.md`
thedixitjain