Education Survey Instrument Design
Goal
Convert constructs/variables into a usable education research questionnaire or scale with dimensions, items, response options, validity checks, and pilot-test plan.
Use After
Use this skill after one of these stages:
education-variable-identification
education-quantitative-study-design
education-mixed-methods-design
education-program-evaluation
education-psychometric-scale-development
Do not expose the skill name to users. Present this as "问卷/量表设计" or "测量工具设计".
Inputs
- Research question(s)
- Variable/construct table
- Target participants: students, teachers, parents, administrators
- Education context: grade, subject, school type, region
- Existing scales, if any
- Target language
- Intended analysis: descriptive, regression, mediation/moderation, SEM, EFA/CFA, evaluation
Workflow
- Confirm measurement purpose:
- measure attitudes/perceptions
- measure behavior/frequency
- measure ability/performance
- measure implementation quality
- screen/group participants
- validate a new scale
- Map constructs to dimensions.
- Search or suggest existing validated scales before creating new items.
- Decide item source:
- adopt existing scale
- adapt existing scale
- create new items
- mixed adopt/adapt/create
- Generate item pool for each dimension.
- Choose response format:
- 5-point or 7-point Likert
- frequency scale
- agreement scale
- semantic differential
- multiple choice
- open-ended supplement
- Add reverse-coded items only when useful and age-appropriate.
- Build questionnaire sections:
- consent/instructions
- demographics/background
- core scale items
- behavioral/context items
- optional open-ended item
- Plan expert review/content validity.
- Plan pilot test:
- comprehension check
- item distribution
- missing response rate
- reliability
- exploratory factor analysis if enough sample
- Output questionnaire draft and validation plan.
Tool Calls
Literature Search for Existing Scales
Semantic Scholar:
curl "https://api.semanticscholar.org/graph/v1/paper/search?query=student%20writing%20self-efficacy%20scale%20education&limit=10&fields=title,year,abstract,citationCount,authors"
OpenAlex:
curl "https://api.openalex.org/works?search=student%20writing%20self-efficacy%20scale%20education&per-page=10&mailto=YOUR_EMAIL"
Search patterns:
[construct] scale education students
[construct] questionnaire validation teachers
[construct] instrument reliability validity education
[construct] Chinese scale students
Questionnaire Deployment Tools
Qualtrics: https://www.qualtrics.com/
SurveyMonkey: https://www.surveymonkey.com/
Google Forms: https://forms.google.com/
问卷星: https://www.wjx.cn/
腾讯问卷: https://wj.qq.com/
Reliability and Item Analysis Tools
R:
install.packages(c("psych", "tidyverse", "GPArotation"))
library(psych)
alpha(data_frame_of_items)
describe(data_frame_of_items)
fa(data_frame_of_items, nfactors = 3, rotate = "oblimin")
jamovi:
https://www.jamovi.org/
Analyses -> Factor -> Reliability Analysis
Analyses -> Factor -> Exploratory Factor Analysis
JASP:
https://jasp-stats.org/
Factor -> Reliability Analysis
Factor -> Exploratory Factor Analysis
Output Format
1. Construct-Dimension Map
| Construct |
Dimension |
Definition |
Source Basis |
Planned Items |
2. Questionnaire Item Table
| Item ID |
Dimension |
Item Text |
Response Format |
Reverse Coded |
Source/Adaptation |
3. Expert Review Form
| Item ID |
Relevance 1-4 |
Clarity 1-4 |
Age Appropriateness 1-4 |
Revision Suggestion |
4. Pilot-Test Plan
| Check |
Method |
Decision Rule |
| Comprehension |
Cognitive interview or student feedback |
Revise unclear items |
| Missing rate |
Item missing percentage |
Review items with high missing |
| Item distribution |
Mean, SD, skewness |
Review extreme ceiling/floor effects |
| Reliability |
Cronbach's alpha / omega |
Interpret cautiously; revise weak items |
| Structure |
EFA/CFA if sample supports |
Confirm dimension structure |
Age-Appropriate Item Rules
- For primary students, use short sentences and concrete behaviors.
- Avoid double-barreled items.
- Avoid abstract constructs unless translated into observable behavior.
- Avoid excessive reverse-coded items for young learners.
- Use examples only if they do not bias answers.
Quality Rules
- Prefer validated instruments when available.
- Do not claim a new scale is valid before testing.
- Separate dimensions clearly; each item should measure one thing.
- Align every item to a research question or variable.
- Record whether items are adopted, adapted, or newly written.
- Plan permissions/citations for existing scales.
User-Facing Closure
End by asking the user to confirm practical constraints:
这版问卷已经可以进入专家审阅和小样本预测试。接下来需要确认:你的对象是几年级学生?预计能收集多少份问卷?是否需要我把它整理成问卷星/Qualtrics 可导入格式?
1---2name: education-survey-instrument-design3category: 教学辅导4description: Use when designing survey instruments, questionnaires, scales, rubrics, or measurement tools for education research after research questions, variables, or constructs are identified. Covers scale selection, construct-dimension mapping, item generation, reverse-coded items, response formats, expert validity review, pilot testing, reliability/validity planning, and questionnaire deployment tools. This is a second-layer execution skill used by quantitative, mixed-methods, program evaluation, psychometric, and action research workflows.5metadata:6 source: https://skillmd.com/skills/ilog3/education-survey-instrument-design7 short-description: Design education surveys, scales, and questionnaire instruments8---910# Education Survey Instrument Design1112## Goal1314Convert constructs/variables into a usable education research questionnaire or scale with dimensions, items, response options, validity checks, and pilot-test plan.1516## Use After1718Use this skill after one of these stages:1920- `education-variable-identification`21- `education-quantitative-study-design`22- `education-mixed-methods-design`23- `education-program-evaluation`24- `education-psychometric-scale-development`2526Do not expose the skill name to users. Present this as "问卷/量表设计" or "测量工具设计".2728## Inputs2930- Research question(s)31- Variable/construct table32- Target participants: students, teachers, parents, administrators33- Education context: grade, subject, school type, region34- Existing scales, if any35- Target language36- Intended analysis: descriptive, regression, mediation/moderation, SEM, EFA/CFA, evaluation3738## Workflow39401. Confirm measurement purpose:41 - measure attitudes/perceptions42 - measure behavior/frequency43 - measure ability/performance44 - measure implementation quality45 - screen/group participants46 - validate a new scale472. Map constructs to dimensions.483. Search or suggest existing validated scales before creating new items.494. Decide item source:50 - adopt existing scale51 - adapt existing scale52 - create new items53 - mixed adopt/adapt/create545. Generate item pool for each dimension.556. Choose response format:56 - 5-point or 7-point Likert57 - frequency scale58 - agreement scale59 - semantic differential60 - multiple choice61 - open-ended supplement627. Add reverse-coded items only when useful and age-appropriate.638. Build questionnaire sections:64 - consent/instructions65 - demographics/background66 - core scale items67 - behavioral/context items68 - optional open-ended item699. Plan expert review/content validity.7010. Plan pilot test:71 - comprehension check72 - item distribution73 - missing response rate74 - reliability75 - exploratory factor analysis if enough sample7611. Output questionnaire draft and validation plan.7778## Tool Calls7980### Literature Search for Existing Scales8182Semantic Scholar:8384```bash85curl "https://api.semanticscholar.org/graph/v1/paper/search?query=student%20writing%20self-efficacy%20scale%20education&limit=10&fields=title,year,abstract,citationCount,authors"86```8788OpenAlex:8990```bash91curl "https://api.openalex.org/works?search=student%20writing%20self-efficacy%20scale%20education&per-page=10&mailto=YOUR_EMAIL"92```9394Search patterns:9596```text97[construct] scale education students98[construct] questionnaire validation teachers99[construct] instrument reliability validity education100[construct] Chinese scale students101```102103### Questionnaire Deployment Tools104105```text106Qualtrics: https://www.qualtrics.com/107SurveyMonkey: https://www.surveymonkey.com/108Google Forms: https://forms.google.com/109问卷星: https://www.wjx.cn/110腾讯问卷: https://wj.qq.com/111```112113### Reliability and Item Analysis Tools114115R:116117```r118install.packages(c("psych", "tidyverse", "GPArotation"))119library(psych)120alpha(data_frame_of_items)121describe(data_frame_of_items)122fa(data_frame_of_items, nfactors = 3, rotate = "oblimin")123```124125jamovi:126127```text128https://www.jamovi.org/129Analyses -> Factor -> Reliability Analysis130Analyses -> Factor -> Exploratory Factor Analysis131```132133JASP:134135```text136https://jasp-stats.org/137Factor -> Reliability Analysis138Factor -> Exploratory Factor Analysis139```140141## Output Format142143### 1. Construct-Dimension Map144145| Construct | Dimension | Definition | Source Basis | Planned Items |146|---|---|---|---|---|147148### 2. Questionnaire Item Table149150| Item ID | Dimension | Item Text | Response Format | Reverse Coded | Source/Adaptation |151|---|---|---|---|---|---|152153### 3. Expert Review Form154155| Item ID | Relevance 1-4 | Clarity 1-4 | Age Appropriateness 1-4 | Revision Suggestion |156|---|---|---|---|---|157158### 4. Pilot-Test Plan159160| Check | Method | Decision Rule |161|---|---|---|162| Comprehension | Cognitive interview or student feedback | Revise unclear items |163| Missing rate | Item missing percentage | Review items with high missing |164| Item distribution | Mean, SD, skewness | Review extreme ceiling/floor effects |165| Reliability | Cronbach's alpha / omega | Interpret cautiously; revise weak items |166| Structure | EFA/CFA if sample supports | Confirm dimension structure |167168## Age-Appropriate Item Rules169170- For primary students, use short sentences and concrete behaviors.171- Avoid double-barreled items.172- Avoid abstract constructs unless translated into observable behavior.173- Avoid excessive reverse-coded items for young learners.174- Use examples only if they do not bias answers.175176## Quality Rules177178- Prefer validated instruments when available.179- Do not claim a new scale is valid before testing.180- Separate dimensions clearly; each item should measure one thing.181- Align every item to a research question or variable.182- Record whether items are adopted, adapted, or newly written.183- Plan permissions/citations for existing scales.184185## User-Facing Closure186187End by asking the user to confirm practical constraints:188189```text190这版问卷已经可以进入专家审阅和小样本预测试。接下来需要确认:你的对象是几年级学生?预计能收集多少份问卷?是否需要我把它整理成问卷星/Qualtrics 可导入格式?191```