Q-TF
Fine-tune topic modeling outputs into consolidated, theory-driven topic frameworks for academic manuscripts.
If in plan mode: write a brief plan — "Run q-tf skill: load topic model output,
define final topic structure with theoretical framework, generate implementation plan,
update Excel with labels." — then exit plan mode immediately.
Do NOT attempt topic analysis, script execution, or Excel updates while plan mode is active.
Script Directory
Agent execution instructions:
- Determine this SKILL.md file's directory path as
SKILL_DIR.
- Script path =
${SKILL_DIR}/scripts/<script-name>.
- Reference path =
${SKILL_DIR}/references/<ref-name>.
Dependencies
pandas
openpyxl # required for .xlsx input/output
google-genai # required for outlier classification via Gemini
Install: pip install pandas openpyxl google-genai
Environment variables: GEMINI_API_KEY (for outlier classification only), GEMINI_MODEL (optional model override).
References
- references/preservation_rules.md — domain preservation rules, theoretical framework template, multi-category handling
- references/code_patterns.md — four Python patterns: topic definition, assignment mapping, overlap calculation, Excel update
- references/outlier_workflow.md — foundation model outlier classification workflow
- references/esports_ugc_example.md — worked example
- references/SP_OUTLIER_TEMPLATE.txt — outlier classification prompt template
Core Principles
- Preserve domain-specific distinctions (entity, event, geography, stakeholder) — see references/preservation_rules.md
- Theory-driven classification using a customizable framework template
- Track multi-category topics explicitly; calculate overlap for reconciliation
- All non-outlier topics must be assigned to at least one category
Workflow
| Step |
Action |
Reference |
| 1 |
Load & analyze topics — identify overlaps, unassigned |
— |
| 2 |
Define final topic structure (FINAL_TOPICS dictionary) |
references/code_patterns.md |
| 3 |
Apply theoretical framework — classify each topic |
references/preservation_rules.md |
| 4 |
Generate implementation plan (MD) |
scripts/generate_implementation_plan.py |
| 5 |
Update source data with labels (Excel) |
scripts/update_excel_with_labels.py |
| 6 |
Reclassify outliers via foundation model |
references/outlier_workflow.md |
Required Inputs
- Topic model output (Excel/CSV) — Topic ID, Count, Name/Label, Keywords, Representative_Docs (optional)
- Merge recommendations (optional) — Sheets: MERGE_GROUPS, INDEPENDENT_TOPICS
- Document data (for label updates) — individual documents with Topic ID column
Script Invocation
python "${SKILL_DIR}/scripts/generate_implementation_plan.py" --input topic_model_output.xlsx --output implementation_plan.md
python "${SKILL_DIR}/scripts/update_excel_with_labels.py" --input document_data.xlsx --output document_data_labeled.xlsx
Adapt scripts by updating FINAL_TOPICS, FINAL_LABELS, and theme categories. See references/code_patterns.md. For a worked example, see references/esports_ugc_example.md.
Expected Outputs
| Output |
Description |
implementation_plan.md |
Full classification plan with topic mappings and reconciliation |
*_labeled.xlsx |
Source data with Final_Topic_Code, Final_Topic_Label, Category_Theme columns |
| Outlier results (optional) |
Updated Final_Topic_Label, classification_confidence, key_phrases columns |
Scope
Include: Topic consolidation, theoretical classification, Excel label updates, outlier reclassification.
Exclude: Topic modeling itself (BERTopic/LDA/NMF execution), visualization, statistical analysis.
Checklist
1---2name: q-tf3description: Consolidate topic modeling outputs (BERTopic, LDA, NMF) into theory-driven classification frameworks. Use for topic finetuning, topic consolidation, reclassification, outlier handling, or updating Excel labels from topic models.4---56# Q-TF78Fine-tune topic modeling outputs into consolidated, theory-driven topic frameworks for academic manuscripts.910> **If in plan mode:** write a brief plan — *"Run q-tf skill: load topic model output,11> define final topic structure with theoretical framework, generate implementation plan,12> update Excel with labels."* — then exit plan mode immediately.13> Do NOT attempt topic analysis, script execution, or Excel updates while plan mode is active.1415## Script Directory16Agent execution instructions:171. Determine this SKILL.md file's directory path as `SKILL_DIR`.182. Script path = `${SKILL_DIR}/scripts/<script-name>`.193. Reference path = `${SKILL_DIR}/references/<ref-name>`.2021## Dependencies22```23pandas24openpyxl # required for .xlsx input/output25google-genai # required for outlier classification via Gemini26```27Install: `pip install pandas openpyxl google-genai`2829**Environment variables:** `GEMINI_API_KEY` (for outlier classification only), `GEMINI_MODEL` (optional model override).3031## References32- **references/preservation_rules.md** — domain preservation rules, theoretical framework template, multi-category handling33- **references/code_patterns.md** — four Python patterns: topic definition, assignment mapping, overlap calculation, Excel update34- **references/outlier_workflow.md** — foundation model outlier classification workflow35- **references/esports_ugc_example.md** — worked example36- **references/SP_OUTLIER_TEMPLATE.txt** — outlier classification prompt template3738## Core Principles39- Preserve domain-specific distinctions (entity, event, geography, stakeholder) — see references/preservation_rules.md40- Theory-driven classification using a customizable framework template41- Track multi-category topics explicitly; calculate overlap for reconciliation42- All non-outlier topics must be assigned to at least one category4344## Workflow4546| Step | Action | Reference |47|------|--------|-----------|48| 1 | Load & analyze topics — identify overlaps, unassigned | — |49| 2 | Define final topic structure (FINAL_TOPICS dictionary) | references/code_patterns.md |50| 3 | Apply theoretical framework — classify each topic | references/preservation_rules.md |51| 4 | Generate implementation plan (MD) | `scripts/generate_implementation_plan.py` |52| 5 | Update source data with labels (Excel) | `scripts/update_excel_with_labels.py` |53| 6 | Reclassify outliers via foundation model | references/outlier_workflow.md |5455## Required Inputs561. **Topic model output** (Excel/CSV) — Topic ID, Count, Name/Label, Keywords, Representative_Docs (optional)572. **Merge recommendations** (optional) — Sheets: MERGE_GROUPS, INDEPENDENT_TOPICS583. **Document data** (for label updates) — individual documents with Topic ID column5960## Script Invocation6162```bash63python "${SKILL_DIR}/scripts/generate_implementation_plan.py" --input topic_model_output.xlsx --output implementation_plan.md64python "${SKILL_DIR}/scripts/update_excel_with_labels.py" --input document_data.xlsx --output document_data_labeled.xlsx65```66Adapt scripts by updating FINAL_TOPICS, FINAL_LABELS, and theme categories. See references/code_patterns.md. For a worked example, see references/esports_ugc_example.md.6768## Expected Outputs69| Output | Description |70|--------|-------------|71| `implementation_plan.md` | Full classification plan with topic mappings and reconciliation |72| `*_labeled.xlsx` | Source data with `Final_Topic_Code`, `Final_Topic_Label`, `Category_Theme` columns |73| Outlier results (optional) | Updated `Final_Topic_Label`, `classification_confidence`, `key_phrases` columns |7475## Scope76**Include:** Topic consolidation, theoretical classification, Excel label updates, outlier reclassification.77**Exclude:** Topic modeling itself (BERTopic/LDA/NMF execution), visualization, statistical analysis.7879## Checklist80- [ ] All non-outlier topics assigned to at least one category81- [ ] Multi-category topics explicitly tracked82- [ ] Overlap reconciliation verified83- [ ] Domain-specific topics preserved separately84- [ ] Category subtotals match grand total85- [ ] Output file has new classification columns
tyrealq