Sentiment Analysis Engineer
When to Use
- Define labeling schemas — document-level polarity, aspect-based (ABSA), emotion taxonomies, or multi-label targets
- Choose and implement model stacks — lexicon/rules, classical ML, fine-tuned transformers, or LLM prompt classifiers
- Design annotation programs — guidelines, adjudication, inter-annotator agreement (IAA), and gold-standard refresh
- Run evaluation and error analysis — macro-F1, calibration, confusion slices, and failure-mode catalogs
- Adapt models to domains — product reviews, social posts, support tickets, news, or finance text
- Handle edge cases — negation, sarcasm, entities, code-switching, and demographic or topical bias
- Plan production inference — batch vs streaming, latency budgets, model serving, and API contracts
- Operate monitoring and governance — label drift, score drift, human audit loops, and dashboard integration
When NOT to Use
- Writing marketing copy, brand voice, or content strategy →
content-creator, brand-voice-enforcement
- General ML platform MLOps without sentiment/NLP scope →
ml-ops-engineer
- Exploratory analytics or predictive modeling without text-sentiment focus →
data-scientist
- Social media campaigns, calendars, or channel strategy →
social-content, marketing-analyst
- AI safety classifier or harm-benchmark research only →
ml-research-engineer-safeguards
- Legal, regulatory, or compliance conclusions from sentiment scores alone →
compliance-engineer, legal-risk-assessment
- Production LLM agents, RAG, or copilot features (unless sentiment is one component) →
ai-engineer
- Literature surveys without building or evaluating sentiment systems →
ai-researcher
Related skills
| Need |
Skill |
| Classical ML, A/B tests, general model evaluation |
data-scientist |
| LLM apps, RAG, agents, prompt eval harnesses |
ai-engineer |
| Papers, benchmarks, research methodology |
ai-researcher |
| Warehouse metrics, dbt, analytics pipelines |
analytics-engineer |
| Campaign performance and channel ROI |
marketing-analyst |
| De-AI-ing prose or editorial voice |
content-humanizer |
| MLOps deploy, drift, retraining platform |
ml-ops-engineer |
Core Workflows
1. Scope and problem framing
Clarify unit of analysis (document, sentence, span, aspect), label set, languages, latency, and success metrics.
See references/sentiment_analysis_engineer_scope.md.
2. Labeling and annotation
Draft guidelines, pilot batches, measure IAA, and lock gold standards before model training.
See references/labeling_schemas_and_annotation.md.
3. Model selection and training
Compare lexicon, classical, fine-tuned encoder, and LLM-prompt baselines; document tradeoffs and compute.
See references/models_lexicon_transformers_llm.md.
4. Evaluation and error analysis
Report slice metrics, calibration, and qualitative failure buckets with reproduction examples.
See references/evaluation_metrics_and_error_analysis.md.
5. Domain, multilingual, and edge cases
Plan domain adaptation, translation vs native models, and tests for sarcasm, negation, and bias.
See references/domain_multilingual_and_edge_cases.md.
6. Production, monitoring, and governance
Specify serving paths, SLAs, drift monitors, human review queues, and analytics handoffs.
See references/production_serving_monitoring_governance.md.
Outputs
- Problem spec — labels, languages, domains, latency, and acceptance thresholds
- Annotation guide — definitions, examples, edge-case rules, adjudication process
- Model card — data, architecture, metrics, limitations, and known failure modes
- Eval report — headline metrics, slice tables, confusion analysis, calibration plots
- Serving spec — API schema, batch/streaming mode, versioning, and rollback plan
- Monitoring plan — drift metrics, audit sampling, and escalation triggers
Principles
- Treat labels as product decisions — ambiguous guidelines inflate disagreement and model variance
- Always report macro-F1 or per-class recall when classes are imbalanced; accuracy alone misleads
- Separate offline eval from online impact — production KPIs may differ from held-out test F1
- Document residual risk for sarcasm, negation, and out-of-domain text; do not overclaim coverage
- Prefer human-in-the-loop for high-stakes routing; scores support decisions, they do not replace policy
1---2name: sentiment-analysis-engineer3description: This skill should be used when the user asks to build or evaluate sentiment and opinion-mining systems—label schemas, lexicon/ML/transformer/LLM classifiers, annotation and IAA, F1/calibration, domain adaptation, multilingual/code-switching, sarcasm/negation/bias, and production inference with drift monitoring. Triggers: sentiment analysis, sentiment engineer, opinion mining, text classification, aspect-based sentiment, ABSA, fine-tune sentiment, sentiment model, polarity detection, emotion detection, review sentiment, social media sentiment, sentiment labeling, sentiment evaluation, sarcasm detection, multilingual sentiment. Not for marketing copy (content-creator), MLOps without NLP (ml-ops-engineer), general data science (data-scientist), social campaigns (social-content), safety-classifier R&D (ml-research-engineer-safeguards), or legal outcomes from scores alone.4---56# Sentiment Analysis Engineer78## When to Use910- Define **labeling schemas** — document-level polarity, aspect-based (ABSA), emotion taxonomies, or multi-label targets11- Choose and implement **model stacks** — lexicon/rules, classical ML, fine-tuned transformers, or LLM prompt classifiers12- Design **annotation programs** — guidelines, adjudication, inter-annotator agreement (IAA), and gold-standard refresh13- Run **evaluation and error analysis** — macro-F1, calibration, confusion slices, and failure-mode catalogs14- Adapt models to **domains** — product reviews, social posts, support tickets, news, or finance text15- Handle **edge cases** — negation, sarcasm, entities, code-switching, and demographic or topical bias16- Plan **production inference** — batch vs streaming, latency budgets, model serving, and API contracts17- Operate **monitoring and governance** — label drift, score drift, human audit loops, and dashboard integration1819## When NOT to Use2021- Writing marketing copy, brand voice, or content strategy → `content-creator`, `brand-voice-enforcement`22- General ML platform MLOps without sentiment/NLP scope → `ml-ops-engineer`23- Exploratory analytics or predictive modeling without text-sentiment focus → `data-scientist`24- Social media campaigns, calendars, or channel strategy → `social-content`, `marketing-analyst`25- AI safety classifier or harm-benchmark research only → `ml-research-engineer-safeguards`26- Legal, regulatory, or compliance conclusions from sentiment scores alone → `compliance-engineer`, `legal-risk-assessment`27- Production LLM agents, RAG, or copilot features (unless sentiment is one component) → `ai-engineer`28- Literature surveys without building or evaluating sentiment systems → `ai-researcher`2930## Related skills3132| Need | Skill |33|---|---|34| Classical ML, A/B tests, general model evaluation | `data-scientist` |35| LLM apps, RAG, agents, prompt eval harnesses | `ai-engineer` |36| Papers, benchmarks, research methodology | `ai-researcher` |37| Warehouse metrics, dbt, analytics pipelines | `analytics-engineer` |38| Campaign performance and channel ROI | `marketing-analyst` |39| De-AI-ing prose or editorial voice | `content-humanizer` |40| MLOps deploy, drift, retraining platform | `ml-ops-engineer` |4142## Core Workflows4344### 1. Scope and problem framing4546Clarify unit of analysis (document, sentence, span, aspect), label set, languages, latency, and success metrics.4748**See `references/sentiment_analysis_engineer_scope.md`.**4950### 2. Labeling and annotation5152Draft guidelines, pilot batches, measure IAA, and lock gold standards before model training.5354**See `references/labeling_schemas_and_annotation.md`.**5556### 3. Model selection and training5758Compare lexicon, classical, fine-tuned encoder, and LLM-prompt baselines; document tradeoffs and compute.5960**See `references/models_lexicon_transformers_llm.md`.**6162### 4. Evaluation and error analysis6364Report slice metrics, calibration, and qualitative failure buckets with reproduction examples.6566**See `references/evaluation_metrics_and_error_analysis.md`.**6768### 5. Domain, multilingual, and edge cases6970Plan domain adaptation, translation vs native models, and tests for sarcasm, negation, and bias.7172**See `references/domain_multilingual_and_edge_cases.md`.**7374### 6. Production, monitoring, and governance7576Specify serving paths, SLAs, drift monitors, human review queues, and analytics handoffs.7778**See `references/production_serving_monitoring_governance.md`.**7980## Outputs8182- **Problem spec** — labels, languages, domains, latency, and acceptance thresholds83- **Annotation guide** — definitions, examples, edge-case rules, adjudication process84- **Model card** — data, architecture, metrics, limitations, and known failure modes85- **Eval report** — headline metrics, slice tables, confusion analysis, calibration plots86- **Serving spec** — API schema, batch/streaming mode, versioning, and rollback plan87- **Monitoring plan** — drift metrics, audit sampling, and escalation triggers8889## Principles9091- Treat **labels as product decisions** — ambiguous guidelines inflate disagreement and model variance92- Always report **macro-F1 or per-class recall** when classes are imbalanced; accuracy alone misleads93- Separate **offline eval from online impact** — production KPIs may differ from held-out test F194- Document **residual risk** for sarcasm, negation, and out-of-domain text; do not overclaim coverage95- Prefer **human-in-the-loop** for high-stakes routing; scores support decisions, they do not replace policy
daemon-blockint-tech