# Longitudinal Measurement

> Tracking AI product quality over time — drift, degradation, and improvement.

- Skill: `gabrielmoreira/longitudinal-measurement` (Agent Skill)
- Install (CLI): `npx skillmds@latest add gabrielmoreira/longitudinal-measurement`
- Raw SKILL.md: https://api.skillmd.com/api/skills/gabrielmoreira/longitudinal-measurement/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: gabrielmoreira (https://skillmd.com/u/gabrielmoreira)
- Updated: 2026-09-09
- Page: https://skillmd.com/skills/gabrielmoreira/longitudinal-measurement

---

# Longitudinal Measurement
AI products change over time — models get updated, usage patterns shift, and quality can drift without anyone noticing. Longitudinal measurement is how you track quality across time and catch degradation before users do.
## What Changes Over Time
- **Model updates**: New model versions may improve some capabilities and regress others
- **Prompt drift**: System prompts accumulate edits that may interact in unexpected ways
- **Usage evolution**: Users discover new use cases that weren't tested for
- **Data drift**: The real-world inputs diverge from what was tested
- **Expectation drift**: Users' expectations change as they become more experienced
## What to Measure Longitudinally
- **Quality scores**: Track rubric scores on a consistent test set over time
- **Task success rates**: Monitor whether users are completing tasks at the same rate
- **Satisfaction signals**: Track trends in explicit and implicit satisfaction
- **Error rates**: Monitor failure frequency and type distribution
- **Latency**: Response time changes can indicate degradation
- **Engagement patterns**: Changes in usage frequency, depth, and breadth
## Measurement Infrastructure
- **Golden test sets**: A fixed set of inputs evaluated regularly to detect quality changes
- **Automated evaluation**: Run golden test sets automatically on a schedule
- **Dashboards**: Visualise trends and set alerts for significant changes
- **Regression detection**: Statistical methods to distinguish real changes from noise
- **User cohort tracking**: Follow specific user groups over time
## Responding to Drift
When measurements show drift:
1. **Detect**: Automated alerts flag significant changes
2. **Diagnose**: Was it a model update, prompt change, data shift, or usage change?
3. **Assess**: Is the drift harmful, neutral, or actually an improvement?
4. **Act**: Adjust prompts, revert changes, update guardrails, or accept the new baseline
5. **Verify**: Confirm the fix worked and set the new baseline
## Design Artefacts
- Longitudinal measurement plan
- Golden test set specifications
- Quality trend dashboards
- Drift detection alert configurations
- Response protocols for detected drift