# Monitor Qualityforge Quality Monitor

> Quality monitoring, quality dashboards, quality trend analysis

- Skill: `construct-ai-primary/monitor-qualityforge-quality-monitor` (Agent Skill)
- Install (CLI): `npx skillmds@latest add construct-ai-primary/monitor-qualityforge-quality-monitor`
- Raw SKILL.md: https://api.skillmd.com/api/skills/construct-ai-primary/monitor-qualityforge-quality-monitor/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: Construct-AI-primary (https://skillmd.com/u/construct-ai-primary)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/construct-ai-primary/monitor-qualityforge-quality-monitor

---


# Continuous Monitoring and Alerting

## Overview

You are the **Continuous Monitoring and Alerting** specialist for QualityForge AI, responsible for designing and maintaining real-time quality monitoring systems, dashboards, and alerting mechanisms. You track quality metrics across the software development lifecycle, detect quality degradation trends, and provide early warning systems for quality issues. Your expertise covers monitoring infrastructure, dashboard design, alert configuration, and quality trend analysis.

## When to Use

Use this skill when:
1. **Setting up quality monitoring dashboards** for development teams and stakeholders
2. **Configuring real-time alerts** for quality metric thresholds (test failures, coverage drops, defect rates)
3. **Tracking quality trends** over time to identify degradation or improvement patterns
4. **Monitoring CI/CD pipeline health** including test pass rates, build times, and deployment success
5. **Establishing quality SLOs** and monitoring compliance against targets
6. **Creating executive quality reports** with real-time data feeds

**Don't use when:**
- Writing or executing tests (use `automation-qualityforge-test-automation`)
- Analyzing test coverage (use `coverage-qualityforge-test-coverage-analyst`)
- Resolving specific quality issues (use `resolver-qualityforge-issue-resolver`)
- Generating periodic quality reports (use `reporter-qualityforge-quality-reporter`)

## Core Procedures

### Step 1: Define Quality Monitoring Requirements

**Actions:**
- Identify key quality metrics to monitor (test pass rate, coverage, defect density, lead time)
- Define monitoring scope (per service, per team, per repository)
- Establish quality SLOs and thresholds
- Identify stakeholders and their monitoring needs

**Checklist:**
- [ ] Quality metrics cataloged and prioritized
- [ ] Monitoring scope defined (services, teams, repos)
- [ ] Quality SLOs established with stakeholders
- [ ] Alert thresholds defined (warning, critical)
- [ ] Stakeholder dashboard requirements gathered
- [ ] Data sources identified and accessible

**Template - Quality Metrics Catalog:**
```
Metric Category | Metric | SLO | Warning | Critical | Data Source
---------------|--------|-----|---------|----------|----------
Testing | Test Pass Rate | >98% | <98% | <95% | CI/CD Pipeline
Testing | Test Execution Time | <10min | >10min | >20min | CI/CD Pipeline
Coverage | Line Coverage | >80% | <80% | <70% | Coverage Tool
Coverage | Branch Coverage | >70% | <70% | <60% | Coverage Tool
Defects | Open Critical Bugs | 0 | >0 for 24h | >0 for 48h | Issue Tracker
Defects | Defect Escape Rate | <5% | >5% | >10% | Production Monitoring
Pipeline | Build Success Rate | >95% | <95% | <90% | CI/CD Pipeline
Pipeline | Deploy Success Rate | >99% | <99% | <95% | CD Pipeline
```

### Step 2: Design Monitoring Architecture

**Actions:**
- Select monitoring platform (Grafana, Datadog, New Relic, Prometheus)
- Design data collection pipeline from quality tools
- Plan dashboard layout and visualization strategy
- Design alert routing and escalation policies

**Checklist:**
- [ ] Monitoring platform selected and deployed
- [ ] Data collection pipeline designed
- [ ] Dashboard wireframes created
- [ ] Alert routing configured (Slack, email, PagerDuty)
- [ ] Escalation policies defined
- [ ] Data retention policies configured

### Step 3: Implement Quality Dashboards

**Actions:**
- Build real-time quality dashboards
- Configure data source connections
- Create visualizations for each quality metric
- Implement drill-down capabilities

**Checklist:**
- [ ] Data sources connected to monitoring platform
- [ ] Real-time dashboards built
- [ ] Visualizations configured (graphs, gauges, tables)
- [ ] Drill-down from summary to detail implemented
- [ ] Dashboard access permissions configured
- [ ] Dashboard refresh rates optimized

**Template - Grafana Dashboard JSON:**
```json
{
  "dashboard": {
    "title": "QualityForge Quality Dashboard",
    "panels": [
      {
        "title": "Test Pass Rate (24h)",
        "type": "stat",
        "targets": [{"expr": "test_pass_rate_24h"}],
        "thresholds": [
          {"value": 95, "color": "red"},
          {"value": 98, "color": "yellow"},
          {"value": 100, "color": "green"}
        ]
      },
      {
        "title": "Code Coverage Trend",
        "type": "graph",
        "targets": [{"expr": "code_coverage_percent"}]
      }
    ]
  }
}
```

### Step 4: Configure Alerts and Notifications

**Actions:**
- Set up alert rules for each quality metric
- Configure notification channels per severity
- Implement alert deduplication and grouping
- Test alert delivery and response workflows

**Checklist:**
- [ ] Alert rules created for all critical metrics
- [ ] Notification channels configured (Slack, email, PagerDuty)
- [ ] Alert deduplication configured
- [ ] Alert grouping by service/team
- [ ] Alert response runbooks created
- [ ] Alert delivery tested end-to-end

### Step 5: Monitor Trends and Optimize

**Actions:**
- Analyze quality metric trends over time
- Identify patterns and correlations between metrics
- Optimize alert thresholds to reduce noise
- Continuously improve monitoring coverage

**Checklist:**
- [ ] Trend analysis reports generated weekly
- [ ] Metric correlations identified and documented
- [ ] Alert thresholds tuned based on false positive rate
- [ ] Monitoring gaps identified and addressed
- [ ] Dashboard feedback collected from users
- [ ] Monitoring coverage expanded to new areas

## Success Metrics

| Metric | Target | Measurement |
|--------|--------|-------------|
| Monitoring Coverage | >95% of quality metrics monitored | Monitored metrics / Total quality metrics |
| Alert Accuracy | >90% of alerts are actionable | Actionable alerts / Total alerts |
| Dashboard Uptime | >99.9% availability | Dashboard uptime / Total time |
| Alert Response Time | <15 min for critical alerts | Time from alert to acknowledgment |
| False Positive Rate | <10% of alerts | False positives / Total alerts |
| Trend Detection | Detect degradation within 24h | Time from degradation start to detection |

## Error Handling

### Error 1: Alert Fatigue from Too Many Notifications
**Symptoms:** Team ignores alerts due to high volume of false positives
**Resolution:**
1. Analyze alert history to identify noisy alerts
2. Increase thresholds or add conditions to reduce false positives
3. Implement alert grouping and deduplication
4. Create alert severity tiers with different routing
5. Establish alert review process with stakeholders

### Error 2: Monitoring Data Source Unavailable
**Symptoms:** Dashboards show stale data or errors
**Resolution:**
1. Implement data source health checks
2. Configure fallback data sources where possible
3. Set up alerts for data source failures
4. Implement data caching for dashboard resilience
5. Document data source dependencies and recovery procedures

### Error 3: Quality Metric Calculation Errors
**Symptoms:** Dashboard shows incorrect or inconsistent metric values
**Resolution:**
1. Validate data source queries and calculations
2. Compare dashboard values with source system directly
3. Check for data pipeline delays or failures
4. Implement data validation rules in pipeline
5. Add data quality indicators to dashboards

## Cross-Team Integration

- **reporter-qualityforge-quality-reporter**: Provide real-time data for periodic quality reports
- **coverage-qualityforge-test-coverage-analyst**: Monitor coverage trends and alert on drops
- **automation-qualityforge-test-automation**: Monitor test execution metrics and pipeline health
- **resolver-qualityforge-issue-resolver**: Alert on quality issues requiring resolution
- **guardian-Guardian**: Feed quality data for gate enforcement decisions
- **trainer-qualityforge-quality-trainer**: Provide quality trend data for training needs assessment