diegosouzapw Experiment Loop for MD Home Care
Tracks content changes, measures their impact on traffic and rankings, and decides whether to keep, iterate, or revert. Runs weekly.
CRITICAL: Lag Times for YMYL Content
YMYL content (aged care, disability services) has longer lag times than SaaS content. Do not evaluate changes too early.
| Change Type | SEO Lag | AEO Lag | Evaluation Window |
|---|---|---|---|
| Service page optimization | 10-21 days | 3-7 days | 3 weeks minimum |
| Location page creation | 14-21 days | 7-14 days | 3 weeks minimum |
| Blog post publishing | 7-14 days | 3-7 days | 2 weeks minimum |
| Provider comparison addition | 7-14 days | 3-7 days | 2 weeks minimum |
| Trust signal enhancement | 10-21 days | 7-14 days | 3 weeks minimum |
| FAQ addition | 7-14 days | 3-7 days | 2 weeks minimum |
Step 1: Weekly Git Scan
Identify all content changes from the past week:
cd ~/Projects/mdhomecarebuild
# All content changes in last 7 days
git log --since="7 days ago" --name-only --pretty=format:"%h %s" -- "src/content/**/*.md" "src/content/**/*.mdx"
# Summarize by type
git log --since="7 days ago" --name-only --pretty=format:"" -- "src/content/blog/*.md" | sort -u | head -20
git log --since="7 days ago" --name-only --pretty=format:"" -- "src/content/services/*.md" | sort -u | head -20
git log --since="7 days ago" --name-only --pretty=format:"" -- "src/content/providers/*.md" | sort -u | head -20
Categorize each change:
- New page: Completely new content file
- Major edit: Structural changes (new sections, comparison tables, rewritten H1/H2)
- Minor edit: Small fixes (typos, link updates, frontmatter changes)
Step 2: Baseline Measurement
For each changed page, capture the pre-change baseline. If baseline was not captured before the change, use the previous period as proxy.
GSC Baseline
cd ~/Projects/mdhomecarebuild
# For each changed page, get keyword data
python3 src/scripts/advanced_gsc_analyzer.py --page "/services/[slug]"
python3 src/scripts/advanced_gsc_analyzer.py --page "/blog/[slug]"
Record:
- Top 10 keywords by clicks
- Average position for primary keyword
- Total impressions and clicks (last 7 days)
PostHog Baseline
# Page traffic
python3 src/scripts/posthog_analytics.py --page "/services/[slug]" --days 7
# AI referral traffic
python3 src/scripts/posthog_analytics.py --ai-referrals --days 7
Record:
- Total pageviews (last 7 days)
- AI referral visits to that page
- Traffic sources breakdown
Step 3: Post-Change Measurement
After the evaluation window has passed (see lag times table), measure again.
# GSC: same page analysis
python3 src/scripts/advanced_gsc_analyzer.py --page "/services/[slug]"
# PostHog: same page traffic
python3 src/scripts/posthog_analytics.py --page "/services/[slug]" --days 7
python3 src/scripts/posthog_analytics.py --ai-referrals --days 7
Step 4: Attribution and Decision
Compare Metrics
For each experiment, calculate:
| Metric | Before | After | Change |
|---|---|---|---|
| Organic clicks (7d) | X | Y | +/- % |
| Impressions (7d) | X | Y | +/- % |
| Avg position (primary KW) | X | Y | +/- positions |
| AI referral visits (7d) | X | Y | +/- % |
| Total pageviews (7d) | X | Y | +/- % |
Decision Framework
KEEP if:
- Organic clicks increased >10%
- OR average position improved by 2+ positions
- OR AI referral visits increased >20%
- OR impressions increased >15% (leading indicator)
- AND no negative impact on other pages (cannibalization check)
ITERATE if:
- Mixed signals (some metrics up, some flat)
- OR small positive movement (<10% clicks) that suggests potential
- OR evaluation window has not fully elapsed
- Action: Make targeted refinements and re-evaluate after another cycle
REVERT if:
- Organic clicks decreased >15%
- AND average position dropped by 3+ positions
- AND no compensating AI referral increase
- Action: Restore previous version via git, document what went wrong
WAIT if:
- Change is too recent (within lag window)
- Action: Re-evaluate next week
Step 5: Log to Playbook
Record every experiment result in PLAYBOOK.md:
## [Date] - [Experiment Name]
**Category:** [Service page optimization / Location page / Blog post / Comparison / Trust signal / FAQ]
**Page:** [URL path]
**Change:** [Brief description of what was changed]
**Hypothesis:** [What we expected to happen]
**Baseline (pre-change):**
- Organic clicks (7d): X
- Avg position (primary KW): X
- AI referrals (7d): X
**Result (post-change, measured [date]):**
- Organic clicks (7d): Y (+/- %)
- Avg position (primary KW): Y (+/- positions)
- AI referrals (7d): Y (+/- %)
**Decision:** KEEP / ITERATE / REVERT / WAIT
**Lesson:** [What we learned]
Experiment Categories
Service Page Optimizations
- Adding comparison tables
- Rewriting H1/byline
- Adding trust signal sections
- Expanding FAQ sections
- Adding AI differentiation paragraphs
Location Page Creation
- New suburb-specific service pages
- Measure: local keyword rankings, location-specific traffic
Blog Post Publishing
- New informational content
- Template/download posts
- Provider comparison posts
- Measure: organic clicks, keyword coverage expansion
Provider Comparison Additions
- New comparison tables on existing pages
- New "vs" blog posts
- Measure: comparison keyword rankings, AI referral traffic
Trust Signal Enhancements
- Adding registration numbers
- Adding testimonials
- Adding clinical governance sections
- Measure: overall page authority signals, position changes
FAQ Additions
- New FAQ sections
- Expanding existing FAQs with PAA questions
- Measure: featured snippet captures, PAA appearances
Weekly Routine
Every week:
- Run git scan (Step 1)
- For changes past their evaluation window, measure results (Step 3)
- Make keep/iterate/revert decisions (Step 4)
- Log results to PLAYBOOK.md (Step 5)
- Capture baselines for new changes (Step 2)
Usage
/experiment-loop
Runs the full weekly cycle: scan, measure, decide, log.
/experiment-loop --check "/services/sil-services"
Check status of a specific page experiment.