# Lifesight Experiment Design

> Use when the user wants to prove or validate causality with an experiment — "is X actually incremental", "prove this channel works", "design a geo-lift test", "set up a holdout", "run an incrementality test", "what should I test next", "read my experiment results", "did the test work", "validate the optimization before we commit". Covers both designing tests and interpreting results. Routed to from the `lifesight` router after the workspace is calibrated.

- Skill: `lifesight/lifesight-experiment-design` (Agent Skill)
- Install (CLI): `npx skillmds@latest add lifesight/lifesight-experiment-design`
- Raw SKILL.md: https://api.skillmd.com/api/skills/lifesight/lifesight-experiment-design/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: lifesight (https://skillmd.com/u/lifesight)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/lifesight/lifesight-experiment-design

---


# Lifesight Experiment Design

Incrementality is the ground truth of the whole platform — the difference between
"this channel got credit" and "this channel *caused* growth." This spoke does two
jobs: **design** a test that will answer a causal question, and **read** the results
of one that ran. It's also where you recommend *what's worth testing* — usually the
biggest, most expensive assumption the user is about to act on.

**Prerequisites (router handles):** workspace calibrated, profile loaded. Operate
under `lifesight-core`; present under `lifesight-rendering`. Load both.

## Designing a test

1. **Pin the causal question.** What decision hangs on it? "Is Linear TV worth its
   spend?" "Will scaling TikTok actually add revenue or just shift it?" A test with no
   decision attached is wasted budget.
2. **Pick the design** (via `ask_mia`'s experiment workflow):
   - **Geo holdout** — withhold a channel in matched control markets to measure the
     lift it's currently driving. The default for "is this incremental?"
   - **Geo scale-up** — increase spend in treatment markets to probe further up the
     response curve. For "should I spend more here?"
   - **Time / segment** designs where geo isn't feasible.
3. **Sanity-check feasibility before launching:** is there enough spend/volume and
   enough matched markets to detect a realistic effect? Name the **minimum detectable
   lift** and the test duration up front — an underpowered test wastes weeks and
   answers nothing.

## Reading results

Pull results via `ask_mia`, then interpret in this order — **significance gates
everything**:

1. **Significance first.** Below ~90%, the result is inconclusive — do not act on the
   lift, no matter how big it looks. Say so plainly.
2. **Then lift + direction.** Note the holdout nuance explicitly: in a *holdout*, a
   **negative** lift means the marketing was **effective** (withholding it dropped the
   metric). Don't misread that as "the channel failed."
3. **Then efficiency.** Incremental ROAS / CPA from the experiment is the causal ground
   truth — it outranks platform-reported numbers and even the MMM estimate.
4. **Power context.** A null result on an *underpowered* test is "we couldn't tell,"
   not "no effect." Compare the observed lift to the minimum detectable lift before
   concluding anything.

## Judgment checks (mandatory)

- **No significance, no conclusion.** The most common error is acting on an
  insignificant lift. Hold the line even under "but the number's big" pressure.
- **Holdout sign convention** — negative lift = effective. State it so it can't be misread.
- **Experiment > model > platform.** When they disagree, the clean experiment wins;
  use it to recalibrate, not to rationalize.
- **Tests take time and assume no contamination.** Don't imply instant answers; flag
  spillover risk (media bleeding into control markets, un-geo-targetable national buys).

## Output shape

- **Design:** the test plan — markets/split, duration, the spend change, what it will
  detect (minimum detectable lift), and the decision it will settle.
- **Read:** significance → lift (with sign explained) → incremental ROAS/CPA → verdict
  → recommended action (scale, cut, recalibrate the model, or re-test with more power).

## Next steps to offer

"Recalibrate the budget with this result" (→ budget-optimization) · "Deep-dive the
channel we tested" (→ channel-deep-dive) · "Explain how geo-lift works" (→
measurement-coach) · "Design the follow-up test".

## Red flags — STOP

- Acting on a lift below ~90% significance → inconclusive, say so
- Reading a holdout's negative lift as failure → it means effective
- Calling an underpowered null "no effect" → it's "couldn't detect"
- Implying an experiment gives an instant answer → name the duration
- Letting platform/MMM numbers override a clean experiment