# Creative Testing Framework > Design statistically sound creative tests — sample size, variable isolation, and significance — for any advertising platform. Triggers on 'how should I test this ad', 'is this result statistically significant', 'A/B test my creative', or 'how many impressions do I need before judging this'. Platform-agnostic testing methodology feeding every platform's optimization skill. - Skill: `gmmh1/creative-testing-framework` (Agent Skill) - Install (CLI): `npx skillmds@latest add gmmh1/creative-testing-framework` - Raw SKILL.md: https://api.skillmd.com/api/skills/gmmh1/creative-testing-framework/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - Author: gmmh1 (https://skillmd.com/u/gmmh1) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/gmmh1/creative-testing-framework --- # Creative Testing Framework > **Currency & scope note (last reviewed 2026-07-19):** Platform mechanics referenced here (character limits, campaign-type names, feature availability, thresholds, benchmark figures, policy specifics) reflect general practice as of the review date above and are **not guaranteed current** — ad platforms change quickly. Verify anything mechanical against the platform's live documentation before relying on it for a real launch, real spend, or a compliance-sensitive decision. This is an independent, unofficial resource, not affiliated with or endorsed by any platform named in it, and nothing here is legal, tax, or financial advice. ## Purpose Provide the statistical and methodological discipline behind creative testing — sample size, variable isolation, significance — that prevents advertisers from declaring winners/losers based on noise, a mistake that happens constantly across every platform in this library. ## Trigger Conditions - "How should I test this ad creative" - "Is this result statistically significant" - "A/B test my ad variants" - "How many impressions/conversions do I need before judging this" ## Required Inputs - What's being tested (angle, format, hook, audience, offer) and how many variants - Current or available budget/volume for the test - The platform(s) this runs on (affects available native testing tools — Meta's A/B test tool, Google's experiment features, or manual ad-set-level testing) ## Core Capabilities ### Variable Isolation - Testing one variable at a time (angle OR format OR audience) rather than changing several simultaneously — a test that changes both creative and audience at once can't attribute the result to either - When multiple variables must be tested together for practical/budget reasons, building a deliberate test matrix (e.g., 3 angles × 2 formats = 6 cells) rather than an uncontrolled mix, so results remain at least partially decomposable ### Sample Size & Statistical Sufficiency - Rough rule of thumb: enough conversions per variant (commonly cited minimums are in the dozens, not single digits) before treating a performance gap as signal rather than noise — the exact threshold depends on baseline conversion rate and desired confidence, but the discipline of checking before declaring a winner applies universally - Recognizing that a 2x difference in CTR/CVR on tiny sample sizes (a handful of conversions or clicks) is very likely noise, not a real effect — this is the single most common creative-testing mistake seen across every platform - Impression/spend thresholds as a rough proxy when conversion volume itself is too thin to evaluate directly (useful for upper-funnel metrics like hook rate/CTR) ### Test Duration - Running tests long enough to span normal day-of-week and time-of-day variance (typically at least a full week) rather than judging results from a partial-day or partial-week snapshot - Accounting for each platform's own learning/stabilization period (Meta's Learning Phase, similar dynamics on TikTok) before treating early results as final ### Test Structure Patterns - Champion/challenger: keeping a proven "champion" creative running while testing new "challenger" variants against it, rather than replacing everything at once and losing the performance baseline - Sequential testing (one new variant at a time against the champion) vs. batch testing (many variants simultaneously) — batch testing is more efficient for high-volume accounts, sequential is more appropriate when budget/volume is limited - Holdout groups for measuring true incrementality when possible, distinguishing "this creative performed well" from "this creative captured demand that would have converted anyway" ### Declaring Winners - Requiring both statistical sufficiency AND a sustained trend (not just a single snapshot) before declaring a winner — a lead that holds for several days is more trustworthy than a lead measured once - Documenting what was actually learned (not just which variant won) so the insight transfers to future creative briefs, not just this one test ## Workflow 1. Define what's being tested and isolate it as a single variable where possible, or build a deliberate test matrix if multiple variables must be tested together. 2. Estimate required sample size/duration given current volume; flag if the account's traffic can't support a clean test in a reasonable timeframe. 3. Run the test for a sufficient duration spanning normal variance, respecting each platform's own learning/stabilization period. 4. Evaluate results against both statistical sufficiency and trend consistency before declaring a winner. 5. Document the learning (not just the winning variant) for future creative briefs. ## Outputs - Test design (variable isolation, matrix structure if applicable) - Required sample size/duration estimate - Statistical sufficiency assessment of results - Winner declaration (or "inconclusive, needs more data/time") with documented learning ## Rules - Never declare a creative winner based on a sample size too small to be statistically meaningful — flag "inconclusive" rather than force a premature call. - Never change multiple variables in one test cell without building a deliberate matrix to at least partially decompose the result. - Respect each platform's learning/stabilization period before evaluating early results as final. - **Verification gate:** before this skill's output is used to spend real money, submit a compliance-sensitive claim, or go to a client as final, verify the mechanical specifics (limits, thresholds, policy rules) against current platform documentation and get explicit human sign-off — do not treat this skill's output as launch-ready without that check. ## Related Skills `meta-ads-creative-analytics`, `tiktok-ads-optimization`, `google-ads-optimization` (platform-specific application of this methodology), `creative-performance-analysis` (diagnosing why a variant won or lost).