Scaffolding Tests

Scaffolding Tests Skill

Scaffolding tests (also called "characterization tests" or "golden master tests") capture the current behavior of code before refactoring. They verify that behavior is preserved without requiring understanding of why the code works.

Purpose: Enable safe refactoring by detecting any behavior change, intentional or not.

When to use: Before any refactoring that modifies code paths, especially when existing test coverage is insufficient.

Core Concept

┌─────────────────────────────────────────────────────────────────┐
│                    SCAFFOLDING TEST FLOW                         │
│                                                                  │
│   1. CAPTURE                                                     │
│      Input A → Current Code → Output A₁  ← Record this          │
│      Input B → Current Code → Output B₁  ← Record this          │
│      Input C → Current Code → Output C₁  ← Record this          │
│                                                                  │
│   2. REFACTOR                                                    │
│      Modify code structure (not behavior)                        │
│                                                                  │
│   3. VERIFY                                                      │
│      Input A → Refactored Code → Output A₂                       │
│      Assert: A₂ == A₁  ← Behavior preserved!                    │
│                                                                  │
│   4. CLEANUP                                                     │
│      Replace scaffolding with proper unit tests                  │
│      Delete scaffolding tests                                    │
└─────────────────────────────────────────────────────────────────┘

Workflow

Step 1: Understand What to Scaffold

You should receive a refactoring spec that identifies:

• Which components need scaffolding
• What input categories to test
• Expected risk level

If no spec exists, identify components using:

Step 2: Generate Representative Inputs

For each component, create inputs that cover:

Generate inputs programmatically when possible:

// Input generator for user creation
func generateUserInputs() []UserInput {
    return []UserInput{
        // Happy path
        {Name: "John Smith", Email: "john@example.com", Age: 30},

        // Boundary: empty/nil
        {Name: "", Email: "valid@example.com", Age: 0},
        {Name: "Valid", Email: "", Age: 25},

        // Boundary: max length
        {Name: strings.Repeat("a", 255), Email: "max@example.com", Age: 120},

        // Error: invalid format
        {Name: "Valid", Email: "not-an-email", Age: 25},
        {Name: "Valid", Email: "valid@example.com", Age: -1},

        // Unicode
        {Name: "María García-López", Email: "maría@ejemplo.com", Age: 28},
        {Name: "田中太郎", Email: "tanaka@example.jp", Age: 35},

        // Realistic
        {Name: "Robert O'Brien Jr.", Email: "rob.obrien+work@company.co.uk", Age: 42},
    }
}

Step 3: Capture Current Outputs

Run inputs through the current code and record outputs exactly:

func TestScaffold_CreateUser(t *testing.T) {
    // This test CAPTURES behavior, it does not ASSERT correctness

    svc := setupRealService(t)  // Use real dependencies if safe

    inputs := generateUserInputs()

    for _, input := range inputs {
        t.Run(describeInput(input), func(t *testing.T) {
            result, err := svc.CreateUser(context.Background(), input)

            // Record the golden output
            golden := GoldenOutput{
                Input:      input,
                Result:     result,
                Error:      errorToString(err),
                ErrorType:  errorType(err),
            }

            // Compare against stored golden (or store if first run)
            assertGolden(t, "create_user", input.Name, golden)
        })
    }
}

// Helper to serialize errors consistently
func errorToString(err error) string {
    if err == nil {
        return ""
    }
    return err.Error()
}

func errorType(err error) string {
    if err == nil {
        return ""
    }
    // Capture error type for errors.Is() compatibility
    return fmt.Sprintf("%T", err)
}

Step 4: Store Golden Outputs

Store captured outputs in a structured format:

testdata/
└── golden/
    └── create_user/
        ├── happy_path_john_smith.golden.json
        ├── boundary_empty_name.golden.json
        ├── boundary_max_length.golden.json
        ├── error_invalid_email.golden.json
        └── unicode_japanese.golden.json

Golden file format:

{
  "input": {
    "name": "John Smith",
    "email": "john@example.com",
    "age": 30
  },
  "output": {
    "result": {
      "id": "GENERATED_ID",
      "name": "John Smith",
      "email": "john@example.com",
      "created_at": "TIMESTAMP"
    },
    "error": "",
    "error_type": ""
  },
  "captured_at": "2026-01-31T10:00:00Z",
  "code_version": "abc123"
}

Handle non-deterministic values:

// Normalize outputs before comparison
func normalizeOutput(output *CreateUserResult) {
    // Replace generated IDs with placeholder
    if output.ID != "" {
        output.ID = "GENERATED_ID"
    }

    // Replace timestamps with placeholder
    if !output.CreatedAt.IsZero() {
        output.CreatedAt = time.Time{}  // or a fixed time
    }
}

Step 5: Create Scaffolding Test File

// user_scaffold_test.go
//
// SCAFFOLDING TESTS - DELETE AFTER REFACTORING
//
// These tests capture current behavior for the user service refactoring.
// They should be replaced with proper unit tests after refactoring is complete.
//
// Refactoring ticket: RF-2026-001
// Created: 2026-01-31
// Expected removal: After RF-2026-001 Phase 4 (cleanup)

package handlers

import (
    "context"
    "encoding/json"
    "os"
    "path/filepath"
    "testing"
)

// TestScaffold_ prefix indicates these are temporary scaffolding tests
func TestScaffold_CreateUser(t *testing.T) {
    if os.Getenv("SKIP_SCAFFOLD") != "" {
        t.Skip("Scaffolding tests skipped")
    }

    svc := setupService(t)

    tests := []struct {
        name  string
        input UserInput
    }{
        {"happy_path", UserInput{Name: "John", Email: "john@example.com"}},
        {"empty_name", UserInput{Name: "", Email: "valid@example.com"}},
        {"invalid_email", UserInput{Name: "John", Email: "not-an-email"}},
        // ... more cases
    }

    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            result, err := svc.CreateUser(context.Background(), tt.input)

            actual := normalize(result, err)
            expected := loadGolden(t, "create_user", tt.name)

            if !equalGolden(actual, expected) {
                // On first run or with -update flag, save the golden
                if os.Getenv("UPDATE_GOLDEN") != "" {
                    saveGolden(t, "create_user", tt.name, actual)
                    return
                }

                t.Errorf("Behavior changed!\nExpected: %s\nActual: %s",
                    formatGolden(expected), formatGolden(actual))
            }
        })
    }
}

func TestScaffold_GetUser(t *testing.T) {
    // Similar structure...
}

// Golden file helpers

func loadGolden(t *testing.T, component, testCase string) Golden {
    t.Helper()
    path := filepath.Join("testdata", "golden", component, testCase+".golden.json")

    data, err := os.ReadFile(path)
    if err != nil {
        if os.IsNotExist(err) {
            return Golden{}  // First run, no golden yet
        }
        t.Fatalf("Failed to load golden: %v", err)
    }

    var golden Golden
    if err := json.Unmarshal(data, &golden); err != nil {
        t.Fatalf("Failed to parse golden: %v", err)
    }
    return golden
}

func saveGolden(t *testing.T, component, testCase string, golden Golden) {
    t.Helper()
    dir := filepath.Join("testdata", "golden", component)
    if err := os.MkdirAll(dir, 0755); err != nil {
        t.Fatalf("Failed to create golden dir: %v", err)
    }

    path := filepath.Join(dir, testCase+".golden.json")
    data, err := json.MarshalIndent(golden, "", "  ")
    if err != nil {
        t.Fatalf("Failed to marshal golden: %v", err)
    }

    if err := os.WriteFile(path, data, 0644); err != nil {
        t.Fatalf("Failed to write golden: %v", err)
    }

    t.Logf("Updated golden: %s", path)
}

Step 6: Verify Scaffolding Before Refactoring

Before starting refactoring:

# Generate/update golden files
UPDATE_GOLDEN=1 go test -v -run TestScaffold_ ./...

# Verify scaffolding captures current behavior
go test -v -run TestScaffold_ ./...

# All scaffolding tests should pass

Step 7: Use During Refactoring

After each refactoring change:

# Quick check - scaffolding tests only
go test -v -run TestScaffold_ ./...

# Any failure means behavior changed
# Either: fix the refactoring, or update the golden (if change is intentional)

Step 8: Cleanup After Refactoring

Once refactoring is complete:

1. Write proper unit tests for the refactored code
2. Verify proper tests cover the same behaviors
3. Delete scaffolding tests and golden files
4. Remove testdata/golden/ directory

# Remove scaffolding artifacts
rm -rf testdata/golden/
rm *_scaffold_test.go

HTTP Handler Scaffolding

For HTTP handlers, capture the full response:

func TestScaffold_UserHandler_Create(t *testing.T) {
    handler := setupHandler(t)

    tests := []struct {
        name   string
        method string
        path   string
        body   string
    }{
        {"valid_user", "POST", "/users", `{"name":"John","email":"john@example.com"}`},
        {"invalid_json", "POST", "/users", `{invalid`},
        {"missing_email", "POST", "/users", `{"name":"John"}`},
    }

    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            req := httptest.NewRequest(tt.method, tt.path, strings.NewReader(tt.body))
            req.Header.Set("Content-Type", "application/json")

            rec := httptest.NewRecorder()
            handler.ServeHTTP(rec, req)

            actual := HTTPGolden{
                StatusCode: rec.Code,
                Headers:    normalizeHeaders(rec.Header()),
                Body:       normalizeBody(rec.Body.String()),
            }

            expected := loadHTTPGolden(t, "user_handler", tt.name)

            if !equalHTTPGolden(actual, expected) {
                if os.Getenv("UPDATE_GOLDEN") != "" {
                    saveHTTPGolden(t, "user_handler", tt.name, actual)
                    return
                }
                t.Errorf("Response changed!\nExpected: %+v\nActual: %+v", expected, actual)
            }
        })
    }
}

Database-Dependent Scaffolding

For code that modifies database state:

func TestScaffold_CreateUser_DBState(t *testing.T) {
    db := setupTestDB(t)  // Fresh DB for each test
    svc := NewService(db)

    input := UserInput{Name: "John", Email: "john@example.com"}

    // Capture before state
    beforeCount := countUsers(t, db)

    // Execute
    result, err := svc.CreateUser(context.Background(), input)

    // Capture after state
    afterCount := countUsers(t, db)
    createdUser := getUserByEmail(t, db, "john@example.com")

    actual := DBGolden{
        Result:       normalize(result),
        Error:        errorToString(err),
        UsersBefore:  beforeCount,
        UsersAfter:   afterCount,
        CreatedUser:  normalizeUser(createdUser),
    }

    // Compare with golden...
}

Naming Conventions

Anti-Patterns

Checklist

Before starting scaffolding:

- [ ] Components to scaffold identified (from refactoring spec or self-analysis)
- [ ] Input categories defined for each component
- [ ] Golden file storage structure decided

During scaffolding:

- [ ] Inputs cover: happy path, boundaries, errors, unicode, realistic data
- [ ] Non-deterministic values normalized (IDs, timestamps)
- [ ] Golden files generated with UPDATE_GOLDEN=1
- [ ] All scaffolding tests pass before refactoring starts

After refactoring:

- [ ] All scaffolding tests pass (behavior preserved)
- [ ] Proper unit tests written for refactored code
- [ ] Scaffolding tests deleted
- [ ] Golden files deleted

Integration with Testing Skill

Scaffolding tests are temporary and have different rules than regular tests:

After refactoring, replace scaffolding with regular tests following the Testing Skill.