Devlab Test Expert

AI 驱动测试 - 最佳实践专家知识库

这是企业级测试团队的共享知识中枢。

聚合最佳实践、反模式预警、故障排除手册等专家知识，供所有测试子技能调用。

触发条件

当用户提到以下关键词时触发此 Skill：

• "测试最佳实践" / "testing best practices"
• "常见陷阱" / "antipatterns"
• "验收标准怎么写" / "acceptance criteria examples"
• "性能测试怎么做" / "performance testing guide"
• "Flaky test 治理" / "flaky test tactics"

知识领域索引

📚 测试方法论

🎯 前端专项

🔧 后端/API 专项

🛡️ 运维与质量

🔄 持续集成

核心知识卡片示例

Card #1: 测试数据管理最佳实践

来源: Enterprise Test Data Research (调研自 Netflix, Spotify, Airbnb)

核心决策树

// Step 1: 评估项目复杂度
const complexity = assessProjectComplexity();

// Step 2: 选择测试数据策略
if (complexity === 'SIMPLE') {
  return {
    strategy: 'IN_MEMORY',
    tools: ['Faker.js', 'Factory Girl'],
    maintenance: 'Low',
    isolation: 'Medium'
  };
} else if (complexity === 'COMPLEX') {
  return {
    strategy: 'TEST_CONTAINERS',
    tools: ['Testcontainers', 'LocalStack'],
    maintenance: 'High',
    isolation: 'High'
  };
} else if (complexity === 'DISTRIBUTED') {
  return {
    strategy: 'DEDICATED_TEST_DB',
    tools: ['Postgres with schemas', 'MySQL with databases'],
    maintenance: 'Very High',
    isolation: 'Maximum'
  };
}

三种主流方案对比

实施建议

推荐组合（渐进式升级）:

Phase 1: 单元测试
  fixtures: faker + factory functions
  scope: business logic validation

Phase 2: 集成测试  
  fixtures: @playwright/test + testcontainers
  scope: database operations

Phase 3: E2E 测试
  fixtures: seed data via API
  scope: cross-service workflows

反模式预警

❌ 不推荐做法:

• 使用生产数据库副本（数据污染风险）
• 硬编码测试数据（维护成本爆炸）
• 依赖全局状态（Flaky Test 根源）

✅ 推荐做法:

• 每个测试独立 Fixture
• 使用 Factory Pattern 生成数据
• 通过 API 创建而非直接 UI 操作
• 测试后自动清理（before/after hooks）

Card #2: UI 选择器稳定性指南

来源: Google Testing Best Practices

选择器优先级矩阵

实战示例

// ❌ ANTI-PATTERN: Fragile CSS selector
await page.click('.card-container .item-list li:last-child button');

// ✅ RECOMMENDED: Semantic role-based selector
await page.getByRole('button', { name: 'Delete item' }).click();

// ✅ ACCEPTABLE: Test ID with clear convention
await page.getByTestId('delete-item-button').click();

ARIA Role 参考表

Card #3: Flaky Test 治理全攻略

来源: Netflix Chaos Engineering Principles

Flaky Test 分类矩阵

诊断流程图

测试失败 → 查看 Trace 文件
         ↓
      是否有 Timeout?
         ├─ YES → Check network requests
         │          ↓
         │      Response status code?
         │         ├─ 503 → Add retry/Mock
         │         └─ Timeout → Increase wait time
         │
         └─ NO → Check assertion
                   ↓
               Mismatched values?
                  ├─ YES → Review expected result
                  └─ NO → Element location issue
                           ↓
                       Update selector

自动化检测脚本

#!/bin/bash
# scripts/detect-flaky.sh

# Run tests multiple times
for i in {1..10}; do
    npx playwright test --reporter=json > report_${i}.json
done

# Analyze results
python scripts/analyze_flakiness.py reports/*.json

# Output:
# flaky_tests.txt (测试列表)
# flaky_analysis.pdf (详细报告)

修复模板

Case 1: Async Operation Timeout

// ❌ WRONG
await page.click('#submit');
await expect(page.getByText('Success')).toBeVisible(); // Might fail

// ✅ FIXED
await page.click('#submit');
await page.waitForResponse(resp => 
  resp.url().includes('/api/submit') && resp.status() === 200
);
await expect(page.getByText('Success')).toBeVisible();

Case 2: Concurrency Issue

// ❌ WRONG (Order-dependent)
test('test A creates user', async () => { ... });
test('test B deletes same user', async () => { ... });  // Depends on A!

// ✅ FIXED (Independent)
test('test A creates unique user', async () => {
  const user = generateUniqueUser();  // Unique each run
});

test('test B deletes its own user', async () => {
  const user = generateUniqueUser();  // Independent from A
});

Card #4: 前端性能测试基准

来源: Web Vitals + Lighthouse Standards

核心指标阈值

自动化测试实现

// tests/performance/lighthouse.spec.ts
import { lighthouse } from '@axe-core/playwright';

test.describe('Performance Benchmarks', () => {
  
  test('homepage should meet Lighthouse performance score', async ({ page }) => {
    await page.goto('/');
    
    const { audits } = await page.evaluate(lighthouse, ['/']);
    
    // Performance score threshold
    expect(audits['performance'].numericValue).toBeGreaterThan(90);
    
    // Individual metric checks
    expect(audits['first-contentful-paint'].numericValue).toBeLessThan(1800);
    expect(audits['largest-contentful-paint'].numericValue).toBeLessThan(2500);
    expect(audits['cumulative-layout-shift'].numericValue).toBeLessThan(0.1);
  });
  
  test('dashboard should load within 3 seconds after authentication', async ({ page }) => {
    const startTime = Date.now();
    
    // Simulate login
    await performLogin();
    
    await page.waitForLoadState('networkidle');
    
    const loadTime = Date.now() - startTime;
    expect(loadTime).toBeLessThan(3000);
  });
});

与子技能集成

devlab-web-test-e2e 引用

/**
 * Source: devlab-test-expert/reference/ui-selection-stability.md
 * See also: https://github.com/team/devlab-test-expert
 */
import { semanticSelectors } from '@playwright/test/utils';

devlab-srv-test-api 引用

/**
 * Source: devlab-test-expert/reference/api-testing-complete-guide.md
 */
const testCases = require('@test/expert/api-test-cases');

贡献指南

添加新知识卡

1. 确定主题覆盖范围 — 确保不与现有卡重复
2. 遵循标准格式 — 使用表格、代码示例、决策树
3. 引用权威来源 — RFC、官方文档、行业白皮书
4. 提供代码示例 — 可执行的 TypeScript/Python 片段
5. 关联相关卡片 — Cross-reference 增强导航

审核流程

Draft PR → Team Review → Merge to main
         ↓
Update index table in SKILL.md
Update references in sub-skills

Managed by Trellis. This skill follows the harness-ai-kit asset pattern for expert knowledge base. Last updated: 2026-07-22 Author: AI-assisted knowledge curation based on enterprise best practices.

参考文档：

• references/REFERENCE-README.md
• references/REFERENCE-TEST-DATA-MANAGEMENT.md
• references/REFERENCE-PROPERTY-BASED-TESTING.md
• references/REFERENCE-AI-SERVICE-TEST-TIERING.md