Rest Assured CI
1. Choose The Pipeline Shape
- Detect the CI system first.
- Run smoke tags before full regression when pipeline time matters.
- Use containers or disposable services for integration tests.
2. Configure Execution
- Use provider guidance from github-actions.md, gitlab.md, or other-ci.md.
- Use junit5-tagging-and-suites.md for JUnit 5 selection.
- Use docker-and-containers.md for containerized execution.
- Use parallelization.md for sharding or split-by-tag execution.
- Use reporting-and-artifacts.md for JUnit XML and logs.
- Use quality-gates.md when the pipeline must fail on coverage, drift, or weak-assertion thresholds.
- Use
python scripts/evaluate_quality_gates.pywhen the quality-gate evaluation must be deterministic and reusable across CI jobs.
3. Add Quality Gates
- Fail the pipeline on contract-enforcement regressions only when the repo has explicitly opted into strict gates.
- Use the coverage-gap report to enforce
no uncovered operationsorno unreviewed drift-only operationswhen the team wants coverage hardening. - Use the assertion-strength report to enforce thresholds such as
no weak testsormax transport-only tests. - Keep quality gates separate for runtime-aligned suites and contract-enforcement suites when both exist.
- Regenerate the reporting bundle before applying report-based gates.
- Prefer
python scripts/evaluate_quality_gates.py --coverage-gap <gap-md> --assertion-strength <strength-md> --contract-mismatches <mismatch-md>instead of rewriting threshold logic inline in CI YAML.
4. Troubleshooting
- Problem: CI cannot reach the service under test. Fix: Verify network, startup ordering, and environment variables before changing the tests.
- Problem: Tag-based selection runs zero tests. Fix: Verify JUnit 5 tag configuration and test task flags.
- Problem: The pipeline fails because report-based gates are stale. Fix: Regenerate the bundle reports before evaluating gate thresholds.
- Problem: CI duplicates threshold logic in multiple jobs.
Fix: Move the evaluation into
scripts/evaluate_quality_gates.py.