Build Assertion Strength Report
1. Gather Inputs
- Require the Rest Assured test sources.
- Require
docs/testing/test-mapping-report.mdwhen endpoint context is needed.
2. Choose The Files
- Prefer
docs/testing/assertion-strength-report.md. - When the user wants a human-friendly report, also generate
docs/testing/html/assertion-strength-report.html.
3. Build The Assessment
- Score each test method from the code, not from assumptions.
- Detect status, header, content-type, body, schema, and workflow-style assertions.
- Classify each test method as
Weak,Moderate, orStrong. - Call out tests that only assert status codes or only transport-level behavior.
- Recommend where semantic assertions should be added next.
4. Output Shape
- Start from assertion-strength-report-template.md.
- Keep the exact section headings stable.
- Run
python scripts/generate_assertion_strength_report.py --tests-root <tests-root> --mapping <mapping-md> --output-md <report-md> --output-html <report-html>.
5. Keep It Actionable
- Highlight the weakest tests first.
- Explain why a test is weak or strong.
- Update the session-state artifact after generating or changing the report.
6. Examples
- Input:
Which tests only assert status codes?Output: Create an assertion-strength report that classifies tests and isolates weak ones. - Input:
Assess the quality of these Rest Assured assertions.Output: Produce a markdown and HTML report with strengths, weaknesses, and recommended next assertions.
7. Troubleshooting
- Problem: The same test method mixes several requests. Fix: Score the whole method and label it as workflow-oriented when multiple HTTP calls are present.
- Problem: The repo uses helper methods for assertions. Fix: Mark the result as approximate and recommend manual review for the affected tests.