# Pytest Coverage > Run coverage.py with pytest to measure Python source code execution (line and branch coverage). Use ONLY when asked to run coverage.py, check which Python lines/branches are executed, or generate a coverage.py HTML report. Do NOT use for tracking which toolkit tools have YAML test cases — use coverage-calculator for that. Use when this capability is needed. - Skill: `tomevault-io/pytest-coverage-4` (Agent Skill, multi-file: 2 files) - Install (CLI): `npx skillmds@latest add tomevault-io/pytest-coverage-4` - Raw SKILL.md: https://api.skillmd.com/api/skills/tomevault-io/pytest-coverage-4/raw - Safety review: pending (external: skill-scanner PASS, skillspector PASS) - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Web & Frontend - Author: tomevault-io (https://skillmd.com/u/tomevault-io) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/tomevault-io/pytest-coverage-4 --- # Pytest Coverage Skill This skill covers running **branch coverage** with `coverage.py` against any Python source module in the Alita SDK. It tells you which lines and branches are never executed. --- ## Concept: What Coverage Measures | | Branch Coverage | |---|---| | What it measures | Code was **executed** | | Catches missing `assert` | No | | Catches wrong assertion | No | | Speed | ~seconds (single run) | | Tool | `coverage run --branch` | Coverage tells you *what was run*. It does **not** tell you whether the tests assert the right things — a line can be executed without any assertion validating its result. --- ## Prerequisites ```bash source venv/Scripts/activate pip install coverage pytest ``` > **Do NOT use `pytest --cov=`** — it triggers circular import errors on import of `alita_sdk`. Always use the `coverage run` wrapper instead. --- ## Step 1: Run Coverage + Report in One Command **Always combine `coverage run` and `coverage report` in a single shell command using `;`** so the report is always generated regardless of test failures. Never run tests separately first to check if they pass — that wastes time and was explicitly flagged as wrong behaviour. ```bash source venv/Scripts/activate && \ python -m coverage run --branch --source=alita_sdk \ -m pytest -q --no-header --no-cov 2>&1; \ python -m coverage report --include="*/path/to/source.py" -m 2>&1 ``` Replace `` with the path to the test file, e.g. `tests/runtime/test_my_module.py`. Replace `*/path/to/source.py` with a glob matching only the source file(s) under test (e.g. `*/codeparser.py`). > **Key rules:** > - Use `;` (not `&&`) between `coverage run` and `coverage report` so the report runs even when tests fail. > - **Do NOT** run `pytest` separately before coverage — the coverage run IS the test run. > - **Do NOT** use `pytest --cov=`** — it triggers circular import errors on import of `alita_sdk`. > - Always pass `--include` to filter the report to only the files you care about. --- ## Step 2: Generate the Report (standalone, if .coverage already exists) If a `.coverage` data file already exists from a previous run: ```bash python -m coverage report \ --include="*/path/to/source1.py,*/path/to/source2.py" -m ``` Use `--include` to filter to only the module(s) you care about. Without it the report covers all of `alita_sdk`. ### Report columns explained ``` Name Stmts Miss Branch BrPart Cover Missing ------------------------------------------------------------------------- alita_sdk/.../security.py 156 71 56 9 50% 84-91, 113-134, ... ``` | Column | Meaning | |---|---| | `Stmts` | Total executable statements | | `Miss` | Statements never executed | | `Branch` | Total conditional branches | | `BrPart` | Branches only partially covered (e.g. `if` taken but `else` never, or vice versa) | | `Cover` | Overall coverage percentage | | `Missing` | Exact line ranges to target with new tests | --- ## Step 3: Generate an HTML Report (optional) For a navigable, line-by-line view: ```bash python -m coverage html --include="*/path/to/source1.py" # Opens htmlcov/index.html in a browser ``` --- ## Step 4: Interpret and Act ### Priority order for new tests 1. **Missing lines (never executed)** — the most urgent gap. There is zero confidence for that code. 2. **Partial branches (`BrPart`)** — one side of a conditional is never tested. Common cause: only the "happy path" is covered. 3. **Lines covered but no assertion validation** — the line runs but no assertion checks the result. Coverage cannot detect this gap; it can only be found by scrutinising test assertions manually. ### Useful coverage thresholds | Coverage | Recommended action | |---|---| | < 60% | Write basic happy-path and error-path tests to reach > 80% before doing anything else | | 60–80% | Add edge-case and branch-flip tests for `BrPart` lines | | > 80% | Add targeted assertion tests to validate correctness of covered logic | | 100% | Coverage is maxed — focus on the quality of assertions in covered code | --- ## Step 5: Re-run After Adding Tests After writing new tests, re-run the same two commands from Steps 1 and 2 to confirm coverage improved. Repeat until the target threshold is reached. --- ## Common Mistakes | Mistake | Fix | |---|---| | Running pytest separately before coverage to "check if tests pass" | Don't — `coverage run` IS the test run. Combine with `;` and let it run regardless of failures | | Using `&&` between `coverage run` and `coverage report` | Use `;` so `coverage report` always runs even when tests fail | | Using `pytest --cov=alita_sdk` | Use `coverage run --branch --source=alita_sdk -m pytest ...` instead | | Forgetting `--branch` | Without it, only statement coverage is measured; partial branches are invisible | | Reporting without `--include` | The full `alita_sdk` report is noisy; filter to the specific files under test | | Treating 100% coverage as "done" | Coverage only proves paths were executed, not that the assertions are correct | --- ## Quick Reference ```bash # Run coverage source venv/Scripts/activate python -m coverage run --branch --source=alita_sdk \ -m pytest tests/path/to/test_module.py -q --no-header --no-cov # Terminal report (with missing line numbers) python -m coverage report --include="*/path/to/my_module.py" -m # HTML report python -m coverage html --include="*/path/to/my_module.py" ``` --- > Source: [ProjectAlita/alita-sdk](https://github.com/ProjectAlita/alita-sdk) — distributed by [TomeVault](https://tomevault.io).