Testforge

testforge

A skill for the Novera-AI-skills suite that turns untested Python source into a runnable pytest suite and then tells you exactly where the coverage is thin.

The problem it solves: AI-generated code often ships with no tests or with tests that only assert is not None. testforge scaffolds the structure so the agent spends its effort on meaningful assertions instead of boilerplate.

Both scripts are pure standard library - no pip install needed to run them.

When to use

Trigger this skill when the user says any of:

• 生成测试 / 写单测 / 补测试 / 给这个函数写测试 / 增加覆盖率
• testforge / generate tests / add coverage

Also use it proactively after producing a Python module that has no tests.

Workflow

Follow these steps in order. Each step names the script and its job.

1. Generate stubs

Run gen_pytest.py against the source file:

python scripts/gen_pytest.py <source.py>

It parses the file with the ast module and writes test_<source>_gen.py next to it, with one skeleton test per top-level function and per class method. Each stub imports the callable, calls it with None placeholders, and leaves a single # TODO placeholder assertion. The script prints the number of stubs and the output path.

What it covers:

• Top-level functions -> def test_<name>().
• Class methods -> def test_<Class>_<method>() (instance, @staticmethod, and @classmethod are each handled; __init__ is skipped).

2. Fill the assertions

Open the generated file and replace every # TODO with a real check. Use references/test-patterns.md for copy-paste templates:

• Normal path: assert the return value for valid input.
• Boundary: empty / zero / None / negative / very large.
• Exception: with pytest.raises(<ExpectedError>): for bad input.
• Isolation: patch slow or external calls with unittest.mock.
• Prefer @pytest.mark.parametrize over copy-pasted tests.

A filled stub looks like this:

def test_divide():
    assert divide(10, 2) == 5


def test_divide_by_zero():
    import pytest
    with pytest.raises(ValueError):
        divide(10, 0)

3. Run pytest

python -m pytest test_<source>_gen.py

gen_pytest.py does not require pytest, but running the suite does. Install it with pip install pytest if missing.

4. Find the gaps

Point coverage_hint.py at the test directory:

python scripts/coverage_hint.py <test_dir>

It counts test files and test functions, flags any test that is still a stub or asserts nothing, and prints concrete next steps (fill placeholders, add exception tests, add boundary cases, use parametrize). If pytest is on the PATH it also prints the exact command to run the suite.

Loop steps 2-4 until coverage_hint reports 0 stub(s) and the suite is green.

Scripts

scripts/gen_pytest.py

• Input: a single .py source file.
• Output: test_<source>_gen.py in the same directory.
• Pure stdlib (ast). Deterministic. Errors are reported on stderr with a non-zero exit code (2 for bad args / missing file, 1 for parse failure).

python scripts/gen_pytest.py path/to/module.py
# -> generated 5 test stub(s) -> path/to/test_module_gen.py

scripts/coverage_hint.py

• Input: a directory containing test files (test_*.py or *_test.py).
• Output: a scan summary plus suggestions. Does not need pytest installed.
• Pure stdlib. Detects the pytest executable and prints the run command when present.

python scripts/coverage_hint.py path/to/tests

Reference material

• references/test-patterns.md - normal-path / boundary / exception / mock patterns with templates to paste into the generated stubs.

Limitations

• gen_pytest.py generates structure, not intelligence. Placeholder arguments are None; the agent must supply real inputs or the call may raise at runtime. That failure is the signal to fill the stub.
• It does not compute line coverage percentages (that needs pytest-cov). It reports structural gaps: missing assertions, unfilled stubs, and missing exception/boundary/parametrize coverage.
• Nested functions and module-level lambdas are not scaffolded; only top-level functions and class methods are.