Build Testing

Build Testing

Rules

1. Every public function in build.py MUST have a test class in tests/test_build.py.
2. Every error path MUST have a test: missing file, unreachable host, invalid env.
3. NEVER mock probe_host to test infrastructure health. Probe the REAL hosts.
4. Mocking subprocess.run is justified — don't actually run ansible-playbook from pytest.
5. Mocking probe_host is ONLY justified in TestMain error-path tests where you need to isolate a DIFFERENT error (e.g., "missing playbook" needs to get past the probe step).
6. NEVER write a test that would pass identically if the infrastructure were offline (unless it's testing pure Python logic like string parsing).

Coverage matrix

Running tests

pytest tests/test_build.py -v

When tests fail

TestInfrastructureHealth failures mean REAL infrastructure problems:

• test_primary_host_reachable FAILED → home is down. Check power/network.
• test_ai_host_reachable FAILED → ai is down. NO WoL. Manual power-on required.
• test_mesh2_host_reachable FAILED → mesh2 is down. Try ./scripts/wol.sh mesh2.

These are NOT flaky tests. They are the early warning system. If they fail, your infrastructure has a real problem.

Justified mock patterns

Every patch() or monkeypatch call MUST include a comment with TWO parts:

1. WHY this mock is necessary (what side effect it prevents)
2. HOW the test still genuinely validates the feature despite the mock

If you cannot write both sentences, the mock is unjustified — remove it and test the real thing.

# JUSTIFIED: prevent pytest from running ansible-playbook (side effect: 5-min
# playbook run). Test validates command construction, not playbook execution.
monkeypatch.setattr("subprocess.run", fake_run)

# JUSTIFIED: test "missing binary" error path (side effect: none, but real
# binary is always present). Test validates the error message, not binary detection.
monkeypatch.setattr(build, "find_ansible_playbook", lambda: None)

# JUSTIFIED in TestMain ONLY: isolate a different error path
# (side effect: TCP probe blocks for 3s per host). Test validates "missing
# playbook" error, not host reachability — that's tested by TestInfrastructureHealth.
monkeypatch.setattr(build, "probe_host", lambda *a, **kw: True)

NEVER-justified mock patterns

# NEVER: mocking probe_host to test infrastructure health
monkeypatch.setattr(build, "probe_host", lambda *a, **kw: True)
env = {"PRIMARY_HOST": "10.0.0.1"}  # FAKE IP!
assert build.resolve_proxmox_host(env) == "10.0.0.1"  # TRIVIALLY OBVIOUS

# NEVER: mocking socket to test probe_host itself
@patch('socket.create_connection')
def test_probe(mock_conn):  # TESTS NOTHING REAL

Previous bug

TestResolveProxmoxHost (5 tests) monkeypatched probe_host with fake IPs. All 5 passed while all 3 WAN hosts (home, ai, mesh2) were offline. The ai host had been crashed by modprobe -r amdgpu and nobody knew because the "host probing tests" never actually probed any hosts. Replaced with TestInfrastructureHealth that probes real hosts from test.env.