1545 Requirements B384ea9b

Requirements: Resolve the Rest of GitHub Issue #134 Completely

Metadata

• Feature: resolving-the-rest-of-134-completely
• Status: APPROVED
• Created: 2026-02-06
• Issue: https://github.com/klambros/ZERG/issues/134
• Title: Performance: Multiple rglob scans traverse directory tree repeatedly

1. Problem Statement

Issue #134 identified that ZERG performs redundant rglob directory traversals across multiple modules. A shared fs_utils.collect_files() utility was created (PR #161) and adopted by repo_map.py and stack_detector.py, but 18 rglob calls across 16 files remain unconverted.

Each unconverted rglob call independently walks the directory tree, wasting I/O on large projects. The fix is mechanical: replace each scattered rglob with a call to fs_utils.collect_files() (or a thin wrapper), preserving existing filter/exclusion behavior.

2. Current State

Already Migrated (2 modules)

fs_utils.py Itself (1 call — canonical, no change needed)

Already Single-Pass (1 call — no change needed)

Remaining rglob Calls to Migrate (18 calls across 16 files)

Pattern A — Single-extension rglob (10 calls) Simple rglob("*.py") or rglob("*.md") replaceable with collect_files(root, extensions={".py"}).

Pattern B — Multi-extension loop (4 calls) Loop over multiple extensions, each calling rglob(ext). Replace with single collect_files() call.

Pattern C — Wildcard rglob with inline filtering (3 calls) rglob("*") with custom is_file() / name checks. May need collect_files() or keep as-is if filter logic is unique.

Pattern D — Specialized pattern (1 call)

Pattern E — Dynamic pattern (1 call)

3. Functional Requirements

FR-1: Migrate Pattern A calls to collect_files()

Convert all 10 single-extension rglob calls to use fs_utils.collect_files(). Each site gets the flat list from the grouped dict: collect_files(root, {".py"})[".py"].

FR-2: Migrate Pattern B calls to collect_files()

Convert all 4 multi-extension loop sites. Replace the for-loop-over-extensions pattern with a single collect_files() call that returns all needed extensions at once.

FR-3: Migrate Pattern C calls where beneficial

The 3 wildcard rglob sites (detector.py, dive_adapter.py, hadolint_adapter.py) already do rglob("*") with filtering. Evaluate whether collect_files() with extensions=None provides meaningful benefit. If the directory is small or the filtering is highly specialized (e.g. Dockerfile name matching), document the decision to keep or convert.

FR-4: Handle Pattern D and E pragmatically

• rush.py:323 — .gsd is tiny. Document as intentionally kept (negligible perf impact).
• ast_analyzer.py:450 — Dynamic pattern parameter. Add collect_files() support or document exception.

FR-5: Extend collect_files() if needed

If any migration site needs capabilities collect_files() doesn't have (e.g. name-based matching, suffix-less files), extend the function minimally. Do NOT over-engineer — only add what actual call sites need.

FR-6: Preserve existing exclusion behavior

Each call site may have its own exclusion logic (e.g. __pycache__, hidden dirs, test files). The migration must preserve identical file sets. Default excludes in fs_utils._DEFAULT_EXCLUDES cover most cases but verify per-site.

FR-7: No behavioral changes

This is a pure refactor. No module should return different results after migration. Tests must pass identically.

4. Non-Functional Requirements

NFR-1: Performance

The whole point of #134. After migration, repeated directory scans within a single invocation should share results where possible.

NFR-2: No new dependencies

fs_utils uses only pathlib. Keep it that way.

NFR-3: Test coverage

Each migrated site must have its existing tests still pass. No new test files needed — this is a refactor.

5. Scope Boundaries

In Scope

• All 19 rglob call sites listed above (minus fs_utils.py itself)
• Minimal extensions to collect_files() if required
• Updating imports in affected modules

Out of Scope

• Test file rglob calls (only production zerg/ code)
• Performance benchmarking (the improvement is self-evident: fewer traversals)
• Caching/memoization of collect_files results across calls (future optimization)

6. Acceptance Criteria

1. grep -r '\.rglob(' zerg/ --include='*.py' returns ONLY:
   • zerg/fs_utils.py:58 (the canonical single-pass)
   • Any explicitly documented exceptions (rush.py, ast_analyzer.py, security_rules.py:220)
2. make test passes (or equivalent CI suite)
3. ruff check passes
4. mypy passes
5. No new files created except if collect_files() needs a helper

7. Risks

8. Open Questions

None — scope is well-defined and purely mechanical.