Find Untested Sources
Purpose
Coverage tools answer "which lines were executed?" — they require a green build
and a passing test run, which is minutes-to-tens-of-minutes on a real repo. The
question this skill answers is different and much cheaper:
Which source files have no test file referencing any of their declared
types/symbols?
That's the question an agent asks before writing a new test — and it can be
answered statically in a few seconds by parsing source files, with no build,
no dependency resolution, and no compilation. The output is a deterministic
test-pairing map that lets the agent pick the next file to test without reading
the entire codebase first.
Two engines — pick one
This skill ships two interchangeable analyzers with a compatible JSON contract:
| Engine |
Script |
Use when |
| Roslyn (C#) |
scripts/Find-UntestedSources.cs |
The repo is .NET-only. Parses every .cs file with the Roslyn syntax API and does strict namespace disambiguation, so it is materially more accurate on duplicated short names like Settings or Context. |
| tree-sitter (polyglot) |
scripts/find_untested_sources.py |
The repo is not exclusively C#, or you want one tool across Python, TypeScript/JavaScript, Go, Java, Rust, Ruby, and C#. |
For a .NET-only repository, prefer the Roslyn engine — its namespace-aware
pairing beats the polyglot engine's identifier overlap.
When to Use
- User asks "where should I add tests?", "which files have no tests?", "find
untested code", "give me a test gap list", "what's the next file to test".
- Before invoking a test-generation agent, to produce a prioritized worklist.
- After generating tests, to verify each new test file pairs to a source file.
- To enumerate "weakly paired" source files (only one referring test) for
follow-up depth checks.
When Not to Use
- Line/branch coverage — use
coverage-analysis.
- CRAP-score / risk hotspots — use
coverage-analysis.
- Are existing tests strong? — use
test-gap-analysis (mutation reasoning)
or assertion-quality.
Roslyn engine (C#)
Prerequisites
- .NET SDK that supports file-based apps (
dotnet run script.cs). Pinned in the
repo's global.json (SDK 11 preview or later).
- No internet access required beyond the initial NuGet restore of
Microsoft.CodeAnalysis.CSharp on first run.
Usage
# From the skill folder
dotnet run scripts/Find-UntestedSources.cs -- <repo-root> [--top N]
# Save the report
dotnet run scripts/Find-UntestedSources.cs -- <repo-root> > pairing.json
# Iterate the untested list, highest-API-surface first
$report = Get-Content pairing.json | ConvertFrom-Json
$report.untested | Select-Object -First 10 source, decl_count, suggested_test_path
Diagnostics go to stderr; JSON goes to stdout.
Output schema
{
"repo": "<absolute path>",
"elapsed_ms": 8883,
"counts": {
"source_files": 3036,
"test_files": 867,
"untested_files": 1852,
"paired_files": 1184
},
"untested": [
{
"source": "src/Foo/Bar.cs",
"decl_count": 8, // # of type declarations in the file
"suggested_test_path": // mirror of source under a discovered test project
"tests/Foo.Tests/Bar/BarTests.cs"
}
],
"source_to_tests": {
"src/Foo/Baz.cs": [
"tests/Foo.Tests/BazTests.cs",
"tests/Foo.IntegrationTests/Scenarios/BazScenarios.cs"
]
}
}
How it works
- File discovery — recursive walk pruning
bin/, obj/, node_modules/,
.git/, .vs/, packages/, and any dotted subdir. Skips generated files
(.g.cs, .Designer.cs, .AssemblyInfo.cs).
- Test vs source classification — walks up to the nearest
.csproj and
marks it a test project if the project name ends in .Tests, .Test,
.UnitTests, .IntegrationTests, .E2E, .EndToEnd, .Spec, .Specs, or
the content references Microsoft.NET.Test.Sdk, MSTest.Sdk,
Microsoft.Testing.Platform, xunit, NUnit, TUnit, or
<IsTestProject>true</IsTestProject>.
- Source index (parallel) — parse each source file with
CSharpSyntaxTree.ParseText (syntax only, no compilation); record every
BaseTypeDeclarationSyntax / DelegateDeclarationSyntax as
(ShortName, EnclosingNamespace, FilePath).
- Test scan (parallel) — parse each test file, collect
using directives +
enclosing namespace, walk every IdentifierToken, look it up in the
short-name index, and disambiguate strictly: an identifier is attributed
only if the declaration's namespace matches one of the test file's using
directives, the enclosing namespace, or a prefix of them. This avoids noise
where common names like Settings or Context match every project.
- Pairing & suggestion — invert into
source → [tests]. Build a
production-to-test project map from <ProjectReference> entries; for each
untested source, mirror its in-project relative path under the referencing
test project to suggest a path.
- JSON emit — ordered by declaration count desc, then alphabetical.
Polyglot engine (tree-sitter)
Prerequisites
- Python 3.10+.
pip install tree-sitter-language-pack (single self-contained wheel that
bundles parsers for 300+ languages and the high-level process() API). No
native build, no per-language grammar install.
Usage
# From the skill folder
python scripts/find_untested_sources.py <repo-root>
# Restrict to a language (repeatable)
python scripts/find_untested_sources.py <repo-root> --lang python --lang typescript
# Truncate the report (top 20 by declared API surface)
python scripts/find_untested_sources.py <repo-root> --limit-untested 20 > pairing.json
# Iterate, highest-API-surface first
$report = Get-Content pairing.json | ConvertFrom-Json
$report.untested_sources | Select-Object -First 10 path, declaration_count, suggested_test_path
Pass --include-tested to additionally emit tested_sources (omitted by
default to keep the payload small for LLM consumption). Diagnostics go to
stderr; JSON goes to stdout.
Output schema
{
"repo_root": "<absolute path>",
"summary": {
"source_files": 3138,
"test_files": 761,
"tested_source_files": 1419,
"untested_source_files": 1719,
"orphan_test_files": 15,
"languages": ["csharp"]
},
"untested_sources": [
{
"path": "src/Foo/Bar.cs",
"language": "csharp",
"declaration_count": 8,
"declarations": ["Bar", "BarOptions", "IBar", "..."],
"suggested_test_path": "src/Foo/BarTests.cs"
}
],
"orphan_tests": [
{ "path": "tests/SomeIntegrationTest.cs", "language": "csharp" }
]
}
How it works
File discovery — recursive walk pruning common build/vendor dirs (bin,
obj, node_modules, target, dist, build, vendor, __pycache__,
.venv, .git, …) and generated files (.d.ts, .g.cs, .Designer.cs,
_pb2.py, *.min.js, AssemblyInfo.cs, …).
Language detection — detect_language_from_path maps the extension to a
supported language; unknown extensions are skipped.
Test-vs-source classification — per-language path heuristics:
| Language |
Test rule |
| Python |
path contains tests//test/; or filename starts with test_ or ends _test.py; or conftest.py. |
| JS/TS/TSX |
path contains __tests__, tests, test, spec, e2e; or filename contains .test./.spec.. |
| Go |
filename ends _test.go. |
| Java |
path contains test/tests; or filename ends Test.java/Tests.java. |
| Rust |
path contains tests//benches/. |
| C# |
path contains tests/; or project segment ends .Tests/.Test/.UnitTests/.IntegrationTests; or filename ends Tests/Test. |
| Ruby |
path contains spec//test/; or filename ends _spec.rb/_test.rb. |
Per-file extraction — process(text, ProcessConfig(structure, imports, symbols)) returns declared items, raw import statements, and a flat declared
-name list.
Pairing — for each test file, union import resolution (per language,
e.g. Python from pkg.mod import x → pkg/mod.py; Java import a.b.C; →
a/b/C.java; C# using is namespace-not-file, so a no-op) with identifier
overlap (word-like tokens, length ≥ 4, matched against declared names).
JSON emit — untested_sources ordered by declaration count descending.
Limitations (be honest with the agent)
Both engines are static, parse-only heuristics that trade a little accuracy for
orders-of-magnitude lower cost than coverage. Known gaps:
- Reflection / DI-resolved types referenced only via a string name or
container resolution won't be detected — the type's short name never appears
in the test source.
- Extension methods invoked as instance methods (C#): the declaring static
class is not named, so its file is not credited.
var, target-typed new(), pattern matching lose the type token; the
file-level union usually still catches it through other references.
- Short identifier names (polyglot, < 4 chars) are dropped to avoid noisy
pairings on names like
id, db, Tag.
- Monorepo path aliases (TS path mapping, Java module-info) are not
resolved; a suffix-match fallback may pick the wrong source if two files share
a trailing path segment.
For these cases, run actual coverage (coverage-analysis) on the unpaired
candidates the agent has already triaged.
Outputs the agent should consume
untested[*].source / untested_sources[*].path — pick the next source file
to test (highest declaration count first).
*.suggested_test_path — drop-in target for the new test file; the Roslyn
engine honors the test project that already <ProjectReference>s the source's
project, so dotnet sln add is not needed.
source_to_tests (Roslyn) / --include-tested tested_sources (polyglot) —
verify a newly written test file lands in the list for the intended source.
orphan_tests (polyglot) — tests that don't reference any same-language
source file; useful for triaging stale or integration-only tests.
1---2name: find-untested-sources3description: >4 Parse-only static analysis that pairs source files with the tests referencing5 them and emits JSON listing untested files ordered by API surface, each with a6 suggested_test_path. Roslyn engine for C#/.NET (namespace-aware), tree-sitter7 engine for polyglot repos (Python, TS/JS, Go, Java, Rust, Ruby).8 USE FOR: where to write tests next, which files have no tests, find untested9 code, build a source-to-test pairing map, prioritized test-gap worklist.10 DO NOT USE FOR: line/branch coverage or CRAP risk (use coverage-analysis);11 whether existing tests are strong (use test-gap-analysis or assertion-quality).12license: MIT13# Agent-orchestrated helper (invoked by name from code-testing-researcher and the14# code-testing pipeline); kept out of the model-facing skill menu so it does not15# consume the plugin's 15,000-char skill-menu budget or add routing noise that16# suppresses activation of the user-facing test skills. Still invocable by name.17metadata:18 source: https://skillmd.com/skills/stanfish06/find-untested-sources19---2021# Find Untested Sources2223## Purpose2425Coverage tools answer "which lines were executed?" — they require a green build26and a passing test run, which is minutes-to-tens-of-minutes on a real repo. The27question this skill answers is different and much cheaper:2829> _Which source files have no test file referencing any of their declared30> types/symbols?_3132That's the question an agent asks **before** writing a new test — and it can be33answered statically in a few seconds by parsing source files, with **no build,34no dependency resolution, and no compilation**. The output is a deterministic35test-pairing map that lets the agent pick the next file to test without reading36the entire codebase first.3738## Two engines — pick one3940This skill ships two interchangeable analyzers with a compatible JSON contract:4142| Engine | Script | Use when |43|--------|--------|----------|44| **Roslyn (C#)** | `scripts/Find-UntestedSources.cs` | The repo is **.NET-only**. Parses every `.cs` file with the Roslyn syntax API and does strict **namespace disambiguation**, so it is materially more accurate on duplicated short names like `Settings` or `Context`. |45| **tree-sitter (polyglot)** | `scripts/find_untested_sources.py` | The repo is **not exclusively C#**, or you want one tool across Python, TypeScript/JavaScript, Go, Java, Rust, Ruby, and C#. |4647For a .NET-only repository, **prefer the Roslyn engine** — its namespace-aware48pairing beats the polyglot engine's identifier overlap.4950## When to Use5152- User asks "where should I add tests?", "which files have no tests?", "find53 untested code", "give me a test gap list", "what's the next file to test".54- Before invoking a test-generation agent, to produce a prioritized worklist.55- After generating tests, to verify each new test file pairs to a source file.56- To enumerate "weakly paired" source files (only one referring test) for57 follow-up depth checks.5859## When Not to Use6061- **Line/branch coverage** — use `coverage-analysis`.62- **CRAP-score / risk hotspots** — use `coverage-analysis`.63- **Are existing tests strong?** — use `test-gap-analysis` (mutation reasoning)64 or `assertion-quality`.6566## Roslyn engine (C#)6768### Prerequisites6970- .NET SDK that supports file-based apps (`dotnet run script.cs`). Pinned in the71 repo's `global.json` (SDK 11 preview or later).72- No internet access required beyond the initial NuGet restore of73 `Microsoft.CodeAnalysis.CSharp` on first run.7475### Usage7677```powershell78# From the skill folder79dotnet run scripts/Find-UntestedSources.cs -- <repo-root> [--top N]8081# Save the report82dotnet run scripts/Find-UntestedSources.cs -- <repo-root> > pairing.json8384# Iterate the untested list, highest-API-surface first85$report = Get-Content pairing.json | ConvertFrom-Json86$report.untested | Select-Object -First 10 source, decl_count, suggested_test_path87```8889Diagnostics go to stderr; JSON goes to stdout.9091### Output schema9293```jsonc94{95 "repo": "<absolute path>",96 "elapsed_ms": 8883,97 "counts": {98 "source_files": 3036,99 "test_files": 867,100 "untested_files": 1852,101 "paired_files": 1184102 },103 "untested": [104 {105 "source": "src/Foo/Bar.cs",106 "decl_count": 8, // # of type declarations in the file107 "suggested_test_path": // mirror of source under a discovered test project108 "tests/Foo.Tests/Bar/BarTests.cs"109 }110 ],111 "source_to_tests": {112 "src/Foo/Baz.cs": [113 "tests/Foo.Tests/BazTests.cs",114 "tests/Foo.IntegrationTests/Scenarios/BazScenarios.cs"115 ]116 }117}118```119120### How it works1211221. **File discovery** — recursive walk pruning `bin/`, `obj/`, `node_modules/`,123 `.git/`, `.vs/`, `packages/`, and any dotted subdir. Skips generated files124 (`.g.cs`, `.Designer.cs`, `.AssemblyInfo.cs`).1252. **Test vs source classification** — walks up to the nearest `.csproj` and126 marks it a test project if the project name ends in `.Tests`, `.Test`,127 `.UnitTests`, `.IntegrationTests`, `.E2E`, `.EndToEnd`, `.Spec`, `.Specs`, or128 the content references `Microsoft.NET.Test.Sdk`, `MSTest.Sdk`,129 `Microsoft.Testing.Platform`, `xunit`, `NUnit`, `TUnit`, or130 `<IsTestProject>true</IsTestProject>`.1313. **Source index (parallel)** — parse each source file with132 `CSharpSyntaxTree.ParseText` (syntax only, no compilation); record every133 `BaseTypeDeclarationSyntax` / `DelegateDeclarationSyntax` as134 `(ShortName, EnclosingNamespace, FilePath)`.1354. **Test scan (parallel)** — parse each test file, collect `using` directives +136 enclosing namespace, walk every `IdentifierToken`, look it up in the137 short-name index, and **disambiguate strictly**: an identifier is attributed138 only if the declaration's namespace matches one of the test file's `using`139 directives, the enclosing namespace, or a prefix of them. This avoids noise140 where common names like `Settings` or `Context` match every project.1415. **Pairing & suggestion** — invert into `source → [tests]`. Build a142 production-to-test project map from `<ProjectReference>` entries; for each143 untested source, mirror its in-project relative path under the referencing144 test project to suggest a path.1456. **JSON emit** — ordered by declaration count desc, then alphabetical.146147## Polyglot engine (tree-sitter)148149### Prerequisites150151- Python 3.10+.152- `pip install tree-sitter-language-pack` (single self-contained wheel that153 bundles parsers for 300+ languages and the high-level `process()` API). No154 native build, no per-language grammar install.155156### Usage157158```powershell159# From the skill folder160python scripts/find_untested_sources.py <repo-root>161162# Restrict to a language (repeatable)163python scripts/find_untested_sources.py <repo-root> --lang python --lang typescript164165# Truncate the report (top 20 by declared API surface)166python scripts/find_untested_sources.py <repo-root> --limit-untested 20 > pairing.json167168# Iterate, highest-API-surface first169$report = Get-Content pairing.json | ConvertFrom-Json170$report.untested_sources | Select-Object -First 10 path, declaration_count, suggested_test_path171```172173Pass `--include-tested` to additionally emit `tested_sources` (omitted by174default to keep the payload small for LLM consumption). Diagnostics go to175stderr; JSON goes to stdout.176177### Output schema178179```jsonc180{181 "repo_root": "<absolute path>",182 "summary": {183 "source_files": 3138,184 "test_files": 761,185 "tested_source_files": 1419,186 "untested_source_files": 1719,187 "orphan_test_files": 15,188 "languages": ["csharp"]189 },190 "untested_sources": [191 {192 "path": "src/Foo/Bar.cs",193 "language": "csharp",194 "declaration_count": 8,195 "declarations": ["Bar", "BarOptions", "IBar", "..."],196 "suggested_test_path": "src/Foo/BarTests.cs"197 }198 ],199 "orphan_tests": [200 { "path": "tests/SomeIntegrationTest.cs", "language": "csharp" }201 ]202}203```204205### How it works2062071. **File discovery** — recursive walk pruning common build/vendor dirs (`bin`,208 `obj`, `node_modules`, `target`, `dist`, `build`, `vendor`, `__pycache__`,209 `.venv`, `.git`, …) and generated files (`.d.ts`, `.g.cs`, `.Designer.cs`,210 `_pb2.py`, `*.min.js`, `AssemblyInfo.cs`, …).2112. **Language detection** — `detect_language_from_path` maps the extension to a212 supported language; unknown extensions are skipped.2133. **Test-vs-source classification** — per-language path heuristics:214215 | Language | Test rule |216 |---|---|217 | Python | path contains `tests/`/`test/`; or filename starts with `test_` or ends `_test.py`; or `conftest.py`. |218 | JS/TS/TSX | path contains `__tests__`, `tests`, `test`, `spec`, `e2e`; or filename contains `.test.`/`.spec.`. |219 | Go | filename ends `_test.go`. |220 | Java | path contains `test`/`tests`; or filename ends `Test.java`/`Tests.java`. |221 | Rust | path contains `tests/`/`benches/`. |222 | C# | path contains `tests/`; or project segment ends `.Tests`/`.Test`/`.UnitTests`/`.IntegrationTests`; or filename ends `Tests`/`Test`. |223 | Ruby | path contains `spec/`/`test/`; or filename ends `_spec.rb`/`_test.rb`. |2242254. **Per-file extraction** — `process(text, ProcessConfig(structure, imports,226 symbols))` returns declared items, raw import statements, and a flat declared227 -name list.2285. **Pairing** — for each test file, union **import resolution** (per language,229 e.g. Python `from pkg.mod import x` → `pkg/mod.py`; Java `import a.b.C;` →230 `a/b/C.java`; C# `using` is namespace-not-file, so a no-op) with **identifier231 overlap** (word-like tokens, length ≥ 4, matched against declared names).2326. **JSON emit** — `untested_sources` ordered by declaration count descending.233234## Limitations (be honest with the agent)235236Both engines are static, parse-only heuristics that trade a little accuracy for237orders-of-magnitude lower cost than coverage. Known gaps:238239- **Reflection / DI-resolved types** referenced only via a string name or240 container resolution won't be detected — the type's short name never appears241 in the test source.242- **Extension methods** invoked as instance methods (C#): the declaring static243 class is not named, so its file is not credited.244- **`var`, target-typed `new()`, pattern matching** lose the type token; the245 file-level union usually still catches it through other references.246- **Short identifier names** (polyglot, < 4 chars) are dropped to avoid noisy247 pairings on names like `id`, `db`, `Tag`.248- **Monorepo path aliases** (TS path mapping, Java module-info) are not249 resolved; a suffix-match fallback may pick the wrong source if two files share250 a trailing path segment.251252For these cases, run actual coverage (`coverage-analysis`) on the unpaired253candidates the agent has already triaged.254255## Outputs the agent should consume256257- `untested[*].source` / `untested_sources[*].path` — pick the next source file258 to test (highest declaration count first).259- `*.suggested_test_path` — drop-in target for the new test file; the Roslyn260 engine honors the test project that already `<ProjectReference>`s the source's261 project, so `dotnet sln add` is not needed.262- `source_to_tests` (Roslyn) / `--include-tested` `tested_sources` (polyglot) —263 verify a newly written test file lands in the list for the intended source.264- `orphan_tests` (polyglot) — tests that don't reference any same-language265 source file; useful for triaging stale or integration-only tests.