Find Untested Sources
Purpose
Coverage tools answer "which lines were executed?" — they require a green build
and a passing test run, which is minutes-to-tens-of-minutes on a real repo. The
question this skill answers is different and much cheaper:
Which source files have no test file referencing any of their declared
types/symbols?
That's the question an agent asks before writing a new test — and it can be
answered statically in a few seconds by parsing source files, with no build,
no dependency resolution, and no compilation. The output is a deterministic
test-pairing map that lets the agent pick the next file to test without reading
the entire codebase first.
Two engines — pick one
This skill ships two interchangeable analyzers with a compatible JSON contract:
| Engine |
Script |
Use when |
| Roslyn (C#) |
scripts/Find-UntestedSources.cs |
The repo is .NET-only. Parses every .cs file with the Roslyn syntax API and does strict namespace disambiguation, so it is materially more accurate on duplicated short names like Settings or Context. |
| tree-sitter (polyglot) |
scripts/find_untested_sources.py |
The repo is not exclusively C#, or you want one tool across C#, Python, TypeScript/JavaScript, Go, Java, Rust, Ruby, Kotlin, Swift, PowerShell, and C++. |
For a .NET-only repository, prefer the Roslyn engine — its namespace-aware
pairing beats the polyglot engine's identifier overlap.
Required workflow
- Use the narrowest repository or package root named by the caller. Do not scan
a parent workspace when the request identifies a subdirectory.
- Execute the appropriate analyzer once. Do not replace analyzer execution with
manual globbing, filename matching, or visual inspection.
For polyglot analysis, pass
--include-tested when the answer must distinguish
paired sources from unpaired sources.
"Static pairing only" prohibits compiling the target repository and running
its tests; it does not prohibit launching this skill's parse-only analyzer.
State that distinction briefly when the caller also says "do not build."
Treat analyzer dependencies as environment prerequisites: do not install
packages, try the wrong engine, build the repository, or fall back to a manual
scan when an analyzer invocation fails. Report the prerequisite failure instead.
- Base the result on the analyzer's JSON. Preserve its paired/unpaired
classification and suggested relative path; do not guess a different path.
- When the caller named a subdirectory, prefix analyzer-relative paths with
that subdirectory so reported paths are workspace-relative.
- Report the requested result plus the static-pairing coverage caveat. Do not
append build, package-install, test-run, or coverage commands. When paired
sources exist, name their covering test files so the unpaired classification
is auditable.
When to Use
- User asks "where should I add tests based on source pairing?", "which files
have no tests?", "find unpaired source files", or "give me a static test gap
list".
- Before invoking a test-generation agent, to produce a source-pairing worklist.
- After generating tests, to verify each new test file pairs to a source file.
- To enumerate "weakly paired" source files (only one referring test) for
follow-up depth checks.
When Not to Use
- Line/branch coverage — use
coverage-analysis.
- Priorities derived from real coverage data — use
coverage-analysis.
- CRAP-score / risk hotspots — use
coverage-analysis.
- Are existing tests strong? — use
test-gap-analysis (mutation reasoning)
or assertion-quality.
Roslyn engine (C#)
Prerequisites
- .NET SDK that supports file-based apps (
dotnet run script.cs). Pinned in the
repo's global.json (SDK 11 preview or later).
- No internet access required beyond the initial NuGet restore of
Microsoft.CodeAnalysis.CSharp on first run.
Usage
# From the skill folder
dotnet run scripts/Find-UntestedSources.cs -- <repo-root> [--top N]
# Save the report
dotnet run scripts/Find-UntestedSources.cs -- <repo-root> > pairing.json
# Iterate the untested list, highest-API-surface first
$report = Get-Content pairing.json | ConvertFrom-Json
$report.untested | Select-Object -First 10 source, decl_count, suggested_test_path
Diagnostics go to stderr; JSON goes to stdout.
Output schema
{
"repo": "<absolute path>",
"elapsed_ms": 8883,
"counts": {
"source_files": 3036,
"test_files": 867,
"untested_files": 1852,
"paired_files": 1184
},
"untested": [
{
"source": "src/Foo/Bar.cs",
"decl_count": 8, // # of type declarations in the file
"suggested_test_path": // mirror of source under a discovered test project
"tests/Foo.Tests/Bar/BarTests.cs"
}
],
"source_to_tests": {
"src/Foo/Baz.cs": [
"tests/Foo.Tests/BazTests.cs",
"tests/Foo.IntegrationTests/Scenarios/BazScenarios.cs"
]
}
}
How it works
- File discovery — recursive walk pruning
bin/, obj/, node_modules/,
.git/, .vs/, packages/, and any dotted subdir. Skips generated files
(.g.cs, .Designer.cs, .AssemblyInfo.cs).
- Test vs source classification — walks up to the nearest
.csproj and
marks it a test project if the project name ends in .Tests, .Test,
.UnitTests, .IntegrationTests, .E2E, .EndToEnd, .Spec, .Specs, or
the content references Microsoft.NET.Test.Sdk, MSTest.Sdk,
Microsoft.Testing.Platform, xunit, NUnit, TUnit, or
<IsTestProject>true</IsTestProject>.
- Source index (parallel) — parse each source file with
CSharpSyntaxTree.ParseText (syntax only, no compilation); record every
BaseTypeDeclarationSyntax / DelegateDeclarationSyntax as
(ShortName, EnclosingNamespace, FilePath).
- Test scan (parallel) — parse each test file, collect
using directives +
enclosing namespace, walk every IdentifierToken, look it up in the
short-name index, and disambiguate strictly: an identifier is attributed
only if the declaration's namespace matches one of the test file's using
directives, the enclosing namespace, or a prefix of them. This avoids noise
where common names like Settings or Context match every project.
- Pairing & suggestion — invert into
source → [tests]. Build a
production-to-test project map from <ProjectReference> entries; for each
untested source, mirror its in-project relative path under the referencing
test project to suggest a path.
- JSON emit — ordered by declaration count desc, then alphabetical.
Polyglot engine (tree-sitter)
Prerequisites
- Python 3.10+.
pip install tree-sitter-language-pack (single self-contained wheel that
bundles parsers for 300+ languages and the high-level process() API). No
native build, no per-language grammar install.
Usage
# From the skill folder
python scripts/find_untested_sources.py <repo-root>
# Restrict to a language (repeatable)
python scripts/find_untested_sources.py <repo-root> --lang python --lang typescript
# Truncate the report (top 20 by declared API surface)
python scripts/find_untested_sources.py <repo-root> --limit-untested 20 > pairing.json
# Iterate, highest-API-surface first
$report = Get-Content pairing.json | ConvertFrom-Json
$report.untested_sources | Select-Object -First 10 path, declaration_count, suggested_test_path
Pass --include-tested to additionally emit tested_sources (omitted by
default to keep the payload small for LLM consumption). Diagnostics go to
stderr; JSON goes to stdout.
Output schema
{
"repo_root": "<absolute path>",
"summary": {
"source_files": 3138,
"test_files": 761,
"tested_source_files": 1419,
"untested_source_files": 1719,
"orphan_test_files": 15,
"languages": ["csharp"]
},
"untested_sources": [
{
"path": "src/Foo/Bar.cs",
"language": "csharp",
"declaration_count": 8,
"declarations": ["Bar", "BarOptions", "IBar", "..."],
"suggested_test_path": "src/Foo/BarTests.cs"
}
],
"orphan_tests": [
{ "path": "tests/SomeIntegrationTest.cs", "language": "csharp" }
]
}
How it works
File discovery — recursive walk pruning common build/vendor dirs (bin,
obj, node_modules, target, dist, build, vendor, __pycache__,
.venv, .git, …) and generated files (.d.ts, .g.cs, .Designer.cs,
_pb2.py, *.min.js, AssemblyInfo.cs, …).
Language detection — detect_language_from_path maps the extension to a
supported language; unknown extensions are skipped.
Test-vs-source classification — per-language path heuristics:
| Language |
Test rule |
| Python |
path contains tests//test/; or filename starts with test_ or ends _test.py; or conftest.py. |
| JS/TS/TSX |
path contains __tests__, tests, test, spec, e2e; or filename contains .test./.spec.. |
| Go |
filename ends _test.go. |
| Java |
path contains test/tests; or filename ends Test.java/Tests.java. |
| Rust |
path contains tests//benches/. |
| C# |
path contains tests/; or project segment ends .Tests/.Test/.UnitTests/.IntegrationTests; or filename ends Tests/Test. |
| Ruby |
path contains spec//test/; or filename ends _spec.rb/_test.rb. |
| Kotlin |
path contains test//tests//spec/; or filename ends Test.kt/Tests.kt/Spec.kt. |
| Swift |
path contains test//tests//uitests//integrationtests/ (case-insensitive); or filename ends Test.swift/Tests.swift. |
| PowerShell |
path contains test//tests//pester/; or filename ends .Tests.ps1/.Test.ps1. |
| C++ |
path contains test//tests//testing/; or filename starts test_ or ends _test.cpp/_tests.cpp. |
Per-file extraction — process(text, ProcessConfig(structure, imports, symbols)) returns declared items, raw import statements, and a flat declared
-name list.
Pairing — for each test file, union import resolution (per language,
e.g. Python from pkg.mod import x → pkg/mod.py; Java import a.b.C; →
a/b/C.java; C# using is namespace-not-file, so a no-op) with identifier
overlap (word-like tokens, length ≥ 4, matched against declared names).
JSON emit — untested_sources ordered by declaration count descending.
Limitations (be honest with the agent)
Both engines are static, parse-only heuristics that trade a little accuracy for
orders-of-magnitude lower cost than coverage. Known gaps:
- Reflection / DI-resolved types referenced only via a string name or
container resolution won't be detected — the type's short name never appears
in the test source.
- Extension methods invoked as instance methods (C#): the declaring static
class is not named, so its file is not credited.
var, target-typed new(), pattern matching lose the type token; the
file-level union usually still catches it through other references.
- Short identifier names (polyglot, < 4 chars) are dropped to avoid noisy
pairings on names like
id, db, Tag.
- Monorepo path aliases (TS path mapping, Java module-info) are not
resolved; a suffix-match fallback may pick the wrong source if two files share
a trailing path segment.
For these cases, run actual coverage (coverage-analysis) on the unpaired
candidates the agent has already triaged.
Always label the final result as a static pairing heuristic, not evidence of
line or branch coverage. Include that caveat even when every requested source
file has an obvious matching or missing test.
Outputs the agent should consume
untested[*].source / untested_sources[*].path — pick the next source file
to test (highest declaration count first).
*.suggested_test_path — drop-in target for the new test file; the Roslyn
engine honors the test project that already <ProjectReference>s the source's
project, so dotnet sln add is not needed. The polyglot engine may suggest a
co-located test when no test root is discoverable. When a source sibling is
already paired, its test directory is the established convention and must be
reused for the missing sibling rather than falling back to source co-location.
source_to_tests (Roslyn) / --include-tested tested_sources (polyglot) —
verify a newly written test file lands in the list for the intended source.
orphan_tests (polyglot) — tests that don't reference any same-language
source file; useful for triaging stale or integration-only tests.
1---2name: find-untested-sources3description: >4 MANDATORY for static source-to-test pairing: find or list source files/modules5 without corresponding tests, or suggest test locations from repository6 structure. Invoke even for a tiny package; do not substitute manual globbing.7 Uses Roslyn for C#/.NET and tree-sitter for Python, TS/JS, Go, Java, Rust,8 Ruby, Kotlin, Swift, PowerShell, and C++. DO NOT USE FOR: real9 line/branch/Cobertura data, coverage-backed test priorities, CRAP risk, or10 grading existing tests.11license: MIT12metadata:13 source: https://skillmd.com/skills/managedcode/find-untested-sources14---1516# Find Untested Sources1718## Purpose1920Coverage tools answer "which lines were executed?" — they require a green build21and a passing test run, which is minutes-to-tens-of-minutes on a real repo. The22question this skill answers is different and much cheaper:2324> _Which source files have no test file referencing any of their declared25> types/symbols?_2627That's the question an agent asks **before** writing a new test — and it can be28answered statically in a few seconds by parsing source files, with **no build,29no dependency resolution, and no compilation**. The output is a deterministic30test-pairing map that lets the agent pick the next file to test without reading31the entire codebase first.3233## Two engines — pick one3435This skill ships two interchangeable analyzers with a compatible JSON contract:3637| Engine | Script | Use when |38|--------|--------|----------|39| **Roslyn (C#)** | `scripts/Find-UntestedSources.cs` | The repo is **.NET-only**. Parses every `.cs` file with the Roslyn syntax API and does strict **namespace disambiguation**, so it is materially more accurate on duplicated short names like `Settings` or `Context`. |40| **tree-sitter (polyglot)** | `scripts/find_untested_sources.py` | The repo is **not exclusively C#**, or you want one tool across C#, Python, TypeScript/JavaScript, Go, Java, Rust, Ruby, Kotlin, Swift, PowerShell, and C++. |4142For a .NET-only repository, **prefer the Roslyn engine** — its namespace-aware43pairing beats the polyglot engine's identifier overlap.4445## Required workflow46471. Use the narrowest repository or package root named by the caller. Do not scan48 a parent workspace when the request identifies a subdirectory.492. Execute the appropriate analyzer once. Do not replace analyzer execution with50 manual globbing, filename matching, or visual inspection.51 For polyglot analysis, pass `--include-tested` when the answer must distinguish52 paired sources from unpaired sources.53 "Static pairing only" prohibits compiling the target repository and running54 its tests; it does not prohibit launching this skill's parse-only analyzer.55 State that distinction briefly when the caller also says "do not build."56 Treat analyzer dependencies as environment prerequisites: do not install57 packages, try the wrong engine, build the repository, or fall back to a manual58 scan when an analyzer invocation fails. Report the prerequisite failure instead.593. Base the result on the analyzer's JSON. Preserve its paired/unpaired60 classification and suggested relative path; do not guess a different path.614. When the caller named a subdirectory, prefix analyzer-relative paths with62 that subdirectory so reported paths are workspace-relative.635. Report the requested result plus the static-pairing coverage caveat. Do not64 append build, package-install, test-run, or coverage commands. When paired65 sources exist, name their covering test files so the unpaired classification66 is auditable.6768## When to Use6970- User asks "where should I add tests based on source pairing?", "which files71 have no tests?", "find unpaired source files", or "give me a static test gap72 list".73- Before invoking a test-generation agent, to produce a source-pairing worklist.74- After generating tests, to verify each new test file pairs to a source file.75- To enumerate "weakly paired" source files (only one referring test) for76 follow-up depth checks.7778## When Not to Use7980- **Line/branch coverage** — use `coverage-analysis`.81- **Priorities derived from real coverage data** — use `coverage-analysis`.82- **CRAP-score / risk hotspots** — use `coverage-analysis`.83- **Are existing tests strong?** — use `test-gap-analysis` (mutation reasoning)84 or `assertion-quality`.8586## Roslyn engine (C#)8788### Prerequisites8990- .NET SDK that supports file-based apps (`dotnet run script.cs`). Pinned in the91 repo's `global.json` (SDK 11 preview or later).92- No internet access required beyond the initial NuGet restore of93 `Microsoft.CodeAnalysis.CSharp` on first run.9495### Usage9697```powershell98# From the skill folder99dotnet run scripts/Find-UntestedSources.cs -- <repo-root> [--top N]100101# Save the report102dotnet run scripts/Find-UntestedSources.cs -- <repo-root> > pairing.json103104# Iterate the untested list, highest-API-surface first105$report = Get-Content pairing.json | ConvertFrom-Json106$report.untested | Select-Object -First 10 source, decl_count, suggested_test_path107```108109Diagnostics go to stderr; JSON goes to stdout.110111### Output schema112113```jsonc114{115 "repo": "<absolute path>",116 "elapsed_ms": 8883,117 "counts": {118 "source_files": 3036,119 "test_files": 867,120 "untested_files": 1852,121 "paired_files": 1184122 },123 "untested": [124 {125 "source": "src/Foo/Bar.cs",126 "decl_count": 8, // # of type declarations in the file127 "suggested_test_path": // mirror of source under a discovered test project128 "tests/Foo.Tests/Bar/BarTests.cs"129 }130 ],131 "source_to_tests": {132 "src/Foo/Baz.cs": [133 "tests/Foo.Tests/BazTests.cs",134 "tests/Foo.IntegrationTests/Scenarios/BazScenarios.cs"135 ]136 }137}138```139140### How it works1411421. **File discovery** — recursive walk pruning `bin/`, `obj/`, `node_modules/`,143 `.git/`, `.vs/`, `packages/`, and any dotted subdir. Skips generated files144 (`.g.cs`, `.Designer.cs`, `.AssemblyInfo.cs`).1452. **Test vs source classification** — walks up to the nearest `.csproj` and146 marks it a test project if the project name ends in `.Tests`, `.Test`,147 `.UnitTests`, `.IntegrationTests`, `.E2E`, `.EndToEnd`, `.Spec`, `.Specs`, or148 the content references `Microsoft.NET.Test.Sdk`, `MSTest.Sdk`,149 `Microsoft.Testing.Platform`, `xunit`, `NUnit`, `TUnit`, or150 `<IsTestProject>true</IsTestProject>`.1513. **Source index (parallel)** — parse each source file with152 `CSharpSyntaxTree.ParseText` (syntax only, no compilation); record every153 `BaseTypeDeclarationSyntax` / `DelegateDeclarationSyntax` as154 `(ShortName, EnclosingNamespace, FilePath)`.1554. **Test scan (parallel)** — parse each test file, collect `using` directives +156 enclosing namespace, walk every `IdentifierToken`, look it up in the157 short-name index, and **disambiguate strictly**: an identifier is attributed158 only if the declaration's namespace matches one of the test file's `using`159 directives, the enclosing namespace, or a prefix of them. This avoids noise160 where common names like `Settings` or `Context` match every project.1615. **Pairing & suggestion** — invert into `source → [tests]`. Build a162 production-to-test project map from `<ProjectReference>` entries; for each163 untested source, mirror its in-project relative path under the referencing164 test project to suggest a path.1656. **JSON emit** — ordered by declaration count desc, then alphabetical.166167## Polyglot engine (tree-sitter)168169### Prerequisites170171- Python 3.10+.172- `pip install tree-sitter-language-pack` (single self-contained wheel that173 bundles parsers for 300+ languages and the high-level `process()` API). No174 native build, no per-language grammar install.175176### Usage177178```powershell179# From the skill folder180python scripts/find_untested_sources.py <repo-root>181182# Restrict to a language (repeatable)183python scripts/find_untested_sources.py <repo-root> --lang python --lang typescript184185# Truncate the report (top 20 by declared API surface)186python scripts/find_untested_sources.py <repo-root> --limit-untested 20 > pairing.json187188# Iterate, highest-API-surface first189$report = Get-Content pairing.json | ConvertFrom-Json190$report.untested_sources | Select-Object -First 10 path, declaration_count, suggested_test_path191```192193Pass `--include-tested` to additionally emit `tested_sources` (omitted by194default to keep the payload small for LLM consumption). Diagnostics go to195stderr; JSON goes to stdout.196197### Output schema198199```jsonc200{201 "repo_root": "<absolute path>",202 "summary": {203 "source_files": 3138,204 "test_files": 761,205 "tested_source_files": 1419,206 "untested_source_files": 1719,207 "orphan_test_files": 15,208 "languages": ["csharp"]209 },210 "untested_sources": [211 {212 "path": "src/Foo/Bar.cs",213 "language": "csharp",214 "declaration_count": 8,215 "declarations": ["Bar", "BarOptions", "IBar", "..."],216 "suggested_test_path": "src/Foo/BarTests.cs"217 }218 ],219 "orphan_tests": [220 { "path": "tests/SomeIntegrationTest.cs", "language": "csharp" }221 ]222}223```224225### How it works2262271. **File discovery** — recursive walk pruning common build/vendor dirs (`bin`,228 `obj`, `node_modules`, `target`, `dist`, `build`, `vendor`, `__pycache__`,229 `.venv`, `.git`, …) and generated files (`.d.ts`, `.g.cs`, `.Designer.cs`,230 `_pb2.py`, `*.min.js`, `AssemblyInfo.cs`, …).2312. **Language detection** — `detect_language_from_path` maps the extension to a232 supported language; unknown extensions are skipped.2333. **Test-vs-source classification** — per-language path heuristics:234235 | Language | Test rule |236 |---|---|237 | Python | path contains `tests/`/`test/`; or filename starts with `test_` or ends `_test.py`; or `conftest.py`. |238 | JS/TS/TSX | path contains `__tests__`, `tests`, `test`, `spec`, `e2e`; or filename contains `.test.`/`.spec.`. |239 | Go | filename ends `_test.go`. |240 | Java | path contains `test`/`tests`; or filename ends `Test.java`/`Tests.java`. |241 | Rust | path contains `tests/`/`benches/`. |242 | C# | path contains `tests/`; or project segment ends `.Tests`/`.Test`/`.UnitTests`/`.IntegrationTests`; or filename ends `Tests`/`Test`. |243 | Ruby | path contains `spec/`/`test/`; or filename ends `_spec.rb`/`_test.rb`. |244 | Kotlin | path contains `test/`/`tests/`/`spec/`; or filename ends `Test.kt`/`Tests.kt`/`Spec.kt`. |245 | Swift | path contains `test/`/`tests/`/`uitests/`/`integrationtests/` (case-insensitive); or filename ends `Test.swift`/`Tests.swift`. |246 | PowerShell | path contains `test/`/`tests/`/`pester/`; or filename ends `.Tests.ps1`/`.Test.ps1`. |247 | C++ | path contains `test/`/`tests/`/`testing/`; or filename starts `test_` or ends `_test.cpp`/`_tests.cpp`. |2482494. **Per-file extraction** — `process(text, ProcessConfig(structure, imports,250 symbols))` returns declared items, raw import statements, and a flat declared251 -name list.2525. **Pairing** — for each test file, union **import resolution** (per language,253 e.g. Python `from pkg.mod import x` → `pkg/mod.py`; Java `import a.b.C;` →254 `a/b/C.java`; C# `using` is namespace-not-file, so a no-op) with **identifier255 overlap** (word-like tokens, length ≥ 4, matched against declared names).2566. **JSON emit** — `untested_sources` ordered by declaration count descending.257258## Limitations (be honest with the agent)259260Both engines are static, parse-only heuristics that trade a little accuracy for261orders-of-magnitude lower cost than coverage. Known gaps:262263- **Reflection / DI-resolved types** referenced only via a string name or264 container resolution won't be detected — the type's short name never appears265 in the test source.266- **Extension methods** invoked as instance methods (C#): the declaring static267 class is not named, so its file is not credited.268- **`var`, target-typed `new()`, pattern matching** lose the type token; the269 file-level union usually still catches it through other references.270- **Short identifier names** (polyglot, < 4 chars) are dropped to avoid noisy271 pairings on names like `id`, `db`, `Tag`.272- **Monorepo path aliases** (TS path mapping, Java module-info) are not273 resolved; a suffix-match fallback may pick the wrong source if two files share274 a trailing path segment.275276For these cases, run actual coverage (`coverage-analysis`) on the unpaired277candidates the agent has already triaged.278279Always label the final result as a static pairing heuristic, not evidence of280line or branch coverage. Include that caveat even when every requested source281file has an obvious matching or missing test.282283## Outputs the agent should consume284285- `untested[*].source` / `untested_sources[*].path` — pick the next source file286 to test (highest declaration count first).287- `*.suggested_test_path` — drop-in target for the new test file; the Roslyn288 engine honors the test project that already `<ProjectReference>`s the source's289 project, so `dotnet sln add` is not needed. The polyglot engine may suggest a290 co-located test when no test root is discoverable. When a source sibling is291 already paired, its test directory is the established convention and must be292 reused for the missing sibling rather than falling back to source co-location.293- `source_to_tests` (Roslyn) / `--include-tested` `tested_sources` (polyglot) —294 verify a newly written test file lands in the list for the intended source.295- `orphan_tests` (polyglot) — tests that don't reference any same-language296 source file; useful for triaging stale or integration-only tests.