Glue Unit Test Bootstrap

Glue Unit Test Bootstrap

GlueCommands.Self/GlueState.Self aren't usable out of the box in a plain xunit host — Glue.exe's real startup (Program.cs/MainGlueWindow.cs) does several one-time registrations first. Any test that drives production code through those statics (not just calling a pure static method directly) needs:

GlueUnitTests.TestSupport.GlueTestBootstrap.EnsureInitialized();

Call it in the test's constructor. It's idempotent (safe to call every test, cheap after the first call). Skipping it is what produces the NRE chain: DI container not built, legacy EditorObjects.IoC.Container locator not registered, ProjectManager.CodeProjectHelper null, FileWatchManager not initialized.

None of this is faked — GlueTestBootstrap runs the same production init calls Glue.exe makes, just outside a live WinForms app. See its doc comment at FRBDK/Glue/Tests/GlueUnitTests/TestSupport/GlueTestBootstrap.cs for exactly what it registers.

RegisterPluginForTesting adds a plugin to both mPluginContainers and ImportedPlugins. Those back different dispatch paths: CallPluginMethod walks the containers, but every ReactTo* event enumerates ImportedPlugins. A plugin in only the first answers direct calls and is never notified of anything — which reads as "the plugin ran and chose to do nothing," not as a wiring bug.

Sibling seams

Also process-wide static state a test may need to control alongside the bootstrap:

• TaskManager.SynchronousMode — set true to run TaskManager.Self.Add/AddAsync inline instead of on the background thread. See GlueUnitTests/Tasks/TaskManagerSynchronousModeTests.cs.

• TaskManager.UiThreadMarshaller — swap in an inline IUiThreadMarshaller to avoid needing a real WinForms message loop for DoOnUiThread/OnUiThread calls.

• TestVisualStudioProjectFactory (GlueUnitTests/TestSupport/TestVisualStudioProjectFactory.cs) — builds a real (not fake) VisualStudioProject from a minimal non-SDK .csproj, for tests that need GlueState.CurrentMainProject to be non-null.

• ObjectFinder.Self.GlueProject — assign a GlueProjectSave for anything reaching ObjectFinder lookups (GetEntitySaveUnqualified, GetAllReferencedFiles).

• GlueState.Self.Current* setters work headless. Selection is by model object, so CurrentNamedObjectSave = nos takes effect as long as nos is in ObjectFinder.Self.GlueProject (an object in no element reads as nothing selected). It really dispatches ReactToItemsSelected to registered plugins: set PluginManager.HandleExceptions = false in such a test, or a throwing handler is swallowed and the plugin silently disabled for the rest of the run. A plugin that builds its WPF view on selection must gate the view on GlueGui.ShowGui (off in the bootstrap); see MainCollisionPlugin.TryHandleSelectedCollidable.

Every one of these is process-wide, which is why the whole assembly runs non-parallel — see GlueUnitTests/AssemblyInfo.cs. So cross-class interleaving isn't a hazard, but leakage still is: a test which assigns one of these must restore it (IDisposable/try-finally), or it silently changes the setup of whatever runs next. The failure mode is a test that passes under --filter and fails in the full run.

Loading a whole real project (gold projects)

GoldProject/GoldProjectCompileTests (GlueUnitTests/TestSupport, GlueUnitTests/Projects) drive a checked-in sample through the real ProjectLoader.LoadProject, regenerate, and build. That is the only way to cover generators that ask another plugin something at generation time. Needs GlueTestBootstrap.EnsureGameProjectPluginsRegistered() and [StaFact]. Two non-obvious requirements:

• Delete *.Generated.cs first. Glue doesn't rewrite a file whose content is unchanged, so otherwise "codegen produced this" and "codegen never ran" are indistinguishable.
• *.Generated.cs is gitignored repo-wide, and the samples are checked in without it. A clean git diff after regenerating proves nothing, and no sample builds until Glue regenerates it. Assert nothing about these files existing beforehand — your working copy has them from earlier runs and a fresh CI clone does not. Before pushing, find Samples -name "*.Generated.cs" -delete and re-run, or CI is the first thing to see the real starting state.

Landmine — a WinForms sync context plus SynchronousMode deadlocks the run with no timeout

WinForms installs a WindowsFormsSynchronizationContext on a thread the moment the first control is created there — FakeMainGlueWindow's PropertyGrid, a MenuStrip, a plugin toolbar. Continuations then post back to it and only run while a message loop pumps, which no test host does. Combined with TaskManager.SynchronousMode (whose RunSynchronously blocks the caller on the awaited task), the first plugin that awaits during a load hangs forever: one blocked thread, no child process, nothing executing, and no timeout anywhere. GlueTestBootstrap sets WindowsFormsSynchronizationContext.AutoInstall = false; don't undo it, and don't diagnose this shape as a slow build. A stack dump (dotnet-stack report -p <pid>, real Windows PID) showing exactly one blocked thread and nothing else running is the signature.

Landmine — do NOT give PluginManager a MenuStrip

GlueGui.Initialize(menuStrip) is needed (PluginBase.AddMenuItemTo reads GlueGui.MenuStrip.Items). PluginManager.ShareMenuStripReference is the opposite: PluginCommand marshals every plugin call through mMenuStrip.Invoke whenever mMenuStrip is non-null, so with no message loop the calls silently never run and CallPluginMethod returns null. Leaving PluginManager's null keeps its "no live menu strip means run inline" guard doing the right thing.

Landmine — code generation swallows its own exceptions

GenerateAllCodeSync calls the async Task CodeWriter.GenerateCode without awaiting it, so an exception during one element's generation is captured into a discarded Task and lost. The symptom is an empty .Generated.cs (the placeholder from CreateGeneratedFileIfNecessary) for that one element, no error anywhere, and every later step for it — its factory, its project entry — silently skipped. To see the real exception, await CodeWriter.GenerateCode(element) directly. ErrorRecordingPlugin subscribes to Glue's error and output channels so a partial regeneration fails a test rather than passing quietly.

Landmine — MSBUILD_EXE_PATH must be set before the first MSBuild evaluation in the process

Microsoft.Build caches its toolset on first use, so setting the variable later has no effect. A test that evaluates a bare non-SDK project first (TestVisualStudioProjectFactory) fixes the toolset for the whole run, and a later real-project load then fails with The SDK 'Microsoft.NET.SDK.WorkloadAutoImportPropsLocator' specified could not be found — while passing when run alone. Call GlueTestBootstrap.EnsureMsBuildEnvironmentVariable() before anything touches MSBuild.

Landmine — calling a real plugin method for the first time can pop a real dialog on the developer's desktop

IMainGlueWindow/IUiThreadMarshaller only cover calls that were already routed through MainGlueWindow.Self/TaskManager. A plugin method you're calling directly for the first time (bypassing PluginManager.CallPluginMethod's silent no-op to get real coverage — see REFACTORING.md's Collision/Gum entries) may still contain its own unseamed MessageBox.Show(...) on some branch (e.g. an "already exists, overwrite?" check). That call blocks the test thread on a real, visible modal dialog on the developer's actual desktop — not something GlueTestBootstrap catches, and not obvious from reading the test in isolation.

Before exercising a plugin method's real logic for the first time: skim it (and what it calls) for MessageBox.Show/similar dialog calls, and design the test to never take that branch (fresh unique temp directory per test, askToOverwrite/equivalent flags set to avoid the prompt, call the method at most once per test run rather than twice to probe an idempotency branch). If you ever see an unexpected pause or the user reports a popup, stop immediately, confirm no process is still blocked waiting on it, and fix the test to avoid the branch rather than building a dialog seam just to unblock one test.

Landmine — a warm obj/bin can report a stale test as green, or a real compile error as passing

dotnet test after an edit does an incremental build: MSBuild decides per-file whether to recompile, and in practice a git stash/git rebase/repeated edit-and-rerun cycle can leave it not recompiling a file you just changed — the run reports the previous binary's result, not the current source's. This produced a green "317/317" locally on issue #2016's PR while the actual pushed source had two CS0103 compile errors (missing usings) that only CI's from-scratch build caught, plus masked a real regression (a broadened FakeFindManager behavior that broke two unrelated tests) for the same reason.

Before reporting "full suite green" as a claim someone will act on (a PR description, a "tests pass" verdict, this file), rebuild from scratch at least once: rm -rf FRBDK/Glue/Tests/GlueUnitTests/obj FRBDK/Glue/Tests/GlueUnitTests/bin then dotnet test without --no-build. --no-build/--no-restore reruns are fine for fast iteration between clean checkpoints, not as the last thing before trusting the result.

Landmine — dotnet test on the bare csproj fails with MSB3073/*Undefined* paths

Running dotnet test FRBDK/Glue/Tests/GlueUnitTests/GlueUnitTests.csproj directly fails building OfficialPlugins.csproj's PostBuild target: it copies its output using $(SolutionDir)Glue\bin\Debug\Plugins\..., and SolutionDir is a solution-scoped MSBuild property that's simply undefined when you build/test a project file directly (no .sln in the invocation) — same failure would hit any of the plugin projects with a similar post-build copy step, not just OfficialPlugins.

Fix: pass SolutionDir explicitly, pointed at FRBDK/Glue/ (the directory containing Glue with All.sln, since the copy path is $(SolutionDir)Glue\...), with a trailing backslash:

dotnet test FRBDK/Glue/Tests/GlueUnitTests/GlueUnitTests.csproj -p:SolutionDir="<repo>\FRBDK\Glue\\"

What a healthy run costs — anything far past this is a hang, not slowness

Rough orders of magnitude on a warm dev machine, so "is it stuck or just slow?" is answerable without guessing. All with --no-build after a successful build:

BuildSmoke is slow by nature - every test copies a sample project to a temp directory and loads it through a real headless editor, and AssemblyInfo.cs disables parallelization assembly-wide because Glue's singletons are shared. A dev machine beats these numbers, so treat them as the ceiling rather than a target.

Wall clock alone will not tell you a run is wedged. The signature is the child dotnet build having already exited while testhost is still alive - that is the pipe hang below, not work in progress.

Landmine — every nested dotnet call must go through NestedDotnetCli

BuildSmoke tests shell out to real dotnet build/dotnet run. The obvious way to write that — redirect both streams, StandardOutput.ReadToEnd(), WaitForExit() — hangs for 10–15 minutes instead of failing, because dotnet build starts MSBuild worker nodes with /nodeReuse:true that outlive it while holding the inherited stdout pipe open, so ReadToEnd() never sees EOF. It is intermittent by nature: only the run that starts the nodes inherits the handles, so the same test hangs once and passes on an immediate retry.

GlueUnitTests/TestSupport/NestedDotnetCli.cs is the only sanctioned way to spawn one — it disables the persistent build servers, pumps both streams concurrently, and kills the entire child tree on timeout. NestedDotnetCliTests fails the build if a redirected Process.Start reappears anywhere in the assembly. See GitHub issue #1969.

A timeout passed to Run covers cold start too — the CLI, MSBuild evaluation and any process the build itself spawns can eat tens of seconds on a CI runner before the command does the thing being measured. A budget sized against a warm dev machine is therefore mostly startup on a cold one, and expires before the test's premise is even true. When the budget is meant to bound something after that warm-up, pass a StartupGate (see NestedDotnetCli.Run): it waits untimed until the gate opens and only then starts the clock. See GitHub issue #1992.

Landmine — Shouldly's string ShouldContain ignores case

someString.ShouldContain("Expected") passes against "expected"; Shouldly defaults string containment to Case.Insensitive. A test asserting that a path or generated line keeps its capitalization must pass Case.Sensitive, or it stays green while the code lower-cases.

Standing rule — clear stragglers after an interrupted run

A dotnet test/dotnet build that is cancelled or killed mid-flight leaves its tree (dotnet, MSBuild nodes, VBCSCompiler, testhost) behind; a run that completes normally now cleans up after itself. So this is a response to an interruption, not a ritual before every command:

Get-Process testhost,vstest.console,MSBuild,VBCSCompiler -ErrorAction SilentlyContinue | Stop-Process -Force

Never end a turn (report status, stop) while a dotnet test/dotnet build is still running detached in the background — wait for it synchronously, or explicitly confirm it finished/kill it first. A background run left alive across a stop/resume cycle is how the count climbs into the dozens over a long session.

Landmine — an orphaned testhost locks the output DLLs, and the next run looks like a hang

A cancelled or timed-out dotnet test can leave testhost.exe alive holding GlueUnitTests/bin/.../*.dll open. The next run then can't copy dependencies into bin, and fails with MSB3021/MSB3027 naming the holder (The file is locked by: "testhost (PID)"). The trap is that the build stalls through its 10 retries × 1s per file while producing no test output — piped through a grep, that buffers to nothing and reads as a hung test suite rather than a locked file.

Before diagnosing a slow or silent run as a test problem, clear the holders:

Get-Process testhost,vstest.console,MSBuild -ErrorAction SilentlyContinue | Stop-Process -Force

Then re-run with --no-build when nothing was recompiled since the last successful build — it skips the copy step the lock blocks, and the full non-BuildSmoke suite finishes in seconds.

Landmine — a broad FullyQualifiedName~ filter silently sweeps in Category=BuildSmoke tests and thrashes the machine

GumRuntimeMemberContractTests, GumGeneratedCodeCompilesTests, and the *CreationSmokeTests classes are tagged [Trait("Category", "BuildSmoke")] because they shell out to real dotnet build/dotnet test child processes (building the whole engine or a scaffolded project) instead of just exercising codegen in-memory. pr-tests.yml deliberately runs them as their own step, Category=LiveGame as another, and everything else with --filter "Category!=BuildSmoke&Category!=LiveGame".

Category=LiveGame behaves the same way and costs more: each of those tests builds and launches a real game process (see glue-live-game-testing).

A filter like --filter "FullyQualifiedName~GumPlugin" does not know about that split — it matches BuildSmoke tests in the same namespace right along with the fast ones. Run several of those together and each spawns its own nested dotnet/testhost/VBCSCompiler tree; a handful of them running serially is enough to make the machine crawl and a run that should take seconds look hung for many minutes.

Always AND in Category!=BuildSmoke&Category!=LiveGame unless you specifically intend to run one of the slow suites:

dotnet test FRBDK/Glue/Tests/GlueUnitTests/GlueUnitTests.csproj -p:SolutionDir="<repo>\FRBDK\Glue\\" --filter "FullyQualifiedName~GumPlugin&Category!=BuildSmoke&Category!=LiveGame"

Deep dive

For the full story of why each seam was needed and what's still NRE-prone (e.g. AvailableAssetTypes.CommonAtis requiring real PluginManager plugin loading), see REFACTORING.md's "Unblock WizardProjectLogic AddGameScreen testing" and "Wizard apply-engine test seams" entries.