Mastraudit

Mastraudit

Audit a Mastra implementation in the order things actually go wrong.

That ordering is the whole point, and it is a correction. An audit that leads with architecture catches a stray @mastra/* import instantly and misses the incident that costs the most hours. Package boundaries are cheap to fix and rarely fatal. Execution semantics - what a step does, how fan-out results are keyed, whether a load-bearing write throws - are where runs die, and they are invisible to a structural pass.

So: execution first, structure second. If you run out of time, you will have spent it on the half that matters.

Source of truth

Mastra moves fast enough that recalled API shapes are wrong more often than right. In priority order:

1. The installed packages. node_modules/@mastra/* types and embedded docs. This is what will actually run.
2. The documentation for that version. A local mirror if one exists, the published docs otherwise.
3. Nothing else. Never model memory for constructor signatures, model routing, storage, memory, workflow, or tool APIs.

Note the installed version and resolve any disagreement in favour of what is installed. Where a codebase ships its own Mastra conventions - a house package, an architecture document, a failure log - read it first; it outranks generic guidance about that codebase.

The five-minute pass

When there is no time for the full audit, these catch the most:

1. modelSettings - is every token cap nested under it, never flat at the top level?
2. Step size - does any workflow step do more than one discrete thing?
3. Fan-out - do arms return receipts rather than bulk, and does the collector key on identity rather than array position?
4. Load-bearing writes - does a write a later step depends on throw on failure?
5. Tool keys - does the model see verb_noun, or a leaked camelCase shorthand?

Use the scanners before you grep

Some projects already provide deterministic checks for import boundaries, tool IDs and annotations, barrel files, model-setting nesting, tool keys or MCP client identity. Discover applicable scanners in the project's installed packages and scripts. Verify their documented coverage instead of assuming a particular package or export exists.

Inspect the installed scanner signatures and bundled usage examples, then call the applicable exports against the scoped implementation root. Do not invent arguments from the export names. If a suitable scanner is unavailable, use manual search and state the coverage gap. Scanner output is a lead: verify it against the implementation and supported contracts.

The check catalogue

• Execution semantics - workflows, fan-out, concurrency, retries, suspend and resume, state and storage, long agent loops. Where the fatal failures live.
• Structure - containment, orchestrator discipline, agents and models, tools.
• Evidence - observability, testing, and what counts as having verified something.

Steps

Use these steps to organize the audit; keep any task list brief and report skipped coverage without copying the workflow verbatim.

Default to source and existing evidence. Audit-only scope does not authorize workflow runs, model calls, storage writes or deployment changes. Local scanner execution is allowed after inspecting it for side effects. Apply house architecture and naming conventions only when the target codebase adopts them; otherwise present them as optional recommendations, not correctness findings.

1. Scope it and say it back. Which package owns Mastra, and which surfaces are in range. Trace runtime owners and import roles; zero or several dependency declarations are discovery signals, not findings. Several independently deployed Mastra apps can be legitimate. Audit clearly identified implementations separately and ask only if unresolved ownership changes the scope.

2. Establish the source of truth. Note installed versions. Find any codebase-local conventions document. Say which you are auditing against.

3. Run the scanners if they are available, and record what each returned including the empty ones. An unrun scanner is not a pass.

4. Audit execution semantics against references/execution.md. This is the longest step and it comes first on purpose.

5. Audit structure against references/structure.md.

6. Audit evidence against references/evidence.md.

7. Report to the contract below.

Output contract

Findings ranked by what a failure costs, not by section order.

Each finding carries: file:line, the failure it invites in one sentence, and the fix. Where a finding matches a known incident class in the codebase's own failure log, cite it - a named prior incident is far more persuasive than a rule.

• Blocking. Would lose a run, corrupt state, or silently produce wrong output.
• Should fix. Real, not yet fatal.
• Noted. Judgement calls the codebase may have made deliberately. Ask rather than assert.
• Not checked. Anything skipped, and why. A short audit honestly scoped beats a long one implying coverage it did not have.

Failure modes

• Leading with architecture. Containment findings are easy to produce and rarely the expensive problem. They go second.
• Asserting an API shape from memory. Read the installed types. This is the single most common way an audit is confidently wrong.
• Over-strict containment. Blanket "no @mastra/* outside the owner" flags legitimate infra clients. See references/structure.md.
• Flagging a pattern whose exception is documented. onStepFinish with structuredOutput is only a bug without structuredOutput.model. Check the narrow form before flagging.
• Reporting a clean pass on checks you did not run. Say what you did not check.