Validate Project Template
Comprehensive validation checklist for project templates. Run each check against the template directory and report pass/fail.
Step 1: Determine Template Level
Read stack.json from the template root and classify:
cat stack.json | python3 -c "import sys,json; d=json.load(sys.stdin); print(f'Level: {d[\"level\"]}, Stack: {d.get(\"stack\",\"none\")}, Parent: {d.get(\"parent\",\"none\")}')"
| Level | Indicator |
|---|---|
| 0 | level: 0, parent: null, stack: null |
| 1 | level: 1, parent: "project-template", stack filled |
| 1.5 | level: 1.5, parent is a Level 1 template |
| 2 | level: 2, parent is a Level 1 template |
If stack.json is missing or invalid, flag as critical error and stop.
Step 2: Validate stack.json
- Valid JSON (parseable)
-
stackfield present (nullat L0, non-empty string at L1+) -
versionpresent and in semver format (X.Y.Z) -
levelis 0, 1, 1.5, or 2 -
parentisnullat L0, non-empty string at L1+ -
layersobject withdata,logic,interfacekeys (all arrays) -
layersarrays empty at L0, at least one non-empty at L1+ -
pluginsobject withtechnology,process,stackkeys (all arrays) -
plugins.technologynon-empty at L1+ (at least one tech plugin)
Severity of the layout checks
Checks marked (WARN) are layout upgrades that came with plugin 2.2.0 (one AGENTS.md, committed .mcp.json, committed .claude/settings.json, workflow skills). A template built on the older layout gets WARN with the migration hint, never FAIL. Every unmarked check FAILs, as before. Two checks are new FAILs, both CRITICAL: a literal secret in the tracked .mcp.json, and env values or tokens in the committed .claude/settings.json. The README section "Upgrading from 2.1.x" lists the migration steps.
Step 3: Check Required Files — All Levels
Root Files
-
stack.jsonexists -
AGENTS.mdexists (shared rules for every coding tool) -
CLAUDE.mdexists - (WARN)
CLAUDE.mdstarts with@AGENTS.mdor is a symlink toAGENTS.md. Without the import Claude reads onlyCLAUDE.md, so a generated or hand-copiedAGENTS.mdis a second copy of the rules -
README.mdexists -
.env.exampleexists -
.mcp.jsonor the legacy.mcp.json.exampleexists (emptymcpServersis fine at Level 0) - (WARN)
.mcp.jsonis tracked by git and.gitignoredoes not exclude it. Legacy layout:.mcp.json.examplecommitted plus a gitignored.mcp.json -
.gitignoreexists -
.gitignoreexcludes.env,.env.local,.claude/settings.local.json,node_modules
Also (WARN): AGENTS.md generated from CLAUDE.md by sync-context.sh — recommend the one-source layout.
.claude/ Directory
- (WARN)
.claude/settings.jsonexists withextraKnownMarketplacesandenabledPluginsobjects -
.claude/settings.local.json.exampleexists
Core Skills (inherited from Level 0)
-
.claude/skills/brainstorming/SKILL.mdexists -
.claude/skills/planning/SKILL.mdexists -
.claude/skills/tdd/SKILL.mdexists -
.claude/skills/debugging/SKILL.mdexists -
.claude/skills/verification/SKILL.mdexists -
.claude/skills/project-config/SKILL.mdexists
Core Workflow Skills
-
.claude/skills/init-stack/SKILL.mdexists -
.claude/skills/commit/SKILL.mdexists -
.claude/skills/pr/SKILL.mdexists -
.claude/skills/plan-feature/SKILL.mdexists -
.claude/skills/code-review-project/SKILL.mdexists -
.claude/skills/sync/SKILL.mdexists
Legacy layout (WARN — recommend migrating): the same workflows as .claude/commands/<name>.md. A command named review never runs (the built-in /review alias of /code-review takes the name) and plan collides with the built-in /plan — flag both and suggest code-review-project and plan-feature.
Core Agent
-
.claude/agents/code-reviewer.mdexists
Core Rules
-
.claude/rules/safety.mdexists -
.claude/rules/search-before-building.mdexists -
.claude/rules/project-conventions.mdexists
Documentation
-
docs/architecture.mdexists -
docs/code-style.mdexists -
docs/api-conventions.mdexists
Step 4: Check Level-Specific Requirements
Level 1+ Additional Checks
- At least one stack-specific skill beyond the core set (e.g.,
new-page,new-component) - (WARN)
.claude/settings.jsonenabledPluginscontains every public plugin fromstack.jsonplugins(technology, process, stack) as<name>@agents-store-claude-plugins. A plugin is public when$PLUGINS_PUBLIC_SOURCE_DIR/<name>exists or the public marketplace lists it; if neither source is available, skip this check and say so. Private plugins are not expected here — and a committed file must never name a private marketplace - (WARN) Every plugin named in
stack.jsonexists in the public marketplace or, when$PLUGINS_PRIVATE_SOURCE_DIRis set, in the private source; stack plugins are namedstack-{name}(no process suffix) -
.env.examplehas stack-specific variables uncommented -
CLAUDE.mdhas filled Tech Stack section (no[e.g.,placeholders) -
CLAUDE.mdhas filled Installed Plugins section
Level 1.5 Additional Checks
- Has application source code (
src/or equivalent) - Has sample data or seed scripts
- Has deployment config (Dockerfile, docker-compose.yml, or equivalent)
Level 2 Additional Checks
-
project-config/SKILL.mdhas actual resource IDs (no empty tables) -
.envor.env.localexists locally (warn if missing, but don't fail — it's gitignored) - Every
${VAR}referenced in.mcp.jsonhas a value locally, in.envor theenvblock of.claude/settings.local.json(warn if missing — both are gitignored)
Step 5: Check CLAUDE.md Quality
The sections below may live in CLAUDE.md or in the AGENTS.md it imports — check the combined content.
- Total line count under 100, counting the imported
AGENTS.md(the 100-line limit is this template system's own rule; Anthropic's guidance is under 200 — WARN between 100 and 200, FAIL above 200) - Has
## Tech Stacksection - Has
## Architecturesection (pointing todocs/architecture.mdby plain path) - Has
## Installed Pluginssection - Has
## Quick Commandssection - Has
## Critical Rulessection - No placeholder text remaining: grep for
\[e\.g\.,,\[Project Name\],TODO,TBD,fill in,<!-- .*-->with empty content around it - At L1+: Tech Stack lists actual technologies (not
[e.g., NocoDB, Supabase, Directus]) - At L1+: Installed Plugins lists actual plugins with descriptions
- (WARN) Long reference content is linked by plain path or lives in
.claude/rules/*.mdwithpaths:. An@docs/...import is not a way to shrink CLAUDE.md: imported files load at launch and count toward the line budget
Step 6: Check Consistency
- Technologies in
stack.jsonlayersmatch CLAUDE.md Tech Stack section - Plugins in
stack.jsonpluginsmatch CLAUDE.md Installed Plugins section - Environment variables in
.env.examplecover every${VAR}that.mcp.jsonreferences - Workflows listed in CLAUDE.md Quick Commands exist as
.claude/skills/*/SKILL.md(legacy:.claude/commands/*.md) - Skills referenced in CLAUDE.md exist as
.claude/skills/*/SKILL.mddirectories - (WARN)
CLAUDE.mdimportsAGENTS.md(or is a symlink to it); there is no second copy of the rules to keep in sync. IfAGENTS.mdis generated fromCLAUDE.md, recommend the one-source layout - (WARN)
.claude/settings.jsonenabledPluginsmatches the public plugins ofstack.jsonplugins(private plugins excluded)
Step 7: Check Security
- No
.envfile committed (checkgit statusand.gitignore) - The tracked
.mcp.jsonholds only${VAR}references — no literal token, key or deployment host (see the check below; a literal secret is a CRITICAL FAIL) - No hardcoded API keys or tokens in any tracked file: grep for patterns like
sk-,Bearer,token: ",key: "with actual-looking values - No real service URLs in
.env.example(only placeholders) -
.claude/settings.json(committed) has noenvvalues or tokens — onlyenabledPluginsandextraKnownMarketplaces(CRITICAL FAIL otherwise) -
.claude/settings.local.jsonis gitignored - No
.env.localcommitted
Committed .mcp.json Holds Only References
.mcp.json is meant to be committed, so validate its content, not its presence. Both commands must print nothing:
# a credential-looking key whose value is not a ${VAR} reference
grep -niE '"[A-Za-z_-]*(token|key|secret|password|authorization)[A-Za-z_-]*"\s*:\s*"[^"]{8,}"' .mcp.json | grep -v '\${'
# "Bearer" followed by anything other than a ${VAR}
grep -nE 'Bearer +[^$ "]' .mcp.json
The first grep assumes a pretty-printed file (one key per line) and drops every line that contains ${; for a minified file run jq . .mcp.json | grep ... instead.
Also read every url, command, args, headers and env value: anything that differs per deployment (a host, port, path, account id) must be ${VAR} or ${VAR:-default}. A literal URL is acceptable only for a published product endpoint that is the same for every user. A literal secret is CRITICAL — rotate it at the source; editing the file does not remove it from git history.
Git-Tracked Secrets (Critical)
The .gitignore check above only verifies RULES — it does not catch files that were committed BEFORE the gitignore rule was added. Run git ls-files to find actually tracked sensitive files:
# Check for tracked secrets that should be gitignored
git ls-files | grep -E '\.env\.local$|\.env$|settings\.local\.json$'
-
git ls-filesreturns NO matches for.env,.env.local, orsettings.local.json(.mcp.jsonis expected to be tracked) - If matches found: flag as CRITICAL — these files contain real credentials and are being tracked by git. Fix:
git rm --cached <file>and rotate all exposed tokens - Check
.claude/settings.local.json.examplefor real URLs or tokens (should contain only placeholders likehttps://your-instance.example.com)
Output Format
Present validation results as:
## Template Validation Report: {template-name}
**Level:** {0 / 1 / 1.5 / 2}
**Stack:** {stack name or "universal base"}
**Parent:** {parent name or "none"}
### Results
| Category | Status | Details |
|----------|--------|---------|
| stack.json | PASS / FAIL | {details} |
| Required Files | PASS / WARN / FAIL | {missing files count, legacy layout} |
| Core Skills | PASS / FAIL | {missing skills} |
| Core Workflow Skills | PASS / WARN / FAIL | {missing skills, legacy commands} |
| Level-Specific | PASS / WARN / FAIL / N/A | {details} |
| CLAUDE.md Quality | PASS / WARN / FAIL | {details} |
| Consistency | PASS / WARN / FAIL | {mismatches} |
| Security | PASS / FAIL | {issues} |
### Issues Found
1. {issue description and how to fix}
2. ...
### Overall: VALID / {N} issues to fix