tomevault-io corpus-test
Run integration tests against the corpus of real-world OpenAPI specifications.
Usage:
/corpus-test— run all corpus tests/corpus-test short— exclude large specs (faster)
Step 1: Check Corpus Availability
Use the Glob tool to check for corpus files:
pattern: testdata/corpus/*.{json,yaml,yml}
If no corpus files found, download them:
make corpus-download
Step 2: Run Corpus Tests
Based on the mode argument:
Full mode (default):
make test-corpus
Short mode (argument = "short"):
make test-corpus-short
Capture and analyze the output.
Step 3: Analyze Results
Note: Corpus specs are real-world documents (Plaid, DigitalOcean, GitHub, etc.) that often contain validation errors intentionally. Validation failures are expected — the key metric is whether the fixer resolves them.
For each corpus spec, report:
- ✅ Parse/validate/fix succeeded
- ❌ Failures with error details (fixer failed to resolve validation errors)
- ⚠️ New warnings compared to expected behavior
Step 4: Report
## Corpus Test Results
| Spec | Parse | Validate | Fix | Notes |
|------|-------|----------|-----|-------|
| petstore.yaml | ✅ | ✅ | ✅ | |
| github-api.json | ✅ | ❌ | ⚠️ | [details] |
| ... | ... | ... | ... | ... |
### Summary
- **Passed**: N/M specs
- **Failed**: [list]
- **New issues**: [list]
Step 5: Offer Actions
If failures are found:
- Investigate — deep dive into specific failures
- Compare with main — check if failures are regressions
- File issues — create GitHub issues for new failures
Converted and distributed by TomeVault — claim your Tome and manage your conversions.