Dogfood

Dogfood Session

A dogfood session exercises hyalo against real knowledgebases to find bugs, verify fixes, assess UX, and discover improvement opportunities. Sessions should feel like a real user working with the tool — not a mechanical checklist. Be curious, follow interesting leads, try edge cases that occur to you naturally.

Phase 1: Prepare

1. Build and install from source, then prove the hyalo on PATH is this tree. The PATH binary is a cargo install and goes stale silently: on 2026-09-04 it was three days and eleven PRs behind target/release/hyalo, and a DEC-284 check against it nearly got filed as a bug.

   cargo build --release
   cargo install --path crates/hyalo-cli --target-dir target --force
   test "$(hyalo --version | awk '{print $3}' | tr -d '(' | sed 's/+dirty$//')" = "$(git rev-parse --short=12 HEAD)" \
     && echo "hyalo on PATH is HEAD" || { echo "STALE hyalo on PATH: $(which hyalo)"; exit 1; }
   

   --target-dir target reuses the release build so the install takes seconds; the version string is hyalo X.Y.Z (<12-char hash>[+dirty] <date>), and +dirty is fine — it only means uncommitted edits such as the report you are about to write. If the check fails, fix the PATH before doing anything else — every later command in this skill, and every explorer agent you spawn, must run the same binary. Never fall back to ./target/release/hyalo for some commands and hyalo for others in one session.

2. Find the last dogfood report to establish the baseline:

   hyalo find --tag dogfooding --sort date --reverse --limit 1 --format text
   

   Read it to learn: which version was tested, which bugs were found, what was verified.

3. Discover what changed since then: Use git log --oneline from that report's date to HEAD. Look for merged iteration branches (iter-N/...). For each, read the iteration file in the knowledgebase to understand what was implemented:

   hyalo find --tag iteration --property "date>=YYYY-MM-DD" --sort date --format text
   

   Then hyalo read <iteration-file> --format text for each to understand the features, bug fixes, and UX changes. These are your priority test targets.

4. Check prior dogfood reports for open bugs to re-verify:

   hyalo find --tag dogfooding --sort date --reverse --limit 5 --format text
   

   Read the most recent reports and collect any bugs marked as found (not yet verified fixed).

5. Pick target knowledgebases: Use a mix of:

   • Own KB (hyalo-knowledgebase/, ~250 files) — well-structured, has schemas/views
   • MDN Web Docs (../mdn/files/en-us/, ~14K files) — large, stress-tests performance
   • GitHub Docs (../docs/content/, ~3.5K files) — complex nested YAML frontmatter
   • VS Code Docs (../vscode/, ~1K files) — mid-size, different structure

   For external KBs, check they exist first (ls ../mdn/files/en-us/ > /dev/null 2>&1). Create a snapshot index for large vaults (500+ files).

Phase 2: Exercise the tool

This is the creative part. Don't follow a rigid script — think about what a real user would do with hyalo and try it. That said, make sure every session covers these categories:

New feature verification

Test the features you discovered in Phase 1 from the iteration files and git log. For each new feature or changed command, don't just check the happy path — try edge cases, wrong inputs, combinations with other flags. The iteration files describe what was implemented and often include acceptance criteria that suggest good test scenarios.

Bug regression testing

Re-verify bugs from the most recent dogfood report(s). Use the exact commands or scenarios that originally exposed them. Mark each as STILL FIXED, REGRESSED, or PARTIALLY FIXED.

Creative exploration

This is where you go beyond verification. Try things like:

• Unusual filter combinations (--property + --tag + BM25 + --section)
• Edge-case inputs (empty strings, special characters, very long queries)
• Commands on files with unusual frontmatter (nested objects, empty values, unicode)
• --jq queries to build ad-hoc reports or dashboards
• Views that chain with additional CLI flags
• mv on files with many inbound links
• lint --fix --dry-run on messy external KBs
• Schema validation on write (--validate or validate_on_write)
• Performance comparisons (indexed vs non-indexed, small vs large vault)
• links fix on KBs with broken links
• Anything that occurs to you as you go — follow your curiosity

UX assessment

Pay attention to:

• Are error messages helpful? Do they suggest what to do?
• Are hints useful and relevant?
• Is --format text output scannable?
• Do help texts match actual behavior?
• Any friction points or confusing behavior?

Performance spot-checks

Time a few representative commands on each KB used:

time hyalo find --limit 1                    # baseline parse speed
time hyalo find "some query"                 # BM25 search
time hyalo summary                           # full vault scan
time hyalo find --property status=completed  # structured filter

Compare with prior reports if available. Flag regressions > 2x.

Phase 3: Write the report

Save to hyalo-knowledgebase/dogfood-results/ with this naming and frontmatter:

---
title: "Dogfood vX.Y.Z — <descriptive subtitle>"
type: research
date: YYYY-MM-DD
status: active
tags: [dogfooding, ...]
related:
  - "[[dogfood-results/previous-report]]"
  - "[[iterations/relevant-iteration]]"
---

Report structure

Organize findings into clear sections. Use this general shape, but adapt as needed:

# Dogfood vX.Y.Z — <subtitle>

Brief context: binary version, which KBs tested, file counts.

## New Feature Verification
### Feature Name (iter-NNN) — WORKING / BROKEN / PARTIAL
Description of what was tested and results.

## Bug Regression Testing
### BUG-N: Description — STILL FIXED / REGRESSED
Commands tested, results observed.

## Bugs Found
### BUG-N: Description (SEVERITY)
Steps to reproduce, expected vs actual, impact.

## UX Issues
### UX-N: Description (SEVERITY)
What's confusing or could be better.

## What Worked Well
Highlight things that are genuinely good — performance wins, helpful errors, etc.

## Performance
Table of timings if measured.

Severity levels: HIGH (data loss, wrong results), MEDIUM (confusing behavior, workaround exists), LOW (cosmetic, minor friction).

After writing the report

Use hyalo set to ensure the frontmatter is correct. Add related links to relevant iterations or prior dogfood reports.

Tips for good dogfood sessions

• Follow the hints. Hyalo outputs drill-down suggestions — use them like a real user would.
• Don't skip external KBs. Many bugs only surface at scale or with unusual frontmatter. The own KB is too well-structured to catch everything.
• Note what's NOT there. Missing features, things you wished you could do — these are just as valuable as bugs.
• Be specific. Include exact commands, exact error messages, exact output. Vague reports like "search felt slow" aren't actionable.
• Test across KBs. A command that works on the own KB might fail on MDN due to scale or on GitHub Docs due to nested YAML.
• Check data quality. Use lint, find --broken-links, find --orphan to surface issues in the KB itself — this is valuable dogfooding too.