Hyalo

Hyalo CLI — Prime Tool for Markdown Knowledgebases in pi

Ranked queries return score and up to three body matches with line, section, and text; text output displays that context directly. Snippets rank by distinct query tokens then line number and share stemming, OR and CJK rules with scoring. Quoted phrases must fit on one line. --section scopes snippets; frontmatter is excluded, so title-only hits can have an empty array. Indexed queries read snippet text only for final results after --limit.

Hyalo is a fast CLI for querying and mutating YAML frontmatter, tags, tasks, and structure in directories of markdown files. If the hyalo pi extension is installed (.pi/extensions/hyalo.ts), prefer its typed tools for the common operations — they take structured parameters, so there is no flag spelling or quoting to get wrong:

Use the generic hyalo tool (subcommand + args) only for operations the typed tools don't cover: summary, lint, mv, links, views, types, backlinks, --jq filters, bulk mutations (--glob, --where-property), etc.

--glob is how you address sequence-keyed documents (iterations, decisions, ...): the number may be zero-padded and the file archived in a subdirectory, so prefer the recursive form — find --glob '**/iteration-02-*.md' reaches both iterations/iteration-2-*.md and iterations/done/iteration-02-links.md. (--file is exact; --glob is the only globbing flag.)

Its killer features are combined filtering (e.g. hyalo find -e "regex" --property status!=done --tag feature) which you can't easily replicate with read/edit/grep/write, and bulk mutations (hyalo set --where-property) that replace multiple read + edit calls.

For pi sessions, ALWAYS use --format text for compact, LLM-friendly output.

Read the CLI's own help before guessing a flag

hyalo -h lists every command grouped by intent (read / write / config), one line each, naming the capability families behind them. hyalo <cmd> -h is one screen for one command; hyalo <cmd> --help is the full syntax reference — property operators, sort keys, --fields values, output shapes, and a cookbook. Both are generated from the binary you are running, so unlike any copy in this file they cannot go stale. Reach for -h first and --help for detail; do not fall back to grep because a filter looked unavailable.

hyalo -h                 # every command, grouped, with composed examples
hyalo find -h            # filters and output flags on one screen
hyalo find --help        # every operator, sort key, field name and recipe

An empty result set is also self-documenting: find that matches nothing echoes the filters it applied and hints at the next step (a did-you-mean over the values the property actually has, and the same query with its most selective filter dropped).

Core Philosophy for pi

• Use hyalo first: Before using read/edit/grep/write on .md files, check if hyalo can do it
• Read -h before guessing: hyalo <cmd> -h for the short page, --help for full syntax
• Batch operations: Use hyalo's bulk mutation features instead of individual edits
• Snapshot indexes: For vaults >500 files, use hyalo create-index + --index for speed
• Follow hints: hyalo outputs drill-down suggestions (-> hyalo ...) — use them
• Schema validation: Use hyalo lint --strict to catch frontmatter issues early

Quick Start in pi

# 1. Check hyalo is installed and configured
bash: which hyalo
bash: hyalo --version

# 2. Get overview of knowledgebase
bash: hyalo summary --format text

# 3. Search for files with BM25 full-text search
bash: hyalo find "iteration" --property status=planned --tag iteration --format text

# 4. Read a specific file's content or section
bash: hyalo read iterations/iteration-66-spec-refresh-drift-fixes.md --section "Scope" --format text

# 5. Update frontmatter properties
bash: hyalo set iterations/iteration-66-spec-refresh-drift-fixes.md --property status=in-progress --format text

BM25 Full-Text Search

The positional argument to find triggers BM25 ranked full-text search with automatic stemming ("running" matches "run", "runner", etc.). Results sorted by relevance score.

hyalo find "rust"                        # single term, stemmed
hyalo find "rust programming"            # AND: both terms required (implicit)
hyalo find "rust OR golang"              # OR: either term matches
hyalo find "rust -java"                  # NOT: exclude documents with "java"
hyalo find '\"error handling\"'          # Phrase: exact consecutive match (after stemming)
hyalo find "rust OR golang -obsolete"    # Mixed: either rust or golang, not obsolete

For literal pattern matching (not stemmed), use regex: hyalo find -e "exact_string".

Property & Tag Filtering

Filters combine freely — content search + property conditions + tag + section + task status in a single call:

hyalo find "error handling" --property status!=completed --tag iteration --section "Tasks" --task todo --format text

Property filters support: K=V (eq), K!=V (neq), K>=V/K<=V/K>V/K<V (comparison), K (existence), !K (absence), K~=pattern or K~=/pattern/flags (regex match):

hyalo find --property '!status'           # files missing the status property
hyalo find --property 'title~=draft'      # title contains "draft"
hyalo find --property 'title~=/^Draft/i'  # case-insensitive regex on title

K may be a dot-path into nested frontmatter. A literal dotted key in a flat map is tried first; otherwise the path is walked. Maps descend by key, and sequences descend too: a numeric segment pins one element, any other segment auto-descends into every element and collects the hits — so the usual list semantics apply (=/~= match when any element matches, != when none does):

hyalo find --property contact.email=team@example.com   # contact: {email: ...}
hyalo find --property contacts.email=ada@example.com   # contacts: [{name, email}, ...] — any element
hyalo find --property contacts.0.email=ada@example.com # first element only
hyalo find --property '!contacts.phone'                # no element has a phone

Schema & Lint Integration

Hyalo supports frontmatter schema validation. Define schemas in .hyalo.toml then run:

# Strict linting (errors on schema violations)
hyalo lint --strict --format text

# Auto-fix lint violations
hyalo lint --fix --dry-run --format text  # preview
hyalo lint --fix --format text           # apply

# Manage lint rules
hyalo lint-rules list --format text
hyalo lint-rules set MD013 --enabled false --format text  # disable line-length rule

Snapshot Index for Performance

For vaults >500 files, create a snapshot index to avoid repeated disk scans:

# Create index (one scan, reused by all queries)
hyalo create-index

# Use --index on all subsequent commands
hyalo find --property status=in-progress --index --format text
hyalo summary --index --format text

# Mutations also work with --index (patches index after each write)
hyalo set note.md --property status=completed --index --format text

# Drop when done
hyalo drop-index

File Movement with Link Rewriting

Always use hyalo mv — never system mv or git mv. It rewrites all [[wikilinks]] and [markdown](links) across the vault that pointed to the old path.

hyalo mv backlog/my-item.md --to backlog/done/my-item.md --dry-run --format text  # preview
hyalo mv backlog/my-item.md --to backlog/done/my-item.md --format text           # execute

Broken Link Detection & Repair

# Detect broken links with proposed fixes
hyalo links fix --format text

# Apply fixes
hyalo links fix --apply --format text

Saved Views for Common Queries

Save frequently-used filter combinations as named views:

# Create views for common queries
hyalo views set stale-in-progress --property status=in-progress --fields tasks
hyalo views set orphans --orphan --fields backlinks
hyalo views set missing-status --property '!status'

# Use views
hyalo find --view stale-in-progress --format text
hyalo find --view orphans --limit 5 --format text

Task Management

# Toggle task checkboxes
hyalo task toggle note.md --line 5,7 --format text
hyalo task toggle note.md --section "Tasks" --all --format text

# Read tasks with status
hyalo read note.md --section "Tasks" --format text

Type Schema Management

# List defined types
hyalo types list --format text

# Create/update iteration type schema
hyalo types set iteration --required title,date,status,branch,tags --format text
hyalo types set iteration --property-values "status=planned,in-progress,completed" --format text
hyalo types set iteration --filename-template "iterations/iteration-{n}-{slug}.md" --format text

When to Use hyalo vs Built-in pi Tools

The typed Pi tools use the generated Hyalo API runtime shipped with this package. They always request JSON without hints and preserve typed envelopes and errors; use the generic hyalo tool when text output, --jq, or an unsupported command is required.

User jq compilation and evaluation each run in a child with a 3-second deadline; source/input/output limits are 64 KiB / 64 MiB / 10 MiB (at most 1,000,000 values). Compilation failure precedes writes; evaluation failure retains committed effects. Linux caps worker address space at 512 MiB. macOS/Windows have no hard memory cap; intermediate allocations can still exhaust host memory before termination.

Setup Checklist for New Projects

1. Install hyalo: Ensure hyalo is on PATH (which hyalo). Use the published cargo install hyalo-cli route or, on a supported Node.js platform, npm install --global @ractive-ch/hyalo.
2. Configure vault: Create .hyalo.toml with dir = "knowledgebase"
3. Add to AGENTS.md: Include: "Use hyalo CLI for all markdown knowledgebase operations. Always use --format text for compact output."
4. Create views: Set up common views (stale-in-progress, orphans, etc.)
5. Define schemas: Create type schemas for consistent frontmatter

Advanced Patterns for pi

Bulk Status Updates

# Preview a native filtered update; rerun without --dry-run after review
hyalo set --glob 'iterations/*.md' --where-property status=planned \
  --property status=deferred --dry-run --format text

Health Dashboard

# Generate KB health report
hyalo summary --format text
hyalo lint --strict --format text
hyalo links fix --format text

Orphan Analysis

# Find orphans with context
hyalo find --orphan --fields properties,links --format text \
  | grep -v "SEED.md\|decision-log.md\|development-roadmap.md"  # exclude expected orphans

Common Pitfalls & Solutions

1. Using the generic tool for common operations: prefer hyalo_find/hyalo_read/hyalo_set/hyalo_task — no flag spelling or quoting to get wrong
2. Missing --format text: Output is verbose JSON — always use --format text in pi (the extension injects it automatically; only needed via bash)
3. Not using --index for large vaults: Queries are slow — create index for >500 files
4. Using system mv instead of hyalo mv: Breaks links — always use hyalo mv
5. Ignoring hints: hyalo suggests next commands — follow them
6. Not validating schemas: Run hyalo lint --strict regularly
7. Guessing flags instead of reading -h: every command's short help fits on one screen; --help has the full syntax

Integration with pi Extension

If the hyalo extension is installed (.pi/extensions/hyalo.ts), use the typed tools first:

{"tool": "hyalo_find", "property": ["status=planned"], "tag": "iteration"}

For anything they don't cover, use the generic hyalo tool:

{
  "subcommand": "find",
  "args": ["iteration", "--property", "status=planned", "--format", "text"],
  "formatText": true
}

Otherwise, use via bash tool:

hyalo find "iteration" --property status=planned --format text