ellmos-tests
Use this module when an LLM should evaluate a local LLM-OS, agent hub, or SKILL.md-based system with the B/O/E method.
What It Provides
- B-tests: automated observation tests for inventory, structure, naming, docs, code metrics, dependencies, and age.
- O-tests: functional output tests for task, memory, tool registry, backup, config, and export/import surfaces.
- E-tests: manual experience tasks for orientation, memory, task workflows, tool use, recovery, and session startup.
- Battery runner: curated checklist-style batteries under
tests/batteries/.
- Optional browser helpers: Playwright wrappers under
system_diff_tests/testing/playwright/.
Standard Commands
PYTHONIOENCODING=utf-8 python system_diff_tests/run_all.py "<system_path>" --only b
PYTHONIOENCODING=utf-8 python system_diff_tests/run_all.py "<system_path>" --only o
PYTHONIOENCODING=utf-8 python system_diff_tests/testing/run_external.py "<system_path>" --profile STANDARD
PYTHONIOENCODING=utf-8 python tests/run_batteries.py --list
Known systems are resolved through system_diff_tests/config.py.
Boundaries
- Results are generated under ignored output folders by default.
- Do not include private corpora, local BACH context,
.db files, result archives, __pycache__, or .pytest_cache in the module package.
- Prefer environment variables (
ELLMOS_BASE_PATH, ELLMOS_ONEDRIVE, BACH_SYSTEM_PATH, BACH_DB_PATH) over hardcoded user paths.
- Playwright is optional; the core module remains Python-stdlib only.
Ownership and Extraction
This module originated by extracting the portable part of BACH system/tools/testing.
For the reusable B/O/E testing surface, ellmos-tests is now the canonical feature
source. BACH consumes it through a compatibility adapter and retains only its
integration contract plus a rollback-safe legacy copy while the gated migration is
validated. Changes to reusable runners, profiles, and B/O/E test definitions belong
here first; BACH-specific lifecycle or handler behavior remains in BACH.
1---2name: ellmos-tests3description: Standalone B/O/E testkit for evaluating SKILL.md-based LLM operating systems.4---56# ellmos-tests78Use this module when an LLM should evaluate a local LLM-OS, agent hub, or SKILL.md-based system with the B/O/E method.910## What It Provides1112- B-tests: automated observation tests for inventory, structure, naming, docs, code metrics, dependencies, and age.13- O-tests: functional output tests for task, memory, tool registry, backup, config, and export/import surfaces.14- E-tests: manual experience tasks for orientation, memory, task workflows, tool use, recovery, and session startup.15- Battery runner: curated checklist-style batteries under `tests/batteries/`.16- Optional browser helpers: Playwright wrappers under `system_diff_tests/testing/playwright/`.1718## Standard Commands1920```bash21PYTHONIOENCODING=utf-8 python system_diff_tests/run_all.py "<system_path>" --only b22PYTHONIOENCODING=utf-8 python system_diff_tests/run_all.py "<system_path>" --only o23PYTHONIOENCODING=utf-8 python system_diff_tests/testing/run_external.py "<system_path>" --profile STANDARD24PYTHONIOENCODING=utf-8 python tests/run_batteries.py --list25```2627Known systems are resolved through `system_diff_tests/config.py`.2829## Boundaries3031- Results are generated under ignored output folders by default.32- Do not include private corpora, local BACH context, `.db` files, result archives, `__pycache__`, or `.pytest_cache` in the module package.33- Prefer environment variables (`ELLMOS_BASE_PATH`, `ELLMOS_ONEDRIVE`, `BACH_SYSTEM_PATH`, `BACH_DB_PATH`) over hardcoded user paths.34- Playwright is optional; the core module remains Python-stdlib only.3536## Ownership and Extraction3738This module originated by extracting the portable part of BACH `system/tools/testing`.39For the reusable B/O/E testing surface, `ellmos-tests` is now the canonical feature40source. BACH consumes it through a compatibility adapter and retains only its41integration contract plus a rollback-safe legacy copy while the gated migration is42validated. Changes to reusable runners, profiles, and B/O/E test definitions belong43here first; BACH-specific lifecycle or handler behavior remains in BACH.
ellmos-ai