xberg-io

Extract text, tables, metadata, and images from 107 document formats (PDF, Office, images, HTML, email, archives, academic) using Xberg. Use when writing code that calls Xberg APIs in Python, Node.js/TypeScript, Rust, or CLI. Covers installation, extraction (sync/async), configuration (OCR, chunking, output format), batch processing, error handling, and plugins.

Alef-managed generated bindings in packages/* and binding crates — the regeneration workflow (task alef:generate / alef:verify), the alef.toml section layout, the core-side edits that break a regen, and the FFI bridge's JSON marshalling requirement. Load before editing anything under packages/* or a binding crate, before adding a trait method or extractor, or when regenerating or verifying Alef output.

MIME type detection and extractor routing in core/mime.rs — the FORMATS registry that EXT_TO_MIME and SUPPORTED_MIME_TYPES are derived from, the path-based and bytes-based detection functions, priority-based registry selection, wildcard MIME families, and the real procedure for adding a format. Load when adding a format, wiring an extractor to a MIME type, or debugging why a file routes to the wrong (or no) extractor.

Chunking, embeddings, and RAG pipeline integration

Cargo feature flags for crates/xberg — ORT-incompatible targets (WASM, Android x86_64 emulator), type-only and tract inference companion features, WASM/Android-safe variants, PDF backend, mutually-exclusive ORT variants, platform-conditional deps, aggregate feature sets, and build profiles. Load when adding, wiring, or debugging a Cargo feature, or when reasoning about what compiles on WASM/Android/Windows/macOS-intel targets.

Decide which xberg-io repository owns a cross-repository fix or API, and coordinate compatible changes across Xberg, Alef, enterprise, crawler, LLM, and OCR repositories. Load when work spans sibling repos; do not use for a self-contained Xberg edit.

Run, diagnose, or change Xberg extraction benchmarks, quality scoring, benchmark fixtures, artifact contracts, and independently sourced ground truth. Load for the Benchmarks workflow or benchmark-harness work, not ordinary unit tests.

How xberg versions are synced and released — Cargo.toml is the single source of truth, `task version:sync` propagates it to alef-managed binding manifests AND the integrations under integrations/, which are versioned and published in lockstep with core (including -rc.N). Load before bumping a version, editing the version-sync task, or touching an integration's version/xberg dependency.

Audit Xberg before a push or release by reconciling CI, Publish Release dry-run, Benchmarks, generated freshness, changelog, and remote branch state. Load for release-readiness work, not routine local commits.

WASM build constraints for the crates/xberg-wasm crate — the wasm-target feature set, no-tokio sync-only internal APIs, the crate-private SyncExtractor trait, the 2 MB HTML size limit, size-optimized build config (opt-level="z"), and the async-wrapper/sync-internal API pattern. Load when building for wasm32, adding or modifying a WASM-compatible extractor, or debugging WASM build/runtime failures.

Use when extracting keywords (YAKE/RAKE) from documents — and, secondarily, when detecting document language or generating embeddings for RAG and search. Covers the keyword config (and its feature gating), `--detect-language`, and the standalone `embed` command with real flags.

Use when extracting text from scanned PDFs, photographed pages, or images that have no embedded text layer. Covers OCR backends, language packs, force-OCR, and performance tuning.

The Xberg workspace layout — the version source of truth (root Cargo.toml [workspace.package] version), the 19 workspace members and 3 excluded crates, the distribution packages under packages/, the tools/ directory, and the ignore-file allowlists a new workspace member must be added to. Load when navigating the repo, deciding where code belongs, or wiring a new crate or binding package.

REST API server and MCP protocol integration

Use when extracting tabular data from PDFs, spreadsheets, or images. Covers layout-aware table detection, table model selection, output formats (markdown / JSON cells), and known limits.

Use when extracting from many files at once with shared config, bounded parallelism, per-file overrides, and error recovery. Covers the `batch` command, `--file-configs`, `--max-concurrent`, and output layout.

Use when choosing an output format for extracted documents — plain text, markdown, djot, HTML, JSON, or DocTags. Maps consumer (LLM, parser, archive) to the right `--format` / `--content-format` pair.

Change or diagnose Xberg PDF extraction, native/Pdfium backend selection, PDF rendering sessions, encrypted documents, OCR fallback, or backend-specific capability gaps. Load for PDF engine work, not generic image OCR.

The test_documents submodule is a bucket-fetched fixture corpus that is not committed. This skill covers read_test_fixture, missing fixtures, valid A/B controls, and submodule push order. Load before running Rust tests on a fresh clone, setting up an A/B control, adding a fixture-backed test, or diagnosing missing-fixture failures.

Use when splitting extracted text into chunks for LLM context windows or RAG ingestion. Covers chunk size, overlap, markdown/yaml/semantic chunkers, tokenizer-based sizing, and the standalone `chunk` command.