Binary File Suppression

Binary File Ingestion Suppression (Hex Buffer Guard)

Overview

When an autonomous agent mistakenly attempts to read a compiled binary (.exe, .wasm, .pyc), SQLite database (.sqlite), media asset (.png, .mp4), or compressed archive (.tar.gz, .zip) into text context, standard tokenizers attempt to decode raw byte streams into UTF-8 characters.

This causes immediate Context Poisoning: thousands of unrenderable replacement tokens (\ufffd, \x00, \x0f) flood the context window, blowing past token limits, degrading model attention weights, and causing model hallucination loops.

The Binary Suppression Protocol enforces pre-flight binary detection via magic bytes and null-byte heuristics, replacing raw file contents with clean structural metadata.

Raw Binary Ingestion vs. Metadata Guard

┌─────────────────────────────────────────────────────────────┐
│                 Binary Ingestion Comparison                 │
│                                                             │
│  Unprotected Binary Read (`cat app.sqlite`):                │
│  SQLite format 3\x00\x10\x00\x01\x01\x00@  \x00\x00\x00... │
│  ↳ 45,000 Garbage Tokens Ingested into Context              │
│  ↳ Context Poisoned, Attention Destroyed, $1.50 Wasted      │
│                                                             │
│  Binary Guard Interception:                                 │
│  [BINARY ARTIFACT SUPPRESSED]                               │
│  • Path: `data/app.sqlite`                                  │
│  • Format: SQLite 3 Database (Magic: `SQLite format 3`)     │
│  • Size: 4.2 MB (4,404,019 bytes)                           │
│  ↳ 25 Clean Metadata Tokens Ingested (99.9% Protection)     │
└─────────────────────────────────────────────────────────────┘

The 3-Tier Binary Detection Ladder

┌───────────────────────────────────────────────────────────────────────────┐
│ 1. EXTENSION BLACKLIST: `.wasm`, `.pyc`, `.dylib`, `.so`, `.png`, `.mp3`   │
│ 2. MAGIC BYTE SIGNATURES: Check first 16 bytes for ELF, SQLite, ZIP, PNG │
│ 3. NULL-BYTE HEURISTIC: If $>0$ null bytes (`\x00`) in first 8KB $\rightarrow$ Binary│
└───────────────────────────────────────────────────────────────────────────┘

Production Python Binary Guard Middleware

Implement this file reader wrapper in agent tool runtimes:

import mimetypes
from pathlib import Path

MAGIC_SIGNATURES = {
    b"SQLite format 3": "SQLite Database",
    b"\x7fELF": "Linux ELF Binary",
    b"MZ": "Windows PE Executable",
    b"\x89PNG\r\n\x1a\n": "PNG Image",
    b"PK\x03\x04": "ZIP / Jar Archive",
    b"\x1f\x8b": "GZIP Compressed Archive",
    b"\x00asm": "WebAssembly Binary"
}

def safe_read_file_content(file_path: Path, max_bytes: int = 8192) -> str:
    """Safely inspects and reads text files while suppressing binary/compiled artifacts."""
    with file_path.open("rb") as f:
        header = f.read(max_bytes)
        
    # 1. Check Magic Byte Signatures
    for signature, description in MAGIC_SIGNATURES.items():
        if header.startswith(signature):
            size_kb = file_path.stat().st_size / 1024
            return f"[BINARY FILE SUPPRESSED: {description} | Size: {size_kb:.2f} KB | Path: {file_path}]"
            
    # 2. Check for Null-Byte presence (Standard UTF-8 text never has null bytes)
    if b"\x00" in header:
        return f"[BINARY DATA DETECTED: Non-text byte sequence suppressed | Size: {file_path.stat().st_size} bytes]"
        
    # 3. Decode verified text
    return file_path.read_text(encoding="utf-8", errors="replace")

High-Risk Binary File Extensions

The agent execution runtime must automatically suppress full-file reads for:

Benchmark Metrics

Agent Operational Directive

MANDATORY: Agent tool runners (view_file, read_file) MUST verify that a target file contains pure text before decoding to UTF-8. If a binary is detected, return structured metadata and suggest the appropriate CLI inspection command.