Sanitize Text

Normalize raw text by removing excessive whitespace, non-printable characters, and standardizing unicode. Use this to clean up text extracted from PDFs or DOCX files before processing with LLMs.

majiayu000 3dca152 2 files · 1.7 KB Updated 567 repo stars

File contents

Sanitize Text

Overview

This skill cleans and normalizes raw text. It is essential for preprocessing text extracted from documents like PDFs, which often contain encoding artifacts, excessive whitespace, or weird control characters.

Usage

Sanitize Script

Syntax:

python3 .agent/skills/sanitize-text/scripts/sanitize.py <input_file> [--output <output_file>]

Arguments:

  • input_file: Path to the file containing raw text.
  • --output: (Optional) Path to write cleaned text to. If omitted, prints to stdout.

Example:

python3 .agent/skills/sanitize-text/scripts/sanitize.py raw_resume.txt --output clean_resume.txt

majiayu000/claude-skill-registry-data/tree/main/data/sanitize-text commit 3dca1529fd

Frequently asked questions

npx skillmds add majiayu000/sanitize-text