Skill Concat Glob Tool

When to use

• Need to concatenate multiple files matching patterns
• Creating LLM context files from source code
• Merging documentation or configuration files
• Building single-file distributions

concat-glob-tool Skill

Purpose

Master file concatenation with glob patterns for efficient file merging, documentation bundling, and LLM context generation. This skill covers CLI usage, library integration, and advanced patterns.

When to Use This Skill

Use this skill when:

• Concatenating files matching glob patterns (.py, src/**/.md)
• Creating context files for Large Language Models
• Merging multiple source files for documentation
• Building single-file distributions with separators
• Integrating file concatenation into Python scripts

Do NOT use this skill for:

• Single file operations (use cat or cp)
• Binary file concatenation (use specialized tools)
• Real-time streaming (this is batch processing)

CLI Tool: concat-glob-tool

A production-ready CLI utility that concatenates files matching glob patterns with intelligent separators. Features dry-run mode, stdin support, and both CLI and library modes.

Installation

# From source
git clone https://github.com/dnvriend/concat-glob-tool.git
cd concat-glob-tool
uv tool install .

# Verify
concat-glob-tool --version

Prerequisites

• Python 3.14+
• uv package manager
• Access to file system

Quick Start

# Preview (dry-run, default)
concat-glob-tool '*.py' -o output.txt

# Execute
concat-glob-tool '*.py' '*.md' -o output.txt --no-dry-run

# Stdin mode
find . -name '*.py' | concat-glob-tool --stdin -o output.txt --no-dry-run

Progressive Disclosure

concat - Concatenate Files with Glob Patterns

Concatenate files matching glob patterns to a single output file with intelligent separators.

Usage:

concat-glob-tool PATTERNS... --output-file FILE [OPTIONS]

Arguments:

• PATTERNS...: One or more glob patterns (e.g., *.py, src/**/*.md)
• -o, --output-file FILE: Output file path (required)
• --separator TEXT: Separator text between files (default: ---)
• -n, --dry-run: Preview without writing (enabled by default)
• --no-dry-run: Actually execute the concatenation
• -f, --force: Overwrite existing output file
• -v, --verbose: Enable verbose output (repeatable: -v, -vv, -vvv)
• --version: Show version
• --help: Show help message with examples

Examples:

# Basic concatenation - preview mode (dry-run)
concat-glob-tool '*.py' -o output.txt

# Execute concatenation
concat-glob-tool '*.py' -o output.txt --no-dry-run

# Multiple patterns
concat-glob-tool '*.py' '*.md' '*.txt' -o combined.txt --no-dry-run

# Recursive patterns
concat-glob-tool 'src/**/*.py' -o all-code.txt --no-dry-run

# Multiple recursive patterns
concat-glob-tool 'src/**/*.py' 'tests/**/*.py' -o codebase.txt --no-dry-run

# Custom separator
concat-glob-tool '*.py' -o output.txt --separator '===' --no-dry-run

# Force overwrite existing file
concat-glob-tool '*.py' -o existing.txt --force --no-dry-run

# Verbose output
concat-glob-tool '*.py' -o output.txt -vv --no-dry-run

# LLM context generation
concat-glob-tool 'src/**/*.py' 'tests/**/*.py' '*.md' \
    -o llm-context.txt \
    --separator '---' \
    --no-dry-run

Output Format: Files are concatenated with this separator format between each file:

---
# /path/to/file.py
---
[file contents]

The separator includes:

• Blank line before separator
• Separator line (configurable, default: ---)
• Comment line with full file path
• Separator line
• Blank line before file contents

Dry-Run Mode (Default): By default, the tool runs in preview mode showing:

• Number of files that would be concatenated
• List of files to process
• Separator format
• Reminder to use --no-dry-run to execute

This prevents accidental overwrites and lets you verify operations first.

stdin - Concatenate from Stdin

Read file paths from stdin and concatenate them. Useful for integration with find, fd, grep, and other tools.

Usage:

find ... | concat-glob-tool --stdin --output-file FILE [OPTIONS]

Arguments:

• -s, --stdin: Read file paths from stdin (one per line)
• -o, --output-file FILE: Output file path (required)
• --separator TEXT: Separator text (default: ---)
• --no-dry-run: Execute the concatenation
• -f, --force: Overwrite existing output file
• -v, --verbose: Verbose output

Examples:

# From find command
find . -name '*.py' -type f | concat-glob-tool --stdin -o output.txt --no-dry-run

# From fd (faster find alternative)
fd -e py | concat-glob-tool --stdin -o output.txt --no-dry-run

# Filter with grep
find . -name '*.py' | grep -v test | concat-glob-tool --stdin -o output.txt --no-dry-run

# Complex pipeline
find . -name '*.py' -type f | \
    grep -v __pycache__ | \
    grep -v .venv | \
    concat-glob-tool --stdin -o filtered.txt --no-dry-run

# With custom separator
find . -name '*.md' | concat-glob-tool --stdin -o docs.txt --separator '===' --no-dry-run

Note:

• Cannot use both --stdin and glob patterns in the same command
• Each line from stdin should be a valid file path
• Invalid paths will cause clear error messages with solutions

Environment Variable Expansion

Glob patterns support environment variable expansion and home directory shortcuts.

Examples:

# Home directory expansion
concat-glob-tool '~/projects/*.py' -o output.txt --no-dry-run

# Environment variables
concat-glob-tool '$HOME/src/**/*.py' -o output.txt --no-dry-run

# Mixed
concat-glob-tool '~/.config/*.conf' '$PROJECT_DIR/**/*.md' -o output.txt --no-dry-run

Library Integration

Use concat-glob-tool as a Python library for programmatic integration.

Installation:

pip install concat-glob-tool
# or
uv add concat-glob-tool

Core API:

from concat_glob_tool import (
    expand_glob_patterns,
    concatenate_files,
    format_separator,
    ConcatError,
    NoMatchesError,
    OutputExistsError,
)
from pathlib import Path

# Expand glob patterns
files = expand_glob_patterns(["*.py", "src/**/*.md"])
print(f"Found {len(files)} files")

# Concatenate files
result = concatenate_files(
    files=files,
    output_file=Path("output.txt"),
    separator="---",
    force=False,
    dry_run=False,
)

print(f"Concatenated {result['files_count']} files")
print(f"Wrote {result['bytes_written']} bytes to {result['output_file']}")

Exception Handling:

from pathlib import Path
from concat_glob_tool import (
    concatenate_files,
    expand_glob_patterns,
    NoMatchesError,
    OutputExistsError,
    ConcatError,
)

try:
    files = expand_glob_patterns(["*.py"])
    result = concatenate_files(
        files=files,
        output_file=Path("output.txt"),
        force=False,
        dry_run=False,
    )
    print(f"Success: {result}")

except NoMatchesError as e:
    print(f"No files found: {e}")
except OutputExistsError as e:
    print(f"Output exists: {e}")
except ConcatError as e:
    print(f"Error: {e}")

Integration Example:

#!/usr/bin/env python3
"""Example: Concatenate Python files with custom logic."""

from pathlib import Path
from concat_glob_tool import expand_glob_patterns, concatenate_files

def main():
    # Find all Python files
    files = expand_glob_patterns(["src/**/*.py", "tests/**/*.py"])
    
    # Filter files (e.g., exclude __init__.py)
    filtered_files = [f for f in files if f.name != "__init__.py"]
    
    # Concatenate
    result = concatenate_files(
        files=filtered_files,
        output_file=Path("codebase.txt"),
        separator="===",
        force=True,
        dry_run=False,
    )
    
    print(f"✅ Concatenated {result['files_count']} files")
    print(f"📝 Output: {result['output_file']}")
    print(f"💾 Size: {result['bytes_written']} bytes")

if __name__ == "__main__":
    main()

Custom Separators

Customize the separator text while maintaining the format structure.

Examples:

# Simple separator
concat-glob-tool '*.py' -o output.txt --separator '---' --no-dry-run

# Equals signs
concat-glob-tool '*.py' -o output.txt --separator '===' --no-dry-run

# Hash separator
concat-glob-tool '*.py' -o output.txt --separator '###' --no-dry-run

# Custom text
concat-glob-tool '*.py' -o output.txt --separator 'FILE' --no-dry-run

Separator Format: The separator structure is always: \n{separator}\n# {filename}\n{separator}\n

Example with --separator '===':

===
# /path/to/file.py
===
[file contents]

Common Issues

Issue: No files matched the patterns

$ concat-glob-tool '*.nonexistent' -o out.txt --no-dry-run
Error: No files matched the patterns: *.nonexistent

Solution: Verify glob patterns are correct. Examples:
  - '*.py' for Python files in current directory
  - '**/*.py' for Python files recursively
  - 'src/**/*.{py,md}' for multiple extensions

Solution:

• Check glob pattern syntax
• Verify files exist in specified locations
• Use --verbose to see detailed logging
• Test pattern with ls first: ls *.py

Issue: Output file already exists

$ concat-glob-tool '*.py' -o existing.txt --no-dry-run
Error: Output file already exists: existing.txt

Solution: Use --force to overwrite or choose a different output file.

Solution:

• Use --force flag to overwrite: concat-glob-tool '*.py' -o existing.txt --force --no-dry-run
• Choose different output filename
• Remove existing file first

Issue: Cannot use both --stdin and glob patterns

$ concat-glob-tool '*.py' --stdin -o out.txt
Error: Cannot use both --stdin and glob patterns.

Solution: Use either --stdin OR provide glob patterns, not both.

Solution:

• Use glob patterns: concat-glob-tool '*.py' -o out.txt --no-dry-run
• OR use stdin: find . -name '*.py' | concat-glob-tool --stdin -o out.txt --no-dry-run

Issue: Forgot --no-dry-run flag

$ concat-glob-tool '*.py' -o output.txt
[DRY-RUN] Would concatenate 10 files to: output.txt
...
To execute, add --no-dry-run flag.

Solution:

• This is expected behavior (dry-run is default)
• Add --no-dry-run to actually execute
• Dry-run mode prevents accidental overwrites

Getting Help

# Full help with examples
concat-glob-tool --help

# Version information
concat-glob-tool --version

# Verbose output for debugging
concat-glob-tool '*.py' -o output.txt -vv --no-dry-run

Verbose Logging Levels

• No flag (default): WARNING only
• -v: INFO level (operations and progress)
• -vv: DEBUG level (detailed information)
• -vvv: TRACE level (library internals)

All logs go to stderr, keeping stdout clean for piping.

Exit Codes

• 0: Success
• 1: Error (NoMatchesError, OutputExistsError, ConcatError, validation errors)

Output Formats

Separator Format:

\n---\n# /path/to/filename\n---\n

Components:

• Blank line before separator
• Separator line (customizable via --separator)
• Comment line with full file path
• Separator line
• Blank line before file contents

Dry-Run Output:

• Number of files to concatenate
• List of file paths
• Separator format preview
• Instructions to execute

Execution Output:

Successfully concatenated 10 files to output.txt (52482 bytes)

Best Practices

1. Always Preview First: Use default dry-run mode to verify operations before executing with --no-dry-run

2. Use Descriptive Output Names: Name output files clearly indicating contents (e.g., llm-context.txt, all-python-code.txt)

3. Leverage Verbose Mode: Use -v or -vv when troubleshooting or learning the tool's behavior

4. Quote Glob Patterns: Always quote patterns in shell to prevent premature expansion: '*.py' not *.py

5. Test Patterns with ls: Verify glob patterns work with ls before using in concat-glob-tool

6. Use stdin for Complex Filtering: Combine with find, fd, grep for advanced file selection

7. Custom Separators for Context: Use meaningful separators when creating context files for specific purposes

Resources

• GitHub: https://github.com/dnvriend/concat-glob-tool
• Documentation: README.md in repository
• Python Package: Available via pip/uv
• License: MIT