# Parallel File Processor Mode Selection

> Sub-skill of parallel-file-processor: Mode Selection (+2).

- Skill: `vamseeachanta/parallel-file-processor-mode-selection` (Agent Skill)
- Install (CLI): `npx skillmds@latest add vamseeachanta/parallel-file-processor-mode-selection`
- Raw SKILL.md: https://api.skillmd.com/api/skills/vamseeachanta/parallel-file-processor-mode-selection/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: vamseeachanta (https://skillmd.com/u/vamseeachanta)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/vamseeachanta/parallel-file-processor-mode-selection

---


# Mode Selection (+2)

## Mode Selection


| Workload Type | Recommended Mode | Reason |
|---------------|------------------|--------|
| File I/O | `THREAD_POOL` | IO-bound, threads avoid GIL issues |
| Data parsing | `THREAD_POOL` | Pandas releases GIL during IO |
| CPU computation | `PROCESS_POOL` | Bypasses GIL for true parallelism |
| Network requests | `ASYNC` | Best for many concurrent connections |
| Simple operations | `SEQUENTIAL` | Overhead may exceed benefit |

## Worker Count


```python
import os

# IO-bound (reading files, network)
io_workers = os.cpu_count() * 2

# CPU-bound (heavy computation)
cpu_workers = os.cpu_count()

# Memory-constrained (large files)
memory_workers = max(2, os.cpu_count() // 2)
```

## Batch Size


- **Small files (<1MB):** Large batches (500-1000)
- **Medium files (1-100MB):** Medium batches (50-100)
- **Large files (>100MB):** Small batches (10-20) or one at a time