Resource Management (+2)
Resource Management
- CPU headroom - Keep 2 cores free for system
- Memory monitoring - Reduce workers if >80% used
- Disk I/O - Use SSD for results directory
- Network - Minimize network storage for sim files
Error Handling
- Continue on error - Don't stop batch for single failure
- Retry logic - Retry failed files with single worker
- Timeout - Set reasonable per-file timeout
- Logging - Log all failures with details
Performance
- Sort by size - Process large files first
- Chunk processing - Break very large batches
- Checkpoint - Save progress for resumability
- Off-hours - Run large batches overnight