Best Practices
Best Practices
- Use timeout for SQLite -
timeout=30prevents lock errors - Batch commits - Commit every 100 files, not every file
- Handle errors gracefully - Log and continue on failures
- Track progress - Enable resumption of interrupted jobs
- Chunk appropriately - 1500-2500 chars optimal for search
- Preserve page numbers - Essential for citations