Django QuerySet Batch Processing
Use this skill when Django code processes many rows. The goal is to avoid loading unnecessary model instances, avoid queryset result-cache blowups, and move writes into set-based database operations when behavior allows.
Workflow
Identify the per-row work.
- Is it read-only export/reporting?
- Does it need model methods, validation, or signals?
- Can the database compute or update the value directly?
Choose the read pattern.
- Use
values()orvalues_list()for scalar exports and reports. - Use
iterator(chunk_size=...)when model instances are needed but queryset caching is not. - Keep ordering deliberate; unnecessary ordering costs work.
- Use
Choose the write pattern.
- Use
QuerySet.update()withF()or expressions for uniform updates. - Use
bulk_update()when each object has a different value. - Use
bulk_create()for inserts, with conflict options only when the project supports their semantics. - Fall back to per-instance
save()only when hooks, validation, side effects, or signals are required.
- Use
Control batch size.
- Keep transactions bounded.
- Avoid huge
INlists and oversizedCASEupdates. - Monitor locks, replication lag, and memory for production jobs.
See batch-patterns.md for examples and caveats.
Safety Notes
- Bulk update/delete operations do not call each model instance's
save()ordelete()methods. - Bulk operations can skip application-level side effects and signals.
- Long transactions can hold locks and delay vacuum or replication.
Verification
Measure rows processed per second, query count, memory, transaction duration, and correctness on a representative batch.
Source: hashgraph-online/awesome-codex-plugins → plugins/LVTD-LLC/skills/skills/django-queryset-batch-processing/SKILL.md