# Django Queryset Batch Processing

> Process large Django querysets and write-heavy jobs with memory-safe reads, values/values_list, iterator chunking, set-based update/delete, bulk_create, bulk_update, F expressions, Func expressions, and batch sizing. Use when a Django command, task, migration, report, or loop reads or writes many rows and is slow, memory-heavy, or query-heavy.

- Skill: `thedixitjain/django-queryset-batch-processing` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds add thedixitjain/django-queryset-batch-processing`
- Raw SKILL.md: https://api.skillmd.com/api/skills/thedixitjain/django-queryset-batch-processing/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: thedixitjain (https://skillmd.com/u/thedixitjain)
- Updated: 2026-09-09
- Page: https://skillmd.com/skills/thedixitjain/django-queryset-batch-processing

---



# Django QuerySet Batch Processing

Use this skill when Django code processes many rows. The goal is to avoid loading unnecessary model instances, avoid queryset result-cache blowups, and move writes into set-based database operations when behavior allows.

## Workflow

1. Identify the per-row work.
   - Is it read-only export/reporting?
   - Does it need model methods, validation, or signals?
   - Can the database compute or update the value directly?

2. Choose the read pattern.
   - Use `values()` or `values_list()` for scalar exports and reports.
   - Use `iterator(chunk_size=...)` when model instances are needed but queryset caching is not.
   - Keep ordering deliberate; unnecessary ordering costs work.

3. Choose the write pattern.
   - Use `QuerySet.update()` with `F()` or expressions for uniform updates.
   - Use `bulk_update()` when each object has a different value.
   - Use `bulk_create()` for inserts, with conflict options only when the project supports their semantics.
   - Fall back to per-instance `save()` only when hooks, validation, side effects, or signals are required.

4. Control batch size.
   - Keep transactions bounded.
   - Avoid huge `IN` lists and oversized `CASE` updates.
   - Monitor locks, replication lag, and memory for production jobs.

See [batch-patterns.md](references/batch-patterns.md) for examples and caveats.

## Safety Notes

- Bulk update/delete operations do not call each model instance's `save()` or `delete()` methods.
- Bulk operations can skip application-level side effects and signals.
- Long transactions can hold locks and delay vacuum or replication.

## Verification

Measure rows processed per second, query count, memory, transaction duration, and correctness on a representative batch.

---

**Source:** [`hashgraph-online/awesome-codex-plugins`](https://github.com/hashgraph-online/awesome-codex-plugins) → `plugins/LVTD-LLC/skills/skills/django-queryset-batch-processing/SKILL.md`