# Celery Knowledge Patch

> Celery

- Skill: `nevaberry/celery-knowledge-patch-2` (Agent Skill, multi-file: 6 files)
- Install (CLI): `npx skillmds@latest add nevaberry/celery-knowledge-patch-2`
- Raw SKILL.md: https://api.skillmd.com/api/skills/nevaberry/celery-knowledge-patch-2/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- License: MIT
- Author: nevaberry (https://skillmd.com/u/nevaberry)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/nevaberry/celery-knowledge-patch-2

---



# Celery Knowledge Patch

Load this skill when maintaining Celery applications, workers, brokers, result
backends, schedules, or task workflows. Start with the quick reference, then
open the topic file that matches the work at hand.

## Reference index

| Reference | Topics |
| --- | --- |
| [runtime-and-lifecycle.md](references/runtime-and-lifecycle.md) | Runtime and dependency requirements, support policy, worker shutdown, signals, pools, and testing |
| [brokers-and-queues.md](references/brokers-and-queues.md) | RabbitMQ quorum queues, Redis, SQS, Google Cloud Pub/Sub, publishing, ETA limits, and queue creation |
| [tasks-and-scheduling.md](references/tasks-and-scheduling.md) | Pydantic tasks, crontab parsing, canvas behavior, timeouts, and gevent revocation |
| [result-backends.md](references/result-backends.md) | Database, Redis, DynamoDB, and Azure Blob Storage result backends |

## Check compatibility before upgrading

### Runtime requirements changed

- For 5.5, use CPython 3.8–3.13 or PyPy 3.10+, Kombu 5.5 or newer,
  redis-py 4.5.2 or newer, Billiard 4.2.1 or newer, and Django 2.2.28
  or newer.
- For 5.6, use CPython 3.9–3.13 or PyPy 3.11, Kombu 5.6 or newer,
  and Billiard 4.2.4 or newer.
- SQLAlchemy 1.4 and 2.0 are supported.
- Celery 4.x is unsupported. Celery 5.x is not an LTS line and is supported
  only until Celery 6.x.

See [runtime-and-lifecycle.md](references/runtime-and-lifecycle.md) before
changing Python, framework, or transport dependencies.

### SQS uses `pycurl` again

Celery 5.5 replaced the `pycurl` dependency with `urllib3`, but the 5.6 SQS
transport reverses that change and uses `pycurl` again. Ensure 5.6 SQS worker
images and deployment environments provide the restored dependency.

### Singleton groups unroll

A one-item `group` chained with `|` unrolls to its contained signature. The
downstream task receives the item result, not a one-element list:

```python
workflow = group(add.s(1, 1)) | consume.s()
```

Here, `consume` receives `2`. Audit downstream tasks that assumed a list.

### Hard timeouts can reject tasks

If a task exceeds its hard time limit while
`task_acks_on_failure_or_timeout` is `False`, the worker rejects it:

```python
task_acks_on_failure_or_timeout = False
```

## Configure queues and delayed tasks

### Use quorum queues deliberately

The default task queue type remains `classic`. To make the default task queue
a quorum queue, configure:

```python
task_default_queue_type = "quorum"
```

Quorum queues support ETA tasks. Workers detect them by default and
automatically enable native delayed delivery. The delayed-delivery queue type
defaults to `quorum`:

```python
broker_native_delayed_delivery_queue_type = "quorum"
worker_detect_quorum_queues = True
```

For queues Celery creates on demand, choose their queue type separately:

```python
task_create_missing_queue_type = "quorum"
```

Use `task_create_missing_queue_exchange_type` when the auto-created queue also
needs an explicit exchange type.

### Cap in-memory ETA tasks

Workers otherwise hold an unlimited number of ETA and countdown tasks in
memory. Set a cap for workloads that can accumulate many scheduled messages:

```python
worker_eta_task_limit = 1000
```

See [brokers-and-queues.md](references/brokers-and-queues.md) for transport,
publishing, and credential details.

## Make shutdown behavior explicit

### Allow a soft-shutdown interval

Soft shutdown gives active tasks a bounded interval to finish before cold
shutdown cancels them. It is disabled by default because its timeout is `0.0`:

```python
worker_soft_shutdown_timeout = 30.0
worker_enable_soft_shutdown_on_idle = True
```

Idle-worker soft shutdown defaults to `False`. Enabling both settings is
especially useful with visibility-timeout brokers such as Redis and SQS.

### Remap container termination

To make `SIGTERM` follow the `SIGQUIT` shutdown path, set:

```bash
export REMAP_SIGTERM="SIGQUIT"
```

This is useful when a container runtime sends `SIGTERM` by default. During
cold shutdown, workers skip timeout-failure handling so tasks are not marked
with spurious timeout failures and can complete or be requeued correctly.

## Use typed Pydantic tasks

Set `pydantic=True` on a task to convert type-hinted Pydantic arguments before
the task runs and dump a returned Pydantic model to a dictionary:

```python
from celery import Celery
from pydantic import BaseModel

app = Celery("tasks")

class Input(BaseModel):
    value: int

@app.task(pydantic=True)
def double(arg: Input) -> Input:
    return Input(value=arg.value * 2)
```

Optional and generic annotations are recognized. Use:

- `pydantic_strict` for strict validation.
- `pydantic_context` to pass validation context.
- `pydantic_dump_kwargs` to customize result serialization.

See [tasks-and-scheduling.md](references/tasks-and-scheduling.md) for optional
models, crontab parsing, publishing timeouts, and task termination.

## Choose backend initialization and credentials

The database result backend creates its tables during backend initialization
because `create_tables_at_setup` defaults to `True`. Set it to `False` to keep
lazy first-use creation when another system manages the schema.

Redis result backends support provider-based authentication through
`redis_backend_credential_provider`, including AWS ElastiCache IAM
authentication. Set `redis_client_name` to label backend connections for
Redis monitoring.

See [result-backends.md](references/result-backends.md) for Redis recovery,
remote local-DynamoDB endpoints, and Azure managed credentials.