# High Concurrency Scalability

> Design and optimize systems for high concurrency, throughput, scalability, and elastic scale—concurrency models (threads, async/await, actors), lock-free patterns, connection pooling, caching stampede mitigation, horizontal scaling, load balancing, backpressure, queueing, rate limiting, bulkheads, read replicas, sharding, pool tuning, profiling, capacity planning, SLO-driven autoscaling, multi-region and CDN edge architecture. Use when the user asks about high concurrency, scalability, throughput, horizontal scaling, connection pooling, backpressure, rate limiting, caching stampede, read replica, sharding, autoscaling, capacity planning, lock contention, async scalability, or load balancing—not service decomposition (microservices-developer), event buses only (event-driven-architecture), generic CRUD (senior-software-engineer), SRE on-call only (site-reliability-engineer), load tests without architecture (performance-engineer), or cost-only FinOps (cloud-economist).

- Skill: `daemon-blockint-tech/high-concurrency-scalability` (Agent Skill, multi-file: 7 files)
- Install (CLI): `npx skillmds@latest add daemon-blockint-tech/high-concurrency-scalability`
- Raw SKILL.md: https://api.skillmd.com/api/skills/daemon-blockint-tech/high-concurrency-scalability/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- Author: daemon-blockint-tech (https://skillmd.com/u/daemon-blockint-tech)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/daemon-blockint-tech/high-concurrency-scalability

---


# High Concurrency & Scalability

## When to Use

- Choose or refactor **concurrency models**—threads, async/await, actors, coroutines—for target throughput and latency
- Reduce **lock contention** and design low-contention, lock-free, or partitioned data paths
- Size **connection pools**, file descriptors, thread pools, and memory limits per dependency
- Design **caching** layers, TTL strategy, and **stampede** / thundering-herd mitigation
- Plan **horizontal scaling**, **load balancing**, session affinity, and stateless vs sticky tradeoffs
- Apply **backpressure**, bounded queues, **rate limiting**, and bulkheads under overload
- Scale the **data layer**—read replicas, routing, sharding concepts, pool tuning, hot keys
- Profile bottlenecks, model **capacity**, and tie scale triggers to **SLOs** and error budgets
- Define **autoscaling** signals, warm pools, and cold-start vs cost tradeoffs
- Architect **multi-region** read paths and CDN/edge caching at a design level

## When NOT to Use

- Decompose monoliths into bounded contexts and inter-service contracts only → `microservices-developer`
- Event schemas, broker selection, and messaging topology only → `event-driven-architecture`
- General feature delivery, RFCs, or CRUD without scale focus → `senior-software-engineer`
- Org-wide SLO program, on-call, incident response, and error-budget policy → `site-reliability-engineer`
- Deep flame graphs, load-test harnesses, and p99 regression hunts as the main task → `performance-engineer`
- Kubernetes platform golden paths and IDP product work → `platform-engineer`
- VPC, managed service provisioning, and landing-zone IaC → `cloud-engineer`
- Cloud spend optimization and unit economics only → `cloud-economist`, `finops-analyst`

## Related skills

| Need | Skill |
|---|---|
| Service boundaries, sagas, circuit breakers between services | `microservices-developer` |
| Brokers, topics, event contracts, outbox | `event-driven-architecture` |
| Profiling, load/soak tests, latency budgets | `performance-engineer` |
| SLI/SLO programs, incident reliability, toil | `site-reliability-engineer` |
| Internal platform, K8s abstractions, golden paths | `platform-engineer` |
| Cloud compute, networking, DR multi-region deploy | `cloud-engineer` |
| Application design and refactoring | `senior-software-engineer` |

## Core Workflows

### 1. Scope and constraints

Clarify traffic shape, SLOs, statefulness, and failure modes.

**See `references/high_concurrency_scalability_scope.md`.**

### 2. Concurrency and synchronization

Pick execution model; partition work; minimize shared mutable state.

**See `references/concurrency_models_and_synchronization.md`.**

### 3. Caching and data-layer scale

Cache hierarchy, replica routing, sharding and hot-key mitigation.

**See `references/caching_and_data_layer_scale.md`.**

### 4. Throughput, backpressure, and queues

Bounded queues, shedding, rate limits, and async pipelines.

**See `references/throughput_backpressure_and_queues.md`.**

### 5. Horizontal scale and load distribution

Replicas, LB algorithms, affinity, autoscaling triggers.

**See `references/horizontal_scaling_and_load_distribution.md`.**

### 6. Capacity, observability, and SLO-driven scale

Metrics, headroom models, scale policies tied to objectives.

**See `references/capacity_planning_observability_slo.md`.**

## Outputs

- **Scale brief** — workload profile, bottlenecks, target RPS/latency, state assumptions
- **Concurrency note** — model choice, pool sizes, contention risks, partitioning plan
- **Cache and data plan** — layers, TTL, invalidation, replica/shard routing, hot-key mitigations
- **Overload matrix** — backpressure, rate limits, bulkheads, degradation modes
- **Capacity model** — headroom, scale triggers, cold-start impact, cost sensitivity
- **Observability checklist** — saturation, queue depth, pool wait, cache hit rate, tail latency

## Principles

- **Measure saturation**—CPU, memory, I/O, pool wait, queue depth—not averages alone
- **Bound everything**—connections, threads, queue length, in-flight requests
- **Prefer partition over lock**—shard by key, actor mailbox, or isolated replica
- **Design for overload**—shed load deliberately; never unbounded retry or queue growth
- **Scale on SLO signals**—error rate and tail latency, not CPU alone
- **Keep hot paths stateless** where possible; isolate stateful tiers explicitly