Senior Backend Engineer
Overview
Design and implement robust, scalable backend systems with a focus on API design, service architecture, data management, and operational excellence. This skill covers RESTful and GraphQL API patterns, message-driven architecture, caching strategies, rate limiting, health checks, and full observability with OpenTelemetry.
Announce at start: "I'm using the senior-backend skill for backend system design and implementation."
Phase 1: API Design
Goal: Define the contract before writing implementation code.
Actions
- Define resource models and relationships
- Design endpoint structure (REST) or schema (GraphQL)
- Establish authentication and authorization strategy
- Define rate limiting and throttling policies
- Create API documentation (OpenAPI/GraphQL schema)
API Style Decision Table
| Factor |
REST |
GraphQL |
gRPC |
| Multiple consumers with different data needs |
Poor fit |
Strong fit |
Poor fit |
| Simple CRUD operations |
Strong fit |
Overkill |
Overkill |
| Real-time subscriptions |
Requires WebSocket add-on |
Built-in |
Built-in (streaming) |
| Service-to-service |
Good |
Overkill |
Strong fit |
| Public API |
Strong fit |
Good |
Poor fit (tooling) |
| Mobile with bandwidth constraints |
Overfetching risk |
Strong fit |
Strong fit |
STOP — Do NOT proceed to Phase 2 until:
Phase 2: Implementation
Goal: Build the service layer with clear separation of concerns.
Actions
- Set up project structure with clear layering
- Implement data access layer (repositories/DAOs)
- Build service layer with business logic
- Create API controllers/resolvers
- Add middleware (auth, logging, error handling, CORS)
- Implement caching strategy
RESTful URL Structure
GET /api/v1/users # List users (paginated)
GET /api/v1/users/:id # Get single user
POST /api/v1/users # Create user
PUT /api/v1/users/:id # Full update
PATCH /api/v1/users/:id # Partial update
DELETE /api/v1/users/:id # Delete user
GET /api/v1/users/:id/orders # Nested resources
POST /api/v1/users/:id/activate # State transitions
HTTP Status Code Decision Table
| Code |
Meaning |
When to Use |
| 200 |
OK |
Successful GET, PUT, PATCH |
| 201 |
Created |
Successful POST creating resource |
| 204 |
No Content |
Successful DELETE |
| 400 |
Bad Request |
Validation errors |
| 401 |
Unauthorized |
Missing or invalid auth |
| 403 |
Forbidden |
Auth valid but insufficient permissions |
| 404 |
Not Found |
Resource does not exist |
| 409 |
Conflict |
Duplicate or state conflict |
| 422 |
Unprocessable Entity |
Semantically invalid input |
| 429 |
Too Many Requests |
Rate limit exceeded |
| 500 |
Internal Server Error |
Unexpected server failure |
Response Format
// Success (single)
{ "data": { "id": "123", "name": "Alice" }, "meta": { "requestId": "req_abc123" } }
// Success (collection)
{ "data": [...], "meta": { "page": 1, "pageSize": 20, "totalCount": 150, "totalPages": 8 } }
// Error
{ "error": { "code": "VALIDATION_ERROR", "message": "Invalid input", "details": [...] } }
Caching Strategy Decision Table
| Strategy |
Description |
Use Case |
| Cache-Aside |
App checks cache, falls back to DB |
General purpose |
| Write-Through |
Write to cache and DB simultaneously |
Strong consistency |
| Write-Behind |
Write to cache, async write to DB |
High write throughput |
| Read-Through |
Cache loads from DB on miss |
Transparent caching |
STOP — Do NOT proceed to Phase 3 until:
Phase 3: Hardening
Goal: Prepare the service for production operation.
Actions
- Add comprehensive error handling
- Implement health checks and readiness probes
- Set up observability (traces, metrics, logs)
- Load test critical paths
- Document runbooks for operational scenarios
Health Check Endpoints
// GET /health — lightweight liveness check
{ "status": "healthy" }
// GET /health/ready — readiness with dependency checks
{
"status": "healthy",
"checks": {
"database": { "status": "healthy", "latency": "5ms" },
"redis": { "status": "healthy", "latency": "2ms" },
"queue": { "status": "healthy", "latency": "8ms" }
},
"uptime": "72h15m",
"version": "1.4.2"
}
Observability: RED Method Metrics
| Metric |
Description |
Implementation |
| Rate |
Requests per second |
Counter incremented per request |
| Errors |
Error rate per second |
Counter incremented per error |
| Duration |
Latency distribution |
Histogram (p50, p95, p99) |
Structured Logging Format
{
"timestamp": "2025-01-15T10:30:00.123Z",
"level": "info",
"message": "User created",
"service": "user-service",
"traceId": "abc123",
"spanId": "def456",
"userId": "usr_123",
"duration": 45
}
Rate Limiting Algorithm Decision Table
| Algorithm |
Pros |
Cons |
Best For |
| Fixed Window |
Simple, low memory |
Burst at boundaries |
Internal APIs |
| Sliding Window |
Smooth distribution |
More memory |
Public APIs |
| Token Bucket |
Controlled bursts |
Slightly complex |
Industry standard |
| Leaky Bucket |
Constant output |
No burst allowed |
Strict rate control |
STOP — Hardening complete when:
Event-Driven Architecture Patterns
Message Queue Pattern Decision Table
| Pattern |
Use Case |
Example |
| Pub/Sub |
Broadcast to multiple consumers |
User registered -> email, analytics, CRM |
| Work Queue |
Distribute tasks across workers |
Image processing, PDF generation |
| Request/Reply |
Async request with response |
Price calculation service |
| Dead Letter |
Handle failed messages |
Retry policy exceeded |
Event Schema
{
"eventId": "evt_abc123",
"eventType": "user.created",
"timestamp": "2025-01-15T10:30:00Z",
"version": "1.0",
"source": "user-service",
"data": { "userId": "usr_123", "email": "alice@example.com" },
"metadata": { "correlationId": "corr_xyz789", "causationId": "cmd_def456" }
}
GraphQL Anti-Patterns
| Anti-Pattern |
Problem |
Fix |
| N+1 queries |
Performance degradation |
DataLoader for batching |
| Unbounded queries |
DoS vulnerability |
Enforce depth and complexity limits |
| Over-fetching in resolvers |
Wasted DB queries |
Select only requested fields |
Anti-Patterns / Common Mistakes
| Anti-Pattern |
Why It Is Wrong |
Correct Approach |
| Exposing database IDs directly |
Security risk, coupling to DB |
Use UUIDs or prefixed IDs |
| Synchronous external service calls in request path |
Single point of failure, latency |
Async with queues or circuit breaker |
| N+1 query patterns |
Linear performance degradation |
Eager loading or DataLoader |
| Catching and swallowing errors |
Silent failures, impossible debugging |
Log and propagate with context |
| Shared mutable state across handlers |
Race conditions, unpredictable behavior |
Stateless request handling |
| Skipping input validation |
Injection, data corruption |
Validate at the edge, always |
| Generic 500 for all errors |
Poor developer experience |
Specific error codes and messages |
| No API versioning |
Breaking changes affect all consumers |
Version from day one (/v1/) |
Documentation Lookup (Context7)
Use mcp__context7__resolve-library-id then mcp__context7__query-docs for up-to-date docs. Returned docs override memorized knowledge.
express — for middleware patterns, routing, or request/response API
fastify — for plugin system, hooks, or schema validation
nestjs — for decorators, modules, providers, or guards
prisma — for schema syntax, client API, or migration commands
Integration Points
| Skill |
Relationship |
senior-architect |
Architecture decisions guide backend service boundaries |
security-review |
Backend security follows OWASP and auth patterns |
performance-optimization |
Backend performance uses caching and query tuning |
testing-strategy |
Backend test strategy defines integration test approach |
code-review |
Review verifies API design and error handling |
acceptance-testing |
API behavior becomes acceptance criteria |
senior-fullstack |
Backend serves the full-stack tRPC layer |
Key Principles
- API versioning from day one (
/v1/)
- Input validation at the edge (Zod, Joi, class-validator)
- Idempotency keys for non-GET endpoints
- Graceful shutdown (drain connections, finish in-flight requests)
- Circuit breaker for external service calls
- Database migrations versioned and reversible
- Secrets in environment variables, never in code
Skill Type
FLEXIBLE — Adapt API style and architecture to the project context. The three-phase process (design, implement, harden) is strongly recommended. Health checks, structured logging, and error handling are non-negotiable for production services.
1---
2name: senior-backend
3description: "Use when the user needs API design, microservices architecture, event-driven systems, database integration, caching strategies, or backend observability. Triggers: REST/GraphQL API implementation, service architecture design, message queue setup, rate limiting, health checks, OpenTelemetry integration."
4---
5
6# Senior Backend Engineer
7
8## Overview
9
10Design and implement robust, scalable backend systems with a focus on API design, service architecture, data management, and operational excellence. This skill covers RESTful and GraphQL API patterns, message-driven architecture, caching strategies, rate limiting, health checks, and full observability with OpenTelemetry.
11
12**Announce at start:** "I'm using the senior-backend skill for backend system design and implementation."
13
14---
15
16## Phase 1: API Design
17
18**Goal:** Define the contract before writing implementation code.
19
20### Actions
21
221. Define resource models and relationships
232. Design endpoint structure (REST) or schema (GraphQL)
243. Establish authentication and authorization strategy
254. Define rate limiting and throttling policies
265. Create API documentation (OpenAPI/GraphQL schema)
27
28### API Style Decision Table
29
30| Factor | REST | GraphQL | gRPC |
31|--------|------|---------|------|
32| Multiple consumers with different data needs | Poor fit | Strong fit | Poor fit |
33| Simple CRUD operations | Strong fit | Overkill | Overkill |
34| Real-time subscriptions | Requires WebSocket add-on | Built-in | Built-in (streaming) |
35| Service-to-service | Good | Overkill | Strong fit |
36| Public API | Strong fit | Good | Poor fit (tooling) |
37| Mobile with bandwidth constraints | Overfetching risk | Strong fit | Strong fit |
38
39### STOP — Do NOT proceed to Phase 2 until:
40- [ ] Resource models are defined
41- [ ] Endpoint structure or schema is documented
42- [ ] Auth strategy is chosen
43- [ ] API contract is reviewable (OpenAPI/GraphQL schema)
44
45---
46
47## Phase 2: Implementation
48
49**Goal:** Build the service layer with clear separation of concerns.
50
51### Actions
52
531. Set up project structure with clear layering
542. Implement data access layer (repositories/DAOs)
553. Build service layer with business logic
564. Create API controllers/resolvers
575. Add middleware (auth, logging, error handling, CORS)
586. Implement caching strategy
59
60### RESTful URL Structure
61
62```
63GET /api/v1/users # List users (paginated)
64GET /api/v1/users/:id # Get single user
65POST /api/v1/users # Create user
66PUT /api/v1/users/:id # Full update
67PATCH /api/v1/users/:id # Partial update
68DELETE /api/v1/users/:id # Delete user
69GET /api/v1/users/:id/orders # Nested resources
70POST /api/v1/users/:id/activate # State transitions
71```
72
73### HTTP Status Code Decision Table
74
75| Code | Meaning | When to Use |
76|------|---------|-------------|
77| 200 | OK | Successful GET, PUT, PATCH |
78| 201 | Created | Successful POST creating resource |
79| 204 | No Content | Successful DELETE |
80| 400 | Bad Request | Validation errors |
81| 401 | Unauthorized | Missing or invalid auth |
82| 403 | Forbidden | Auth valid but insufficient permissions |
83| 404 | Not Found | Resource does not exist |
84| 409 | Conflict | Duplicate or state conflict |
85| 422 | Unprocessable Entity | Semantically invalid input |
86| 429 | Too Many Requests | Rate limit exceeded |
87| 500 | Internal Server Error | Unexpected server failure |
88
89### Response Format
90
91```json
92// Success (single)
93{ "data": { "id": "123", "name": "Alice" }, "meta": { "requestId": "req_abc123" } }
94
95// Success (collection)
96{ "data": [...], "meta": { "page": 1, "pageSize": 20, "totalCount": 150, "totalPages": 8 } }
97
98// Error
99{ "error": { "code": "VALIDATION_ERROR", "message": "Invalid input", "details": [...] } }
100```
101
102### Caching Strategy Decision Table
103
104| Strategy | Description | Use Case |
105|----------|------------|----------|
106| Cache-Aside | App checks cache, falls back to DB | General purpose |
107| Write-Through | Write to cache and DB simultaneously | Strong consistency |
108| Write-Behind | Write to cache, async write to DB | High write throughput |
109| Read-Through | Cache loads from DB on miss | Transparent caching |
110
111### STOP — Do NOT proceed to Phase 3 until:
112- [ ] Project structure follows layered architecture
113- [ ] Input validation is at the edge (Zod, Joi, class-validator)
114- [ ] Error handling returns structured error responses
115- [ ] Caching strategy is implemented with invalidation plan
116
117---
118
119## Phase 3: Hardening
120
121**Goal:** Prepare the service for production operation.
122
123### Actions
124
1251. Add comprehensive error handling
1262. Implement health checks and readiness probes
1273. Set up observability (traces, metrics, logs)
1284. Load test critical paths
1295. Document runbooks for operational scenarios
130
131### Health Check Endpoints
132
133```json
134// GET /health — lightweight liveness check
135{ "status": "healthy" }
136
137// GET /health/ready — readiness with dependency checks
138{
139 "status": "healthy",
140 "checks": {
141 "database": { "status": "healthy", "latency": "5ms" },
142 "redis": { "status": "healthy", "latency": "2ms" },
143 "queue": { "status": "healthy", "latency": "8ms" }
144 },
145 "uptime": "72h15m",
146 "version": "1.4.2"
147}
148```
149
150### Observability: RED Method Metrics
151
152| Metric | Description | Implementation |
153|--------|------------|---------------|
154| **Rate** | Requests per second | Counter incremented per request |
155| **Errors** | Error rate per second | Counter incremented per error |
156| **Duration** | Latency distribution | Histogram (p50, p95, p99) |
157
158### Structured Logging Format
159
160```json
161{
162 "timestamp": "2025-01-15T10:30:00.123Z",
163 "level": "info",
164 "message": "User created",
165 "service": "user-service",
166 "traceId": "abc123",
167 "spanId": "def456",
168 "userId": "usr_123",
169 "duration": 45
170}
171```
172
173### Rate Limiting Algorithm Decision Table
174
175| Algorithm | Pros | Cons | Best For |
176|-----------|------|------|----------|
177| Fixed Window | Simple, low memory | Burst at boundaries | Internal APIs |
178| Sliding Window | Smooth distribution | More memory | Public APIs |
179| Token Bucket | Controlled bursts | Slightly complex | Industry standard |
180| Leaky Bucket | Constant output | No burst allowed | Strict rate control |
181
182### STOP — Hardening complete when:
183- [ ] Health check endpoints respond correctly
184- [ ] Structured logging is configured
185- [ ] Metrics are exported (RED method)
186- [ ] Load test completed on critical paths
187- [ ] Error handling returns appropriate status codes
188
189---
190
191## Event-Driven Architecture Patterns
192
193### Message Queue Pattern Decision Table
194
195| Pattern | Use Case | Example |
196|---------|----------|---------|
197| Pub/Sub | Broadcast to multiple consumers | User registered -> email, analytics, CRM |
198| Work Queue | Distribute tasks across workers | Image processing, PDF generation |
199| Request/Reply | Async request with response | Price calculation service |
200| Dead Letter | Handle failed messages | Retry policy exceeded |
201
202### Event Schema
203
204```json
205{
206 "eventId": "evt_abc123",
207 "eventType": "user.created",
208 "timestamp": "2025-01-15T10:30:00Z",
209 "version": "1.0",
210 "source": "user-service",
211 "data": { "userId": "usr_123", "email": "alice@example.com" },
212 "metadata": { "correlationId": "corr_xyz789", "causationId": "cmd_def456" }
213}
214```
215
216---
217
218## GraphQL Anti-Patterns
219
220| Anti-Pattern | Problem | Fix |
221|-------------|---------|-----|
222| N+1 queries | Performance degradation | DataLoader for batching |
223| Unbounded queries | DoS vulnerability | Enforce depth and complexity limits |
224| Over-fetching in resolvers | Wasted DB queries | Select only requested fields |
225
226---
227
228## Anti-Patterns / Common Mistakes
229
230| Anti-Pattern | Why It Is Wrong | Correct Approach |
231|-------------|----------------|-----------------|
232| Exposing database IDs directly | Security risk, coupling to DB | Use UUIDs or prefixed IDs |
233| Synchronous external service calls in request path | Single point of failure, latency | Async with queues or circuit breaker |
234| N+1 query patterns | Linear performance degradation | Eager loading or DataLoader |
235| Catching and swallowing errors | Silent failures, impossible debugging | Log and propagate with context |
236| Shared mutable state across handlers | Race conditions, unpredictable behavior | Stateless request handling |
237| Skipping input validation | Injection, data corruption | Validate at the edge, always |
238| Generic 500 for all errors | Poor developer experience | Specific error codes and messages |
239| No API versioning | Breaking changes affect all consumers | Version from day one (`/v1/`) |
240
241---
242
243## Documentation Lookup (Context7)
244
245Use `mcp__context7__resolve-library-id` then `mcp__context7__query-docs` for up-to-date docs. Returned docs override memorized knowledge.
246- `express` — for middleware patterns, routing, or request/response API
247- `fastify` — for plugin system, hooks, or schema validation
248- `nestjs` — for decorators, modules, providers, or guards
249- `prisma` — for schema syntax, client API, or migration commands
250
251---
252
253## Integration Points
254
255| Skill | Relationship |
256|-------|-------------|
257| `senior-architect` | Architecture decisions guide backend service boundaries |
258| `security-review` | Backend security follows OWASP and auth patterns |
259| `performance-optimization` | Backend performance uses caching and query tuning |
260| `testing-strategy` | Backend test strategy defines integration test approach |
261| `code-review` | Review verifies API design and error handling |
262| `acceptance-testing` | API behavior becomes acceptance criteria |
263| `senior-fullstack` | Backend serves the full-stack tRPC layer |
264
265---
266
267## Key Principles
268
269- API versioning from day one (`/v1/`)
270- Input validation at the edge (Zod, Joi, class-validator)
271- Idempotency keys for non-GET endpoints
272- Graceful shutdown (drain connections, finish in-flight requests)
273- Circuit breaker for external service calls
274- Database migrations versioned and reversible
275- Secrets in environment variables, never in code
276
277---
278
279## Skill Type
280
281**FLEXIBLE** — Adapt API style and architecture to the project context. The three-phase process (design, implement, harden) is strongly recommended. Health checks, structured logging, and error handling are non-negotiable for production services.