Backend Grpc Patterns

Backend gRPC Patterns

Purpose

Design consistent, production-grade gRPC services. Every RPC must follow the same conventions for protobuf schemas, streaming patterns, interceptors, error handling, and service versioning.

Agent Protocol

Trigger

Exact user phrases: "gRPC", "protobuf", "protocol buffers", "streaming RPC", "unary call", "server streaming", "client streaming", "bidirectional streaming", "gRPC interceptor", "gRPC error handling", "protobuf schema", "service definition", "design an RPC", "gRPC contract".

Input Context

Before activating, verify:

• The service or feature being designed is known.
• The proto package name and version are known.
• The streaming model (unary/server-stream/client-stream/bidi) is chosen. If not, ask: "What streaming model do you need?"
• The language/runtime for code generation is known.

Output Artifact

No file output unless the user requests it. Produces protobuf schema and RPC specifications as text.

Response Format

For each RPC:

rpc {MethodName}({RequestType}) returns ({ResponseType})
Streaming: {none/server/client/bidirectional}
Auth: {required/optional/none}
Deadline: {timeout suggestion}
Errors: {list of gRPC status codes}

For a full service definition:

service {ServiceName} {
  {list of RPCs}
}

No preamble. No postamble. No explanations. No filler/hedging/transitions. Compress output — why use many token when few do trick.

Completion Criteria

• All protobuf packages follow the naming conventions below.
• Every RPC has: method signature, streaming type, auth requirement, deadline, error codes.
• Protobuf messages are versioned in the package name.
• Error responses use standard gRPC status codes with rich detail.
• All streaming RPCs include proper backpressure handling.
• Interceptor chain is documented.

Max Response Length

Per RPC: 6 lines. Per service: unlimited.

Decision Tree

Which RPC Pattern?

What is the communication pattern?
  ├── Request → Response (standard CRUD)
  │   └── Unary RPC — simplest, HTTP/2 under the hood
  ├── Request → stream of events (subscribe, watch, notifications)
  │   └── Server-streaming RPC — send multiple responses for one request
  ├── Stream of requests → single response (batch upload, long-form input)
  │   └── Client-streaming RPC — aggregate client data into one result
  ├── Stream → stream (chat, real-time sync, gaming)
  │   └── Bidirectional streaming — full-duplex communication
  └── Fire-and-forget (event, notification, pub/sub)
      └── Consider message queue instead — gRPC streaming is for persistent connections

When to Use gRPC vs REST vs Message Queue?

gRPC:               Internal service-to-service, low latency, high throughput
                    Streaming, real-time, bidirectional
                    Polyglot environments (codegen for many languages)
REST:               External/public APIs (browsers, mobile, third-party)
                    Simple CRUD, cacheable resources
                    When HTTP semantics matter (caching, content negotiation)
Message Queue:      Async communication, event-driven, fire-and-forget
                    When services must be decoupled in time
                    When you need guaranteed delivery, retries, DLQ

Workflow

Step 1: Choose Streaming Model

Unary:              request → response                    (standard CRUD)
Server-streaming:   request → stream<response>            (list, watch, subscribe)
Client-streaming:   stream<request> → response            (batch upload, long-form input)
Bidirectional:      stream<request> → stream<response>    (chat, real-time sync, gaming)

Step 2: Define Protobuf Package and Version

syntax = "proto3";

package acme.users.v1;

option go_package = "github.com/acme/gen/go/users/v1;usersv1";
option java_package = "com.acme.users.v1";

Step 3: Define Messages

Use proto3. Follow these rules:

• message names are PascalCase: CreateUserRequest, UserResponse.
• Field names are snake_case: user_id, created_at.
• Field numbers are dense starting at 1. Never reuse a field number.
• Use google.protobuf.Timestamp for time, not int64 or string.
• Use google.protobuf.FieldMask for partial updates.
• Use google.protobuf.Empty for void responses.
• oneof for mutually exclusive fields.
• map for key-value pairs (never for large datasets).

message CreateUserRequest {
  string name = 1;
  string email = 2;
  google.protobuf.Timestamp birth_date = 3;
  oneof contact_preference {
    string phone = 4;
    string telegram = 5;
  }
}

Step 4: Standard RPC Patterns

// CRUD service
service UserService {
  rpc CreateUser(CreateUserRequest) returns (User);
  rpc GetUser(GetUserRequest) returns (User);
  rpc ListUsers(ListUsersRequest) returns (ListUsersResponse);
  rpc UpdateUser(UpdateUserRequest) returns (User);
  rpc DeleteUser(DeleteUserRequest) returns (google.protobuf.Empty);
}

// Streaming service
service EventService {
  rpc Subscribe(SubscribeRequest) returns (stream Event);
  rpc Publish(stream PublishRequest) returns (PublishSummary);
  rpc Chat(stream ChatMessage) returns (stream ChatMessage);
}

Step 5: Pagination for Streaming Lists

Server-streaming is preferred for large lists to avoid OOM:

message ListUsersRequest {
  string page_token = 1;    // opaque cursor, not a number
  int32 page_size = 2;      // max items to return
}

message ListUsersResponse {
  repeated User users = 1;
  string next_page_token = 2;  // empty means no more
}

Step 6: Error Handling

Always return standard gRPC status codes with rich detail via google.rpc.Status:

Rich error detail:

import "google/rpc/error_details.proto";

message ErrorInfo {
  string reason = 1;
  string domain = 2;
  map<string, string> metadata = 3;
}

Step 7: Interceptor Chain

Order matters — apply in this sequence:

Client side:
  1. Deadline/timeout interceptor (outermost)
  2. Auth token injection
  3. Tracing (OpenTelemetry span injection)
  4. Logging (request/response summary)
  5. Circuit breaker / retry (innermost)

Server side:
  1. Tracing (span creation) (outermost)
  2. Logging (request metadata)
  3. Auth validation (JWT, mTLS)
  4. Rate limiting
  5. Deadline enforcement (innermost)

// gRPC server interceptor — auth + logging (Node.js)
import { ServerInterceptor, status } from '@grpc/grpc-js';

const authInterceptor: ServerInterceptor = (call, definition) => {
  const deadline = call.getDeadline();
  if (deadline && Date.now() > deadline.toMillis()) {
    call.emit('error', { code: status.DEADLINE_EXCEEDED, details: 'Deadline exceeded' });
    return;
  }
  const metadata = call.metadata;
  const token = metadata.get('authorization')[0] as string;
  if (!token) {
    call.emit('error', { code: status.UNAUTHENTICATED, details: 'Missing token' });
    return;
  }
  call.user = verifyToken(token);
  return new (definition as any)(call, definition);
};

// gRPC server interceptor — logging (Go)
import (
  "google.golang.org/grpc"
  "google.golang.org/grpc/status"
)

func loggingUnaryInterceptor(ctx context.Context, req any, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (any, error) {
  start := time.Now()
  resp, err := handler(ctx, req)
  duration := time.Since(start)
  level := slog.LevelInfo
  if err != nil {
    level = slog.LevelError
  }
  slog.LogAttrs(ctx, level, "gRPC call",
    slog.String("method", info.FullMethod),
    slog.Duration("duration", duration),
    slog.String("status", status.Code(err).String()),
  )
  return resp, err
}

Step 8: Streaming Implementation Patterns

Server-streaming with backpressure (Go):

func (s *EventService) Subscribe(req *pb.SubscribeRequest, stream pb.EventService_SubscribeServer) error {
  // Backpressure: server controls send rate
  ch := s.eventBus.Subscribe(req.Topic)
  defer s.eventBus.Unsubscribe(req.Topic, ch)

  for {
    select {
    case event := <-ch:
      if err := stream.Send(event); err != nil {
        // Client disconnected or network error
        return err
      }
    case <-stream.Context().Done():
      // Client cancelled
      return stream.Context().Err()
    }
  }
}

Bidirectional streaming with flow control (Node.js):

async function chat(call: ServerDuplexStream<ChatMessage, ChatMessage>) {
  call.on('data', (msg: ChatMessage) => {
    // Process incoming message
    // Use backpressure: call.write returns boolean (true = buffer ok, false = backpressure)
    const canWrite = call.write({
      id: msg.id,
      text: `Echo: ${msg.text}`,
      timestamp: now(),
    });
    if (!canWrite) {
      call.pause();  // Wait for drain event
      call.once('drain', () => call.resume());
    }
  });
  call.on('end', () => call.end());
}

Step 9: Client-Side Patterns

// gRPC client with deadline and retry
import { credentials, ServiceError } from '@grpc/grpc-js';

async function callWithRetry<T>(fn: () => Promise<T>, maxRetries = 3): Promise<T> {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    try {
      return await fn();
    } catch (err) {
      const grpcErr = err as ServiceError;
      if (grpcErr.code === status.UNAVAILABLE && attempt < maxRetries - 1) {
        await delay(Math.pow(2, attempt) * 100);  // exponential backoff
        continue;
      }
      throw err;
    }
  }
  throw new Error('Unreachable');
}

const client = new UserServiceClient('localhost:50051', credentials.createInsecure());
const response = await callWithRetry(() => client.getUser({ userId: '123' }));

Step 10: Service Versioning

Package version in proto:   acme.users.v1 → acme.users.v2
Never break existing v1 RPCs — add new RPCs in v2 or add fields to v1 messages
Field deprecation:
  string old_field = 3 [deprecated = true];
  // Add replacement field
  string new_field = 4;
Client migration: run both versions simultaneously, migrate clients one by one

Production Considerations

Performance

Security

Anti-Patterns

Rules

• Never reuse field numbers in protobuf — even after deletion, reserve them.
• All RPCs must have a deadline/timeout. Server enforces it; client sets it.
• Always return google.rpc.Status for errors, never bare strings.
• Use RESOURCE_EXHAUSTED for rate limiting, not PERMISSION_DENIED.
• Always paginate list RPCs with cursor tokens, never offset/limit.
• Server-streaming calls must support a cancel mechanism at the application level.
• Deprecate fields with [deprecated = true], do not remove them until the next major package version.
• Client-streaming and bidi RPCs must handle client disconnect gracefully.
• Never use reflection in production.
• Always enable keep-alive pings on both client and server.
• Every interceptor must call the next handler exactly once.
• Use connection pooling for high-throughput scenarios.

References

• references/grpc-error-handling.md — gRPC Error Handling
• references/grpc-interceptors.md — gRPC Interceptors
• references/grpc-performance.md — gRPC Performance
• references/grpc-security.md — gRPC Security
• references/grpc-streaming.md — gRPC Streaming Patterns
• references/grpc-testing.md — gRPC Testing
• references/grpc-vs-rest.md — gRPC vs REST
• references/protobuf-basics.md — Protocol Buffer Basics

Handoff

No artifact produced unless requested. Next skill: backend-message-queue — if the service needs async communication or event-driven patterns. Next skill: backend-caching — if the service needs response caching or data store optimization. Carry forward: protobuf schemas, service definitions, auth requirements, deadline configurations.

Implementation Patterns

Observer Pattern for Event Handling

` interface EventObserver { onEvent(event: T): Promise; }

class EventBus { private observers: Set<EventObserver> = new Set(); subscribe(observer: EventObserver): void { this.observers.add(observer); } unsubscribe(observer: EventObserver): void { this.observers.delete(observer); } async emit(event: T): Promise { const results = Array.from(this.observers).map(o => o.onEvent(event)); await Promise.allSettled(results); } } `

Configuration-Driven Approach

config: defaults: timeout: 30s retryCount: 3 overrides: production: timeout: 60s retryCount: 5 development: timeout: 300s retryCount: 1

Production Considerations

Deployment Checklist

• Configuration validated against schema before startup
• Health check endpoints registered and monitored
• Graceful shutdown with draining period (30s timeout)
• Resource limits configured (CPU, memory, file descriptors)
• Log level set appropriate for environment
• Metrics endpoint secured and exposed
• Rate limiting configured per-tier
• TLS certificates valid and auto-renewing
• Database migrations run as separate deployment step
• Feature flags ready for gradual rollout

Monitoring and Alerting

Anti-Patterns

Performance Optimization

Caching Strategy

Cache hierarchy: L1 (in-memory local) → L2 (distributed Redis/Memcached) → L3 (CDN/Edge). Cache invalidation: TTL-based (simple, stale), event-based (complex, fresh), write-through (consistent, higher write latency), write-behind (fast writes, eventual consistency).

Resource Pooling

• Database connections: Pool of reusable connections (HikariCP, pgBouncer)
• HTTP connections: Keep-alive + connection pooling for external calls
• Thread pool: Bounded thread pools for async task execution

Profiling Methodology

1. Establish baseline with production traffic profile
2. Profile CPU with sampling profiler (pprof, perf, async-profiler)
3. Profile memory with heap dumps and allocation tracking
4. Profile I/O with strace/perf trace for syscall analysis
5. Profile latency with distributed tracing (OpenTelemetry)
6. Identify bottleneck, formulate hypothesis, implement fix
7. Re-profile to verify improvement, repeat

Security Considerations

Threat Modeling (STRIDE)

• Spoofing: Identity validation, authentication
• Tampering: Integrity checks, digital signatures
• Repudiation: Audit logs, non-repudiation
• Information disclosure: Encryption, access control
• Denial of service: Rate limiting, resource quotas
• Elevation of privilege: Principle of least privilege

Supply Chain Security

• Dependency scanning: Snyk, Dependabot, Trivy
• SBOM generation: CycloneDX or SPDX format
• Signed commits: GPG or SSH commit signing
• Artifact verification: Checksum validation, signature verification

Secrets Management

• Secrets never in code — always in secrets manager (Vault, AWS Secrets Manager)
• Rotation policy: Rotate database credentials every 90 days
• Access audit: Log every secrets access, alert on anomalies
• Encryption at rest and in transit for all secrets
• Principle of least privilege: each service gets only its own secrets

Rules

• Default-deny security posture — allow only explicitly required access.
• All inputs validated, all outputs encoded, all errors handled.
• Defend in depth — multiple layers of security controls.
• Fail securely — errors default to safe behavior.
• Log security-relevant events for audit and investigation.
• Keep dependencies updated — automate vulnerability scanning.
• Design for observability from day one, not as an afterthought.
• Document all architectural decisions with rationale.
• Review code for security, performance, and correctness before merging.