Grpc Development

gRPC Development

This skill covers best practices for building gRPC-based services and APIs using Protocol Buffers, including service design, streaming patterns, interceptors, security, and observability.

Workflow for Building a gRPC Service

1. Define the service contract — Write .proto files with service definitions, RPC methods, and message types following the style and naming conventions below.
2. Generate language stubs — Run protoc with the appropriate language plugin (e.g., protoc-gen-go-grpc, grpcio-tools) to produce server and client code.
3. Implement the server — Create handler functions for each RPC method, register them with a gRPC server, and configure TLS, interceptors, and health checks.
4. Implement the client — Create a channel to the server, instantiate the generated client stub, and call RPC methods with proper deadlines and error handling.
5. Add interceptors — Wire in server and client interceptors for logging, authentication, metrics, and tracing.
6. Write tests — Unit-test handlers with mocked dependencies; integration-test with a real gRPC connection.
7. Deploy and observe — Enable distributed tracing (OpenTelemetry), structured logging, and metrics dashboards before going to production.

Core Principles

• gRPC uses Protocol Buffers as both its Interface Definition Language (IDL) and message interchange format
• Design services around the idea of defining methods that can be called remotely with their parameters and return types
• Prioritize type safety, performance, and backward compatibility
• Leave NO todos, placeholders, or missing pieces in the implementation

Protocol Buffer Best Practices

File Organization (1-1-1 Pattern)

• Structure definitions with one top-level entity (message, enum, or extension) per .proto file
• Correspond each .proto file to a single build rule
• This promotes small, modular proto definitions
• Benefits include simplified refactoring, improved build times, and smaller binary sizes

Message Design

• Use structured messages for extensibility - Protocol Buffers supports adding fields without breaking existing clients
• Be careful to use structs in places you may want to add fields later
• Don't re-use messages across RPCs - APIs may change over time, avoid coupling separate RPC calls tightly together
• Fields should always be independent of each other - don't have one field influence the semantic meaning of another

Field Guidelines

• Use descriptive field names with underscore_separated_names
• Reserve field numbers for deleted fields to prevent future conflicts
• Use optional for fields that may not always be present
• Consider using oneof when users need to choose between mutually exclusive options

Enum Best Practices

• Ensure the first value is always 0
• Use an "UNSPECIFIED" default value (e.g., STATUS_UNSPECIFIED = 0)
• Use prefixes to avoid naming collisions (e.g., ORDER_STATUS_CREATED vs STATUS_PENDING)
• Reserve enum values that are removed to prevent accidental reuse

Style Guidelines

• Keep line length to 80 characters
• Prefer double quotes for strings
• Package names should be in lowercase
• Use CamelCase (with initial capital) for message names
• Use underscore_separated_names for field names
• Use CamelCase for service and RPC method names

Service Design

RPC Patterns

• Unary RPC: Client sends single request, server responds with single response
• Server Streaming: Client sends request, server responds with stream of messages
• Client Streaming: Client sends stream of messages, server responds with single response
• Bidirectional Streaming: Both sides send streams of messages

Example: Proto Definition

syntax = "proto3";

package order.v1;

option go_package = "gen/order/v1;orderv1";

// OrderService manages customer orders.
service OrderService {
  // Creates a new order and returns the created resource.
  rpc CreateOrder(CreateOrderRequest) returns (CreateOrderResponse);
  // Streams real-time status updates for an order.
  rpc WatchOrder(WatchOrderRequest) returns (stream OrderStatus);
}

message CreateOrderRequest {
  string customer_id = 1;
  repeated OrderItem items = 2;
}

message CreateOrderResponse {
  string order_id = 1;
  OrderStatus status = 2;
}

message WatchOrderRequest {
  string order_id = 1;
}

message OrderItem {
  string product_id = 1;
  int32 quantity = 2;
}

message OrderStatus {
  string order_id = 1;
  OrderState state = 2;
  string updated_at = 3;
}

enum OrderState {
  ORDER_STATE_UNSPECIFIED = 0;
  ORDER_STATE_CREATED = 1;
  ORDER_STATE_PROCESSING = 2;
  ORDER_STATE_SHIPPED = 3;
  ORDER_STATE_DELIVERED = 4;
}

Example: Go Server Implementation

package main

import (
	"context"
	"log"
	"net"
	"time"

	"google.golang.org/grpc"
	"google.golang.org/grpc/codes"
	"google.golang.org/grpc/status"

	pb "example.com/gen/order/v1"
)

type orderServer struct {
	pb.UnimplementedOrderServiceServer
}

func (s *orderServer) CreateOrder(ctx context.Context, req *pb.CreateOrderRequest) (*pb.CreateOrderResponse, error) {
	if req.GetCustomerId() == "" {
		return nil, status.Error(codes.InvalidArgument, "customer_id is required")
	}
	orderID := "ord-" + time.Now().Format("20060102150405")
	return &pb.CreateOrderResponse{
		OrderId: orderID,
		Status: &pb.OrderStatus{
			OrderId: orderID,
			State:   pb.OrderState_ORDER_STATE_CREATED,
		},
	}, nil
}

func (s *orderServer) WatchOrder(req *pb.WatchOrderRequest, stream pb.OrderService_WatchOrderServer) error {
	for i, state := range []pb.OrderState{
		pb.OrderState_ORDER_STATE_PROCESSING,
		pb.OrderState_ORDER_STATE_SHIPPED,
		pb.OrderState_ORDER_STATE_DELIVERED,
	} {
		select {
		case <-stream.Context().Done():
			return stream.Context().Err()
		case <-time.After(time.Duration(i) * time.Second):
			if err := stream.Send(&pb.OrderStatus{
				OrderId:   req.GetOrderId(),
				State:     state,
				UpdatedAt: time.Now().Format(time.RFC3339),
			}); err != nil {
				return err
			}
		}
	}
	return nil
}

func main() {
	lis, err := net.Listen("tcp", ":50051")
	if err != nil {
		log.Fatalf("failed to listen: %v", err)
	}
	srv := grpc.NewServer(
		grpc.UnaryInterceptor(loggingUnaryInterceptor),
	)
	pb.RegisterOrderServiceServer(srv, &orderServer{})
	log.Println("serving on :50051")
	if err := srv.Serve(lis); err != nil {
		log.Fatalf("failed to serve: %v", err)
	}
}

// loggingUnaryInterceptor logs each unary RPC call.
func loggingUnaryInterceptor(
	ctx context.Context, req any, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler,
) (any, error) {
	start := time.Now()
	resp, err := handler(ctx, req)
	log.Printf("method=%s duration=%s err=%v", info.FullMethod, time.Since(start), err)
	return resp, err
}

API Design

• Design clear, intuitive service interfaces
• Group related methods in the same service
• Use meaningful method names that describe the action
• Document each RPC with comments describing behavior, parameters, and return values

Performance Optimization

Channel Management

• Reuse channels when working with gRPC
• Creating a gRPC channel is costly as it creates a new HTTP/2 connection
• Implement connection pooling for high-throughput scenarios
• Configure keepalive settings appropriately

Message Optimization

• Keep messages reasonably sized - large messages impact performance
• Consider streaming for large data transfers
• Use compression for bandwidth-constrained environments
• Avoid deeply nested message structures

Error Handling

Status Codes

• Use appropriate gRPC status codes (OK, INVALID_ARGUMENT, NOT_FOUND, etc.)
• Include meaningful error messages in status details
• Use rich error details for complex error scenarios
• Document expected error conditions in service definitions

Retry Logic

• Implement retry with exponential backoff for transient failures
• Use deadlines/timeouts for all RPC calls
• Handle UNAVAILABLE and RESOURCE_EXHAUSTED with retries
• Don't retry non-idempotent operations blindly

Security

Authentication

• Use TLS for transport security in production
• Implement per-RPC authentication using metadata/headers
• Support multiple authentication mechanisms (JWT, OAuth2, mTLS)
• Validate credentials on every request

Authorization

• Implement method-level access control
• Use interceptors for centralized authorization logic
• Validate all input data regardless of authentication status
• Follow the principle of least privilege

Interceptors and Middleware

Server Interceptors

• Use interceptors for cross-cutting concerns (logging, auth, metrics)
• Order interceptors carefully - execution order matters
• Keep interceptors focused on single responsibilities
• Handle errors gracefully within interceptors

Client Interceptors

• Add metadata (headers) for tracing and authentication
• Implement request/response logging
• Add automatic retry logic
• Collect client-side metrics

Testing

Unit Testing

• Mock gRPC services for isolated testing
• Test message serialization/deserialization
• Verify error handling paths
• Test interceptor logic independently

Integration Testing

• Test with real gRPC connections where possible
• Verify streaming behavior end-to-end
• Test timeout and cancellation scenarios
• Load test with realistic traffic patterns

Observability

Distributed Tracing

• Use OpenTelemetry for distributed tracing across service boundaries
• Propagate trace context in metadata
• Instrument both client and server sides
• Start spans for each RPC call

Metrics

• Track RPC latency histograms
• Monitor error rates by method and status code
• Count active connections and streams
• Alert on anomalies and SLA violations

Logging

• Use structured logging with consistent fields
• Log RPC method, duration, and status
• Include trace IDs for correlation
• Avoid logging sensitive data

Language-Specific Guidelines

Go

• Use the official google.golang.org/grpc package
• Implement services as interface types
• Use context for cancellation and deadlines
• Leverage code generation with protoc-gen-go-grpc

Python

• Use grpcio and grpcio-tools packages
• Implement async services with grpcio-aio for better concurrency
• Use type hints with generated stubs
• Handle blocking calls appropriately in async contexts

Node.js/TypeScript

• Use @grpc/grpc-js (pure JavaScript implementation)
• Consider using nice-grpc for better TypeScript support
• Leverage async/await patterns
• Use static codegen for type safety