Webstatus Backend

webstatus-backend

This skill provides guidance for developing the Go-based backend API for webstatus.dev.

Core Components

• HTTP Server (backend/pkg/httpserver): Handles routing and requests via oapi-codegen stubs.
• Storage Adapter (lib/gcpspanner/spanneradapters): Translates API types to database types.
• Spanner Client (lib/gcpspanner): Core logic for Spanner interactions using the Mapper pattern.
• Valkey Cache (lib/valkeycache): Isolated via Private Service Connect (PSC) for secure internal access.

Architecture

For a technical deep-dive into the backend implementation patterns, request flows, and auth middleware, see references/architecture.md.

Guides

• Add a New API Endpoint: Mandatory spec-first process.
• Spanner Mapper Pattern: How to use the generic entity helpers.
• Spanner Best Practices: Efficient and safe querying.
• Shared Libraries & Utilities: Guidelines for lib/, util/, and the OptionallySet pattern.

Architectural Patterns: Abstraction & Adapters

We use a Hexagonal-style Adapter Pattern to decouple application logic from infrastructure.

• Ports: Interfaces should be defined in the application package (e.g., pkg/sender).
• Adapters: Implementations live in lib/ (e.g., lib/gcpspanner/spanneradapters or lib/valkeycache).
• Why: This enables painless unit testing using mock adapters and ensures that swapping external services (e.g., Valey to Redis) doesn't leak into business logic.

Error Handling & Isolation Patterns

• Centralized Error Interpreter: Use a dedicated function in a leaf package (e.g., lib/backendtypes) to map raw internal errors to safe, static sentinel errors before they cross boundaries. This prevents dynamic context or database details from leaking to presentation layers.
• Decoupled Schema Types: Define identical-looking local types inside versioned schemas (e.g., v1.QueryError in blobtypes) instead of using internal types directly. This maintain boundary isolation and prevents changes in the backend from breaking saved logs or deliveries.
• Orchestrator Data Ownership: To prevent leaking translation logic into isolated schema interfaces (e.g. adding getters for errors), the orchestrator should hold and compare the data directly if it already possesses it (e.g. in executionData).
• Exhaustive Enum Conversion: When mapping enums between packages, ALWAYS use an exhaustive switch case instead of direct type casting. This ensures that all valid source enum values are explicitly handled and mapped to valid destination enum values, preventing unintended values from leaking across package boundaries.
• Structured Types Across Boundaries: Avoid flattening structured objects (like slices of structs) into primitive types (like slices of strings) when passing data across package boundaries or returning from interfaces (e.g., Load methods). Using structured types allows for future extension and maintains type safety.
• Explicit Presence for Collections: When dealing with collections (like slices of errors) in schemas or state, use a wrapper like generic.OptionallySet[[]T] instead of relying on nil or empty slices to distinguish between "empty but validly checked" and "not set/not applicable". This ensures explicit intent and avoids ambiguity.

General Do's and Don'ts

• DO cross-reference all code against the official Google Go Style Guide. If you are unsure about a specific style rule, DO NOT assume; you MUST ask the user for clarification.
• DO use spanneradapters for DB interactions in the API.
• DON'T call gcpspanner.Client directly from httpserver handlers.
• DO use row.ToStruct(&yourStruct) instead of manual column scanning.
• DO define new Spanner table structs and query logic within lib/gcpspanner.
• DO update FeaturesSearchVisitor.go when adding new filter terms to the search grammar.
• DO use Canonical Transport Types from lib/workertypes for any data crossing service boundaries (e.g. results sent to Pub/Sub).
• DO write integration tests using testcontainers-go for any changes to the lib/gcpspanner layer.
• DO add response caching for new read-only endpoints in backend/pkg/httpserver/cache.go.
• DON'T import lib/backendtypes into lib/gcpspanner (prevents circular dependencies), and DON'T import lib/gcpspanner into lib/backendtypes (prevents architectural layer inversion).
• DO place all Pub/Sub message payloads in dedicated versioned packages under lib/event/<name>/<version>/types.go, implement event.Event (Kind(), APIVersion()), and publish via event.New(evt).
• DO handle business key to internal ID translation inside the gcpspanner client.
• DO ensure Merge functions in mappers copy ALL fields, including UpdatedAt.
• DO use ...WithTransaction variants of helpers when inside a ReadWriteTransaction.
• DO call eventPublisher.PublishSearchConfigurationChanged in handlers that modify user saved searches to trigger immediate notification dispatcher updates.
• DO define noAuth: under components.securitySchemes in openapi.yaml whenever any operation specifies noAuth: [] in its security: requirements. In oapi-codegen v2.7+, <Scheme>ContextKey types (noAuthContextKey) are generated strictly from components.securitySchemes; omitting noAuth causes undefined: noAuthContextKey build failures.
• DO pass pointers (*string) when populating optional OpenAPI response header fields (like Location in 301 responses), as oapi-codegen v2.7+ models optional response headers as pointer types.
• DO type strict middleware closures using backend.StrictHandlerFunc and backend.StrictMiddlewareFunc directly from the generated package rather than importing from github.com/oapi-codegen/runtime/strictmiddleware/nethttp.
• DO use modern Go 1.26+ new(expr) built-in syntax (e.g., new("my-string") or new(42)) when creating pointers to values or literals. DON'T introduce custom pointer helper functions (e.g., stringPtr, intPtr, or generic ptr(...)).
• DO resolve entity keys (e.g., FeatureKey -> WebFeatureID) upfront and seek directly on secondary indexes (e.g., @{FORCE_INDEX=MetricsFeatureChannelBrowserTime}) when querying high-volume timeseries tables (WPTRunFeatureMetrics, DailyChromiumHistogramMetrics). Never perform an unindexed driving join from parent tables across multi-year timeseries ranges.
• DO match Go parameter types directly to Spanner column types (civil.Date for DATE, time.Time for TIMESTAMP) so SQL query predicates remain strictly sargable without wrapping table columns in SQL conversion functions (e.g., avoid TIMESTAMP(dchm.Day)).

Testing & Linting

• Precommit Suite: Run make precommit to execute the full suite of Go tests, formatting, and linting.
• Linting: Run make go-lint to lint all Go code using golangci-lint.
  • goconst & Spanner Queries: In .golangci.yaml, goconst is explicitly excluded for lib/gcpspanner/.*\.go (linters.exclusions.rules). Because lib/gcpspanner consists of dozens of files whose sole job is executing SQL queries where parameter map keys (map[string]any{"startAt": startAt}) naturally repeat across methods to match raw SQL parameters (@startAt), excluding goconst on this path eliminates false positives cleanly without requiring a brittle ignore-string-values whitelist or forcing DRY constants across independent queries.
  • For all other packages outside lib/gcpspanner (e.g. workers, API handlers), repeating magic strings ("type", "text", error messages) MUST be declared as package constants to satisfy goconst.
• Quick Test Iteration: Because this project uses a multi-module workspace (go.work), to run tests quickly for a single package without running the whole suite, execute go test from within the specific module directory, or provide the full module path:

  cd backend && go test -v ./pkg/...
  # Or
  go test -v github.com/GoogleChrome/webstatus.dev/lib/gcpspanner/...
  

• Integration Tests: Any changes to lib/gcpspanner MUST include integration tests using testcontainers-go against the Spanner emulator.

Documentation Updates

When making significant architectural changes, adding new major endpoints, or altering the database schema:

• Trigger the "Updating the Knowledge Base" prompt in GEMINI.md to ensure I am aware of the changes.
• Update docs/ARCHITECTURE.md if the system boundaries change.
• Update these very skills files if you introduce new established patterns.