Secure API

Secure & Performant API Skill

Apply this skill as a unified filter when writing or reviewing any code that makes or handles HTTP/WebSocket requests. Security and performance are treated as equal constraints, not trade-offs.

Quick self-check before any API output

If any answer is "no" or "not sure", fix it before returning the code.

1. Secrets and credentials

Rule: secrets never appear in source code, logs, or error messages.

// WRONG — key hardcoded
let client = VtClient::new("AIzaSyXXXXXXX");

// RIGHT — from environment at startup
let api_key = std::env::var("VT_API_KEY")
    .expect("VT_API_KEY must be set");

// WRONG — key in client bundle
const key = "abcd1234";

// RIGHT — only on server; never sent to frontend
const key = process.env.VT_API_KEY!;

Rules:

• Load secrets once at startup via env::var (Rust), process.env (Node), os.environ (Python).
• Never log the key itself — log only the first 4 chars for traceability: key[..4].
• Never include secrets in error responses or stack traces.
• Rotate keys: treat them as ephemeral. Use short TTL tokens when the API supports it.
• In .env files: add .env to .gitignore before the first commit, not after.

→ For key management patterns by language, see references/secrets.md.

2. Authentication

Rule: use the minimum scope required. Validate tokens server-side on every request.

// Bearer token — attach to every request, never store in cookies without Secure+HttpOnly
let response = client
    .get(url)
    .header("Authorization", format!("Bearer {}", token))
    .send()
    .await?;

Rules:

• Prefer short-lived tokens (OAuth2 access tokens) over long-lived API keys when available.
• Never send tokens in query strings (they appear in server logs and browser history).
• For server-to-server: use service accounts or managed identities (Azure MSI, AWS IAM roles).
• Validate JWTs: check exp, iss, aud — never trust unverified claims.
• Scope: request only the permissions the operation actually needs.

3. Input validation and output trust

Rule: validate everything that crosses a trust boundary — both inbound requests and API responses.

// Validating inbound request body
#[derive(Deserialize, Validate)]
struct EnrichRequest {
    #[validate(ip)]         // rejects non-IP strings
    ip: String,
    #[validate(range(min = 1, max = 65535))]
    port: Option<u16>,
}

// Validating API response before use
let body: VtResponse = response.json().await
    .map_err(|e| AppError::InvalidResponse(e.to_string()))?;

// Never do: trust an arbitrary field from external API without checking
let score = body.data.attributes.last_analysis_stats.malicious; // OK — typed
let raw: serde_json::Value = response.json().await?;
let score = raw["data"]["attributes"]["malicious"].as_u64(); // risky — None silently

Rules:

• Use typed deserialization (Rust serde, TS zod, Python pydantic) — never raw Value/any/dict in production paths.
• Reject requests with unexpected fields (deny_unknown_fields in serde, strict() in zod).
• Sanitize strings before interpolating into SQL, shell commands, or log messages.
• Treat every external API response as untrusted until deserialized into a validated type.

→ See references/validation.md for patterns per language.

4. Error handling — what to expose, what to hide

Rule: errors shown to clients must never contain secrets, stack traces, or internal paths.

// WRONG — leaks internal detail
return Err(format!("DB error: {}", db_err)); // db_err may contain query + credentials

// RIGHT — log internally, return generic message externally
tracing::error!(error = %db_err, "database query failed");
return Err(AppError::Internal); // maps to HTTP 500 with generic body

// WRONG — leaks which API key was used
throw new Error(`VT request failed with key ${apiKey}: ${err.message}`);

// RIGHT
logger.error({ err, keyPrefix: apiKey.slice(0, 4) }, "VT enrichment failed");
throw new ApiError(503, "Enrichment temporarily unavailable");

Rules:

• Log full errors internally (with tracing/structured logs), return only a code + generic message externally.
• Use typed error enums — never anyhow::Error as a direct HTTP response body.
• Include a request_id in responses so users can report errors without you exposing internals.

5. Rate limiting and retry strategy

Rule: every external API call must have a timeout, a retry budget, and a concurrency cap.

// Retry with exponential backoff — using the `backoff` crate
use backoff::{ExponentialBackoff, future::retry};

async fn call_with_retry(client: &Client, url: &str) -> Result<Response> {
    retry(ExponentialBackoff::default(), || async {
        client
            .get(url)
            .timeout(Duration::from_secs(10))   // explicit timeout always
            .send()
            .await
            .map_err(backoff::Error::transient)
    })
    .await
}

// Concurrency cap with semaphore — avoid hammering the API
const sem = new Semaphore(10); // max 10 concurrent requests

async function enrichBatch(ips: string[]): Promise<EnrichResult[]> {
    return Promise.all(ips.map(ip =>
        sem.use(() => enrichOne(ip))
    ));
}

Rules:

• Always set an explicit timeout — never rely on the default (often infinite).
• Retry only on transient errors (5xx, network timeout, 429). Never retry 4xx.
• On 429 (rate limit): respect Retry-After header if present; otherwise exponential backoff.
• Cap concurrency: Semaphore(n) where n = API's documented rate limit / safety factor of 2.
• Circuit breaker: after N consecutive failures, stop sending requests for T seconds.

→ See references/retry-patterns.md for full patterns in Rust, TypeScript, and Python.

6. Payload size and compression

Rule: send and receive the minimum data needed. Enable compression.

// Request compression
let response = client
    .get(url)
    .header("Accept-Encoding", "gzip, br")  // ask for compressed response
    .send()
    .await?;

// Selective fields — if the API supports field projection
let url = format!("https://www.virustotal.com/api/v3/ip_addresses/{}?fields=last_analysis_stats,country,asn", ip);

Rules for API server (exposing endpoints):

• Partition large payloads: never return the full graph in one response. Split by type (IPs, domains, hashes) and load on demand.
• Support field projection (?fields=id,severity) so clients fetch only what they render.
• Paginate: default page size ≤ 100 items. Always include next_cursor or next_page.
• Enable gzip/brotli compression on responses > 1 KB.
• Return ETag on GET responses so clients can use If-None-Match for 304 Not Modified.

Payload partitioning pattern (your IOC graph case):

GET /api/graph/ips       → nodes type=ip only (~50 KB for 10k nodes)
GET /api/graph/domains   → nodes type=domain (background load)
GET /api/graph/hashes    → nodes type=hash (background load)
GET /api/ioc/:id         → full detail for one IOC (on-demand)

7. Caching strategy

Rule: cache at the right layer with the right TTL. Never cache secrets or PII.

// Stale-while-revalidate pattern
async function getWithCache<T>(key: string, fetcher: () => Promise<T>, ttlMs: number): Promise<T> {
    const cached = await idb.get(key);

    if (cached) {
        // return immediately, revalidate in background
        fetcher().then(fresh => idb.set(key, { data: fresh, cachedAt: Date.now() }));
        return cached.data;
    }

    // no cache — must wait
    const fresh = await fetcher();
    await idb.set(key, { data: fresh, cachedAt: Date.now() });
    return fresh;
}

Rules:

• Use IndexedDB for client-side cache > 500 KB (localStorage blocks the main thread and has a 5 MB hard cap).
• Always store cachedAt alongside data — never cache without a TTL check on read.
• ETag flow: store the ETag from the response; send If-None-Match on the next request; on 304, use the cached data as-is.
• Never cache: auth tokens on disk, API keys anywhere, user PII beyond session.

8. HTTPS and transport security

Rules:

• Never disable TLS verification — danger_accept_invalid_certs(true) / verify=False is never acceptable in production code.
• Pin certificates only if the API documentation explicitly recommends it.
• Use TLS 1.2 minimum; prefer TLS 1.3.
• For internal service-to-service (Fluent Bit → SOC), use mutual TLS or a shared key — never plain HTTP on a "private" network.

// WRONG — disables cert verification
let client = Client::builder()
    .danger_accept_invalid_certs(true)
    .build()?;

// RIGHT — strict by default (reqwest default), explicit for clarity
let client = Client::builder()
    .tls_built_in_root_certs(true)
    .build()?;

9. Logging and observability

Rule: log enough to debug, not enough to leak.

// Structured log — machine-parseable, no secrets
tracing::info!(
    ip = %ioc.ip,
    region = %ioc.region,
    score = vt_score,
    duration_ms = elapsed.as_millis(),
    "IOC enrichment complete"
);

// WRONG — logs the full response (may contain user data or keys)
tracing::debug!("VT response: {:?}", full_response);

// RIGHT — log only what you need to debug
tracing::debug!(
    malicious = stats.malicious,
    suspicious = stats.suspicious,
    "VT analysis stats"
);

Rules:

• Use structured logging (tracing in Rust, pino/winston in Node, structlog in Python).
• Never log full request/response bodies in production — log only derived metrics.
• Include request_id, duration_ms, status_code on every outbound API call.
• Redact headers before logging: Authorization, X-Api-Key, Cookie.

10. Language-specific references

Read the relevant reference file when implementing in a specific language/context: