Node HTTP Proxy Ttfb Timeout

Node.js HTTP Proxy TTFB Timeout (Keep-Alive Safe)

Problem

When implementing HTTP proxy/reverse-proxy code with TTFB (time-to-first-byte) timeouts, socket-based timeout logic fails on keep-alive connections. The first request works fine, but subsequent requests on reused sockets hang forever even though the upstream is dead.

Context / Trigger Conditions

• First HTTP request through proxy succeeds
• Second request (on same keep-alive connection) hangs indefinitely
• Upstream is a tunnel (cloudflared, ngrok) or service that can go offline
• Using socket.setTimeout() or proxyReq.on('socket', ...) for timeout logic
• Timeout callback never fires for reused connections
• Half-open TCP connections (remote closed, local doesn't know)

Root Cause

Socket-based timeouts have race conditions with keep-alive connections:

1. Fresh connection: socket.connecting === true, timer starts on 'connect' event ✓
2. Reused connection: socket.connecting === false, timer starts immediately BUT the response callback may already be pending, causing a race
3. Half-open sockets: TCP connection is open locally but remote closed silently. Writes succeed but reads hang forever. Socket-level checks don't detect this.

Solution

Use request.setTimeout() instead of socket-based timers. This works at the HTTP layer and fires reliably for both fresh and reused connections.

Pattern: Two-Phase Timeout

const TTFB_TIMEOUT = 8_000;     // 8s to get response headers
const BODY_TIMEOUT = 45_000;    // 45s for streaming body

const proxyReq = https.request(targetUrl, options, (proxyRes) => {
  // SUCCESS - extend timeout for body streaming
  proxyReq.setTimeout(BODY_TIMEOUT);

  // Handle response...
  proxyRes.pipe(res);
});

proxyReq.on('error', (err) => {
  // Handle connection/timeout errors
  console.error('Request failed:', err.message);
});

// TTFB timeout: abort if no response headers within 8s
// Works reliably for both fresh and keep-alive connections
proxyReq.setTimeout(TTFB_TIMEOUT, () => {
  console.error('TTFB timeout - upstream unreachable');
  proxyReq.destroy(new Error('TTFB timeout'));
});

proxyReq.write(body);
proxyReq.end();

What NOT to Do (Anti-Pattern)

// DON'T: Socket-based timeout - fails on keep-alive connections
proxyReq.on('socket', (socket) => {
  if (socket.connecting) {
    socket.setTimeout(CONNECT_TIMEOUT, () => { /* ... */ });
    socket.once('connect', () => {
      // Start TTFB timer after connect
      const ttfbTimer = setTimeout(() => { /* ... */ }, TTFB_TIMEOUT);
    });
  } else {
    // Reused socket - this branch races with response callback!
    const ttfbTimer = setTimeout(() => { /* ... */ }, TTFB_TIMEOUT);
  }
});

Verification

1. Start upstream service, send request → should succeed
2. Send second request immediately → should succeed (keep-alive reuse)
3. Kill upstream, send request → should timeout in TTFB_TIMEOUT, not hang forever
4. Restart upstream, send request → should succeed

Example: Complete Proxy Implementation

function tryProvider(targetUrl, body) {
  return new Promise((resolve, reject) => {
    let settled = false;

    const proxyReq = https.request(targetUrl, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
    }, (proxyRes) => {
      if (settled) return;

      // Extend timeout for body streaming
      proxyReq.setTimeout(45_000);
      settled = true;
      resolve(proxyRes);
    });

    proxyReq.on('error', (err) => {
      if (settled) return;
      settled = true;
      reject(err);
    });

    // TTFB timeout - catches "CDN up, origin dead" scenarios
    proxyReq.setTimeout(8_000, () => {
      if (!settled) {
        proxyReq.destroy(new Error('TTFB timeout'));
      }
    });

    proxyReq.write(body);
    proxyReq.end();
  });
}

Notes

• request.setTimeout() resets the timer on each data event, making it suitable for both TTFB detection and stalled-body detection
• The settled guard prevents double-resolution in timeout vs response races
• For circuit breaker patterns, combine with health probes that pre-emptively open the circuit when the upstream is detected as down
• Keep-alive connections are the default in Node.js (via http.Agent). The issue manifests when the upstream dies while connections are pooled.

Related Patterns

• Health Probe: Run periodic checks against upstream to detect failures early
• Circuit Breaker: After N failures, skip the dead upstream entirely
• Connection Pool Tuning: Reduce agent.keepAlive timeout to match upstream

References

• Node.js HTTP Documentation
• A Complete Guide to Timeouts in Node.js | Better Stack
• How to Use Timeouts in Node.js | AppSignal Blog
• Tuning HTTP Keep-Alive in Node.js