# Close Conformance Gaps

> drive the vendored Durable Streams conformance suite toward 332/332 when a run, workflow, or maintainer asks to close conformance gaps; treat the conformance suite as the source of truth, not the current server implementation

- Skill: `humanlayer/close-conformance-gaps` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add humanlayer/close-conformance-gaps`
- Raw SKILL.md: https://api.skillmd.com/api/skills/humanlayer/close-conformance-gaps/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- Author: humanlayer (https://skillmd.com/u/humanlayer)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/humanlayer/close-conformance-gaps

---


# Close Conformance Gaps

Use this skill when an automated run or a maintainer asks you to raise this
repo's Durable Streams conformance pass rate, implement a gated/skipped
conformance section, or fix a regression that turned a passing conformance test
red.

The goal is to make the server implementation satisfy more of the **vendored
conformance suite** without regressing any test that currently passes. The
conformance suite is the protocol's source of truth; the server is what must
adapt to it.

## Core Requirements

- The source of truth is the vendored suite at
  `packages/server-conformance-tests/src/index.ts`. Read the test that defines
  the behavior before changing the server. Never edit the conformance suite to
  make a test pass.
- Validate against the **Node host** (`bun run conformance:node`), which boots a
  server and runs the full black-box suite. CI cannot rely on the Durable Object
  host (it needs `wrangler dev`); note DO parity as follow-up rather than
  blocking on it.
- Prefer implementing protocol behavior **once in the core**
  (`packages/effect-durable-streams-server-core/src`) so every host benefits.
  Only touch a host's substrate Layer when the behavior is genuinely
  platform-specific.
- Never leave the default suite red. `bun run conformance:node` must still pass
  every test it passed before (currently 326/332). If you implement only a slice
  of a gated section, keep that section gated in `conformance/run.test.ts` so the
  default run stays green, and document the remaining work in your response.
- Land one reviewable unit per run — a coherent slice of a section, or a single
  regression fix — not a sprawling multi-feature change.

## Background: where the gaps are

The suite passes 326/332 on both hosts. The 6 non-passing tests are the
**`Reserved subscription APIs`** section
(`describe.runIf(options.subscriptions)` near the bottom of
`packages/server-conformance-tests/src/index.ts`), which exercises the reserved
`__ds/subscriptions/{id}` and `__ds/subscriptions/{id}/callback` webhook /
pull-wake endpoints. That section is gated off by `subscriptions: false` in
`conformance/run.test.ts`.

Two kinds of gap qualify as targets:

1. **A gated/unimplemented section** — implement a reviewable slice of it.
2. **A regression** — a test that used to pass now fails. Fixing it always
   takes priority over new feature work.

## Workflow

### 1. Establish the baseline

Run the full suite against the Node host and record the pass/fail/skip counts:

```bash
bun run conformance:node
```

If any test that should pass is failing, that regression is your target for this
run — skip ahead to step 4. Otherwise, the target is a gated section (the
subscription APIs).

Completion criterion: you know the current pass count and whether there is a
regression or you are working a gated section.

### 2. Read the conformance spec for your target

Open `packages/server-conformance-tests/src/index.ts` and read the exact tests
for your target section. Note every endpoint, header, status code, and body
shape they assert. These assertions are the contract.

For the subscription section, read the `Reserved subscription APIs` describe
block in full before writing any code.

Completion criterion: you can list the specific HTTP behaviors the failing tests
require.

### 3. Make the gated section runnable (first time only)

The section is gated by a literal `subscriptions: false` in
`conformance/run.test.ts`. To validate slices without forcing the whole gated
section green at once, make the gate env-driven (a small, reviewable change):

```ts
runConformanceTests({
  baseUrl,
  longPollTimeoutMs,
  subscriptions: process.env.CONFORMANCE_SUBSCRIPTIONS === "true",
})
```

This keeps the default `bun run conformance:node` behavior identical (section
stays gated) while letting you opt the section in during development. Only flip
the default to enabled once the **entire** section passes.

Completion criterion: you can run the target section on demand without changing
the default suite's behavior.

### 4. Implement the smallest correct slice in the core

Add or fix behavior in `packages/effect-durable-streams-server-core/src`:

- New routes belong in `src/http/router.ts`; engine logic in
  `src/stream-service.ts`; pure protocol helpers in `src/domain/*`.
- If the feature needs a new capability the host must provide (e.g. delivering a
  webhook), model it as a substrate Service tag under `src/services/` and add an
  in-memory implementation under `src/memory/`, then wire the **Node** host
  (`packages/node-durable-streams-server/src`) so CI can validate it. Note the
  Durable Object substrate as a follow-up if you don't implement it this run.
- Follow the existing Effect v4 idioms: typed `StreamError` channel mapped at the
  edge, Effects for fallible operations, no non-null assertions or unguarded
  `JSON.parse`.

Keep the slice small enough to review in one sitting.

Completion criterion: the slice compiles and targets the specific assertions you
listed in step 2.

### 5. Validate

Run, in order, the narrowest relevant check first:

```bash
# the target section (env-gated) against the Node host
CONFORMANCE_SUBSCRIPTIONS=true bun run conformance:node -t "Reserved subscription APIs"

# the full default suite — MUST stay green (no regressions)
bun run conformance:node

# typecheck every package and run core unit tests
bun run check
bun run test
```

If you only implemented a slice, the target section may still have failures —
that is acceptable as long as the default suite (step's second command) is fully
green and you report the remaining work. If the full section now passes, flip the
default `subscriptions` to enabled in `conformance/run.test.ts` and re-run
`bun run conformance:node` to confirm the new tests count as passing by default.

Completion criterion: the default suite passes with no regressions, `check` and
`test` pass, and you can state the exact new pass count or the precise blocker.

### 6. Format the response

When running as a CI agent, format your final response according to
`references/response-template.md`. This response becomes the PR body.

Completion criterion: the response follows the template and reports baseline →
new pass counts, what was implemented, and what remains.

## Review Checklist

- The conformance suite was read before the server was changed, and the suite
  itself was not edited to force a pass.
- Protocol behavior lives in the core unless it is genuinely platform-specific.
- `bun run conformance:node` still passes every test it passed before (no
  regression below the baseline).
- A partially-implemented gated section is left gated by default so the default
  run stays green; remaining work is documented.
- `bun run check` and `bun run test` pass.
- The diff is one reviewable slice or one regression fix, not a sprawl.
- The final response follows the response template with baseline → new counts.

## Anti-Patterns to Avoid

- Editing `packages/server-conformance-tests/src/index.ts` to make a test pass.
- Flipping `subscriptions: true` by default while the section still has failures
  (this turns the default suite red).
- Implementing a feature in a host substrate when it belongs in the core.
- Reintroducing non-null assertions, unguarded `JSON.parse`, or untyped error
  throws to take a shortcut.
- Bundling unrelated cleanup or refactors into a conformance PR.