Characterization Tests
A regular test fails when behavior is wrong; a characterization test fails when behavior is different. Record what the system does — bugs included — and pin it as the expected value. Correctness is the information the legacy system lost; observed behavior is the only surviving specification, and downstream consumers may depend on the bugs.
Characterization tests are system tests: black-box, against the running system, from the outside — the system-tests skill owns that contract. The only difference from ordinary system tests is the origin of the expectations — recorded, not specified.
Surfaces
The observable surface is a property of the system, not of this skill. Inventory all of them:
| Surface |
Stimulus → Observation |
Mechanics |
| HTTP (REST, server-rendered pages) |
request → response |
references/http.md |
| Messaging (JMS, queues) |
message in → message(s) out |
same discipline; document per system |
| Batch / file exchange |
input file → output file, report |
same discipline; document per system |
| CLI |
invocation → exit code, output |
same discipline; document per system |
| Database side effects |
any stimulus → state delta |
record only when no outer surface shows the effect |
HTTP mechanics are bundled; for other surfaces apply the identical record/normalize/replay discipline and document the surface mechanics next to the recordings. Prefer the outermost surface that shows the behavior — assert database state only when nothing above it does.
Record
- Inventory the system's surfaces and their entry points (JAX-RS resources,
web.xml mappings, queue names, batch jobs, and the external contracts in migration/CONCEPTS.md when present).
- Ask the user: target coordinates (base URL, broker, job trigger — never guess), environment, credentials, and which business flows matter most. Confirm the target is not an unprotected production system.
- Observe-only by default: record stimuli without side effects. Record mutating stimuli (writes, consumed messages, batch runs) only after explicit confirmation, only against a disposable environment with resettable seed data.
- Execute each stimulus, normalize the observation per references/recording-format.md and the surface reference, store one file per interaction under
migration/characterization/recordings/.
- Each recording must replay in isolation — note seed-data assumptions in its header.
Replay
- Ask for the target coordinates of the migrated system.
- Re-execute every recording; normalize the observation with the same rules; compare against the stored expectation.
- Write
migration/characterization/REPORT.md: one verdict per recording (same | different with diff excerpt | unreachable), plus summary counts. Every recording appears in the report — skipped is a verdict, not an omission.
- For repeated replays (CI, per-carving checks), generate a replay suite following the composed stack skill's system-test conventions — an
-st module with JUnit 5 for microprofile-server targets, zunit for java-cli-app, a zero-dependency single-file Java 25 runner when no stack skill is composed.
Reconcile
A diff is a question, not a verdict: behavior changed — intended?
- Regression → the migrated system is fixed; the recording stays.
- Accepted change (e.g. a bug deliberately fixed) → update the recording and append a decision line to its header:
accepted: <name>, <YYYY-MM-DD> — <reason>.
Never update a recording silently, and never decide yourself — acceptance is a human call, same rule as the clarifier.
Rules
- Record actual observations, including apparently wrong ones — correctness judgments happen at reconcile time, by a human.
- Normalize before storing and before comparing; never diff raw observations.
- The suite encodes what is, not what should be — it is scaffolding with an expiry: retire recordings BC by BC as sbce specs and ears-tests encode intent, delete the rest with
migration/.
- Recording exercises a real system: respect rate limits, stop on unexpected error storms, and never authenticate with credentials the user has not explicitly provided for this purpose.
1---2name: characterization-tests3description: Pin the observed behavior of a running legacy system as replayable golden masters — record normalized stimulus/observation pairs into migration/characterization/, replay them against the lifted or re-architected system, report behavior diffs. Owns the record/replay/reconcile discipline; the observable surface (HTTP, messaging, batch files, CLI, database state) is system-dependent. Characterization tests are system tests whose expectations are recorded, not specified — composes with system-tests (contract) and the stack skill (test syntax). Brackets every code-changing migration step; retires as sbce specs and ears-tests take over. Invoke explicitly as /characterization-tests record|replay. Not for spec-derived tests — use ears-tests.4---56# Characterization Tests78A regular test fails when behavior is wrong; a characterization test fails when behavior is *different*. Record what the system does — bugs included — and pin it as the expected value. Correctness is the information the legacy system lost; observed behavior is the only surviving specification, and downstream consumers may depend on the bugs.910Characterization tests are **system tests**: black-box, against the running system, from the outside — the system-tests skill owns that contract. The only difference from ordinary system tests is the origin of the expectations — recorded, not specified.1112## Surfaces1314The observable surface is a property of the system, not of this skill. Inventory all of them:1516| Surface | Stimulus → Observation | Mechanics |17|---|---|---|18| HTTP (REST, server-rendered pages) | request → response | [references/http.md](references/http.md) |19| Messaging (JMS, queues) | message in → message(s) out | same discipline; document per system |20| Batch / file exchange | input file → output file, report | same discipline; document per system |21| CLI | invocation → exit code, output | same discipline; document per system |22| Database side effects | any stimulus → state delta | record only when no outer surface shows the effect |2324HTTP mechanics are bundled; for other surfaces apply the identical record/normalize/replay discipline and document the surface mechanics next to the recordings. Prefer the outermost surface that shows the behavior — assert database state only when nothing above it does.2526## Record27281. Inventory the system's surfaces and their entry points (JAX-RS resources, `web.xml` mappings, queue names, batch jobs, and the external contracts in `migration/CONCEPTS.md` when present).292. Ask the user: target coordinates (base URL, broker, job trigger — never guess), environment, credentials, and which business flows matter most. Confirm the target is not an unprotected production system.303. Observe-only by default: record stimuli without side effects. Record mutating stimuli (writes, consumed messages, batch runs) only after explicit confirmation, only against a disposable environment with resettable seed data.314. Execute each stimulus, normalize the observation per [references/recording-format.md](references/recording-format.md) and the surface reference, store one file per interaction under `migration/characterization/recordings/`.325. Each recording must replay in isolation — note seed-data assumptions in its header.3334## Replay35361. Ask for the target coordinates of the migrated system.372. Re-execute every recording; normalize the observation with the same rules; compare against the stored expectation.383. Write `migration/characterization/REPORT.md`: one verdict per recording (`same` | `different` with diff excerpt | `unreachable`), plus summary counts. Every recording appears in the report — skipped is a verdict, not an omission.394. For repeated replays (CI, per-carving checks), generate a replay suite following the composed stack skill's system-test conventions — an `-st` module with JUnit 5 for microprofile-server targets, zunit for java-cli-app, a zero-dependency single-file Java 25 runner when no stack skill is composed.4041## Reconcile4243A diff is a question, not a verdict: behavior changed — intended?4445- **Regression** → the migrated system is fixed; the recording stays.46- **Accepted change** (e.g. a bug deliberately fixed) → update the recording and append a decision line to its header: `accepted: <name>, <YYYY-MM-DD> — <reason>`.4748Never update a recording silently, and never decide yourself — acceptance is a human call, same rule as the clarifier.4950## Rules5152- Record actual observations, including apparently wrong ones — correctness judgments happen at reconcile time, by a human.53- Normalize before storing and before comparing; never diff raw observations.54- The suite encodes what *is*, not what *should be* — it is scaffolding with an expiry: retire recordings BC by BC as sbce specs and ears-tests encode intent, delete the rest with `migration/`.55- Recording exercises a real system: respect rate limits, stop on unexpected error storms, and never authenticate with credentials the user has not explicitly provided for this purpose.
AdamBien