API Mapper
A web application already speaks a private API to itself. Every button click is an HTTP request. If you watch those requests closely enough, you can work out what the API is: which endpoints exist, what each one takes, which of them read and which of them write, and what each one demands before it will answer.
That reconstruction is the output. It is a map, one per domain, and it accumulates. Learning to create a product is one row in a map of Shopify that can also hold search, list, read, update, and delete. Firing one of those rows headlessly is a use of the map, not the point of it.
The browser is a teacher, not an interface. You drive it to learn an endpoint, then you do not touch it again for that endpoint.
Markdown holds judgment. Python holds determinism.
What it needs
Python 3.11 or newer, Playwright with Chromium, network access to the target,
and a Chrome profile directory where a human has already signed in. Install with
make install and make browser; scripts/bootstrap_profile.py opens a headed
window for the signing in, which this skill never does itself.
make demo runs the whole pipeline against a fixture app on loopback, needing
none of the above beyond the install, and is the fastest way to see what the
output looks like before pointing it at anything real.
Two constraints matter to you specifically, because you have no terminal.
capture.py prompts on stdin by default, so pass --steps when a steps file
for the action exists, and --wait-for sentinel or --wait-for close when one
does not, so the operator and you can be separate sessions. And auth probing a
write fires the endpoint three more times, which is refused without
--allow-writes.
The unit of output
One map entry per endpoint. Each entry carries:
| Field | What it says |
|---|---|
method and url |
Where the endpoint is, and how it is addressed |
parameters |
What the caller supplies, with types |
class |
read, write, or unknown, decided by evidence, never assumed |
auth |
public, requires_any_auth, or requires_fresh_auth, established by probing |
status |
verified or ui_only, with a reason either way |
last_verified_at |
When the entry was last shown to still work |
sequence |
The classified steps, the same shape a single template has |
A single learned action produces one of those entries. The map is built from templates, so the single template output stays exactly as it is; the map is what holds many of them together and keeps them current.
An endpoint that could not be verified is recorded with the reason, not dropped. A map that marks what it could not reach is more useful than one that pretends full coverage.
Division of labour
Some parts of this job have exactly one correct answer given the same inputs.
Diffing two captures, extracting a schema, comparing a replay against an
expectation, deciding whether a response has the shape it had last time. Those
live in scripts/. A script gets the same answer every time and can be tested
offline, which is what makes the result trustworthy.
Other parts need context that is not in the data. Which action to perform, what a meaningfully different second input looks like, whether the page visibly showed success, whether three requests are one logical sequence or three unrelated ones, which endpoints are worth mapping at all. Those stay with you.
Never do by hand what a script already does. Do not eyeball two capture
files to spot the difference; run diff.py. Do not skim a capture for the
interesting request; run reduce.py. Do not decide by eye whether a POST is a
read; run classify.py. Hand analysis is where quiet errors enter, and a wrong
template that returns 200 is worse than no template, because it looks like it
works. Your job at each step is to supply the judgment the script cannot, then
read what it produced and decide.
What is automated, and what is not
Three different things get confused with one another, so this states each one plainly. Getting the line right matters more than where the line sits, because a skill that overstates what it does produces work nobody can trust.
Discovery is automated. Everything downstream of the capture: separating signal from noise, classifying every field, deciding read or write, replaying, probing what the server actually requires, keeping the map current. That is what the scripts do, and they do it the same way every time.
Demonstrating the action is scripted once, by a human. The action still has
to be performed in a browser for there to be traffic to observe. Someone has to
say what performing it means. They can do that live, at the browser, which is
the default; or they can write it down once as a steps file, and capture.py --steps performs it on every run afterwards. Either way a person decided what
the action is. The steps file just means they decide it once instead of once per
run, which is what makes run B, run C, and every later re-verification cheap.
Inferring the steps from a goal is out of scope. Nothing here turns "create a product" into a list of fields to fill and a button to press. That is a separate capability with a separate failure mode, and pretending to have it would be the worst kind of overreach for this skill: the output of a wrong inference is not an error, it is a clean capture of an action nobody asked for, which diffs cleanly, templates cleanly, verifies cleanly, and is wrong.
The same reasoning runs one level down, inside the steps file. A target is described in plain language and resolved against the live page, and resolution refuses to guess: zero matches stops the run, and so does more than one, with the candidates printed so the human can sharpen the description or scope it to a container. An ambiguous target is a steps file problem for a person to fix, not a coin flip for the agent to take.
Step 0, check for a public API first
Before capturing anything, check whether a documented API already covers this surface and whether the user can actually reach it. Look for developer docs, an OpenAPI or GraphQL schema, an official SDK, or a documented webhook.
If a public API covers the work and the user has access, use it and stop here. A documented endpoint has a stability contract; a private one does not. Mapping a private API when a public one exists buys you nothing and costs you a maintenance burden the vendor will not warn you about.
Continue only after you have stated out loud which of these applies:
- No public API exists for this product.
- A public API exists but does not cover what is needed.
- A public API covers it but access is gated, by approval, pricing tier, a waitlist, or an enterprise contract the user does not have.
Say which one, and why, before you capture. If none applies, do not proceed.
Preconditions
All five must hold. If one does not, stop and say which.
The operator is entitled to automate this target. Ask, once, before anything else, and record the answer in what you report. Having a session is not the answer: it proves a product granted access to a person, and says nothing about what that person may delegate to a script. Many products separate the two in their terms and some prohibit automated access outright, so this is the operator's call and never yours to infer from the presence of a profile. If the answer is no, or nobody can give one, stop. The same rule appears again under Refusing to map, where it arrives later and after the captures have been taken.
An authenticated browser session exists. The operator is already logged in,
in a Chrome profile directory you can point Playwright at. This skill never
handles credentials and never performs a login. It reuses a session that a human
established, which keeps passwords, MFA, and session establishment entirely
outside this system. If no such profile exists yet, scripts/bootstrap_profile.py
opens one headed so a human can sign in once; it waits the same three ways
capture.py does, so you can start it and let someone else do the typing.
The target is non production. A sandbox, a staging tenant, a test account, a throwaway workspace. You will be sending the same write twice at minimum, once per capture run, more during verification, and three more times if you auth probe it. Doing that against real data is destructive.
A success oracle exists for every write you intend to map. An independent way to confirm the effect actually happened: a list endpoint the new record appears in, a status field that flipped, a count that incremented. Without an oracle, verifying a write collapses into checking a status code, and a status code proves only that a server answered. It does not prove the record was created, the setting was saved, or the message was sent. GraphQL makes this sharper, because it returns 200 for failures and buries the error in the response body. An oracle is what separates a verified write from a hopeful one. Reads need no oracle; they are verified by structural equivalence instead, which is covered in step 7.
The action is repeatable with different inputs. You need at least two runs with meaningfully different inputs to tell a parameter from a constant. A one-off action that cannot be performed twice cannot be learned, because there is nothing to diff.
Workflow
Ten steps, run once per endpoint you are adding to the map. Each names its script and how it fails.
1. Check the map
scripts/registry.py map check DOMAIN
scripts/registry.py check DOMAIN [ACTION]
If a verified entry for this endpoint already exists and its integrity hash still matches, use it and stop. You are done.
If an entry exists but the integrity hash has drifted, the upstream operation
changed, for example a new persisted query hash or a new bundle version.
Relearn from step 2. If the entry is marked ui_only, read the recorded reason
before spending effort; someone already found this unreachable and said why.
Failure mode: no map file yet, which is not an error. It means this domain is new and the map starts empty.
2. Drive run A
scripts/capture.py --url URL --action "create a project named alpha-one" \
--out runs/a.json --profile ~/.api-mapper/profile --trace-id TRACE
The script opens the browser with the persistent profile, records a short baseline window, prints the action description, and waits. The operator performs the action and signals that they are done. Every request and response pair is written to the output file, and only once the run finishes, so a path that exists always holds a whole capture.
You choose the action and the input. Say exactly what to do, in one sentence, so the same action can be repeated identically in run B apart from the input.
One capture often contains several endpoints worth mapping, since the page that
creates a record usually lists them too. Map one endpoint at a time from it
rather than trying to generalize several at once; the diff is only meaningful
for the request whose input you actually varied.
Choosing a wait mode. --wait-for decides how the action window ends,
which is how the operator signals that they are finished unless --steps is
driving the action instead. The browser is headed in every mode; headed is not
the same as interactive.
| Mode | Signal | Use when |
|---|---|---|
tty (default) |
operator presses enter on stdin | a person is running the command themselves |
sentinel |
a file appears, default runs/.a.done |
you are driving the pipeline and a human is at the browser |
close |
every page in the window is closed | bootstrapping a profile, where nothing is being recorded |
steps |
the steps finished and the network went quiet | --steps was given, and it is the default then |
You have no TTY, so tty mode will refuse to run for you. If a steps file for
this action exists, use it and no operator is needed. Otherwise drive the
capture with sentinel mode and tell the operator which file ends the run, or
hand them the command to run in their own terminal. Prefer sentinel over close
for a capture: a capture keeps the browser alive for a moment after the signal
so responses still in flight can land, and close mode has no browser left to
wait in. The causal request's response is what diff.py reads, what the oracle is
built from, and what a read is verified against, so losing it costs the run.
Driving the action from a steps file. When a human has written the action
down, --steps performs it and no operator is needed in the action window at
all:
scripts/capture.py --url URL --action "create a product" \
--steps steps/create_product.jsonl --arg title=AAA111 --arg price=9.99 \
--out runs/a.json --trace-id TRACE
Targets are described in plain language and resolved against the live page, and resolution refuses to guess: zero matches stops the run with a listing of what the page is actually showing, and so does more than one, with both candidates printed. Every step is then confirmed to have taken effect before the next one runs. An ambiguous target is a steps file problem for a person to fix, and the listing in the refusal is what they fix it from.
Two things about it are worth carrying without opening the reference, because
they are what a steps file most often gets wrong. An element is named the way
the page names it, not the way a person would: Shopify's product title input
is accessibly named Title, so the product title field matches nothing and
the Title field works. And where no rewording separates two controls, within
scopes the target to a container, which is the only thing that tells apart two
buttons sharing a role, a name and their text.
Wait modes in full, the steps file format, the resolution ladder, within, and
what each confirmation does and does not prove are in
references/CAPTURE.md. Read it when a target refuses to
resolve or a capture comes back empty.
Failure mode: the profile is not authenticated, so the operator lands on a
login page. Stop; this skill does not log in. Run scripts/bootstrap_profile.py
once against the target, then capture again.
Failure mode: a target matches two elements. The run stops and prints both.
Sharpen the description, for example by naming the kind of element, since "the
Save button" and "the Save link" tell apart two things both called Save. When
the kind is the same as well, as with two buttons both named Save, name the
container with within. Do not work around it by picking one at random in your
head; the whole point of the refusal is that a wrong pick is invisible
afterwards.
3. Drive run B with a meaningfully different input
scripts/capture.py --url URL --action "create a project named beta-two" \
--out runs/b.json --profile ~/.api-mapper/profile --trace-id TRACE
Same action, different input. Meaningfully different matters; see the judgment section below.
With a steps file it is the same file and different --arg values, which is
what the file is for:
scripts/capture.py --url URL --action "create a product" \
--steps steps/create_product.json --arg title=BBB222 --arg price=41.50 \
--out runs/b.json --trace-id TRACE
Failure mode: the operator performs a slightly different action, for example uses a keyboard shortcut instead of the button, which changes the request path. The diff then shows differences that are not about input at all. If run B looks structurally unlike run A, redo run B. A steps file removes this failure mode entirely, which is the other reason to write one: the same steps are performed the same way both times, so anything that differs between the runs differs because the input did.
4. Reduce to candidates
scripts/reduce.py runs/a.json runs/b.json --out candidates.json --trace-id TRACE
Filters out noise and ranks what remains. Output is a ranked shortlist with a score and a plain reason per candidate.
The script ranks, you select. It does not auto select, because scoring is a heuristic and the cost of silently picking the wrong request is a template that appears to work. Read the reasons, pick the one that matches the effect you actually saw, and say why you picked it.
Two lists come out, and they answer different questions.
may have caused the effect is the candidate ranking. It normalizes query
strings away, because two polls that differ by a cache buster are the same
request, and a write carries its input in a body.
reads worth mapping is the second list. A read carries its input in the query
string, so the thing the first list discards as noise is the whole signal here,
and a read is never going to appear in a ranking built to find a write. Each
read names the query keys that actually varied, which are the ones diff.py will
parameterize. Values that vary the way a cache buster varies, a timestamp, a
uuid, a random hex nonce, are discounted, so polling traffic does not come back
as a surface worth mapping.
Reads are ranked but never auto selected, and they are never merged into the
candidate list, because a read cannot have caused the effect and offering it as
though it might have would be a worse answer than not finding it. Select one by
index with diff.py --select.
A capture of a search surface has no candidates at all, and that is the expected result rather than a failure. The reads around a write are the cheapest endpoints to map next: the list endpoint beside a create is usually a GET that costs one capture and no risk, and it is often the same endpoint the write's oracle already reads.
Failure mode: everything is ranked low, which usually means the causal request went out over a WebSocket, or was a form post that triggered a full navigation, or the action was handled entirely client side and synced later.
5. Diff and generalize
scripts/diff.py runs/a.json runs/b.json --candidates candidates.json \
--out template.json [--run-c runs/c.json] --trace-id TRACE
Classifies every field of the selected candidate, walking nested structures, not just top level keys.
| Observation | Classification | What to do with it |
|---|---|---|
| Changed with the input you varied | parameter | Expose as a template argument |
| Identical across both runs | constant | Hard code it in the template |
| Changed but the input did not change it | generated | Mint fresh per replay, for example a nonce, a timestamp, a request id |
| Auth bearing header or cookie | credential | Re mint from the live session per run, never store |
Credentials are never written to the template, the registry, or the map. A stored bearer token is a leaked bearer token with a delay on it, and it will expire anyway, so storing it buys nothing.
If a field is ambiguous, meaning it changed between A and B but might be
generated rather than driven by input, record a third run with the same input as
run A and pass it as --run-c. Anything still changing across A and C is
generated, because the input held constant. The template output records which
fields were disambiguated this way and which remain uncertain. Uncertain fields
are the ones that break replay later, so do not leave them unresolved if a third
run is cheap.
Failure mode: the request body is opaque, for example signed, encrypted, or protobuf. See "Refusing to map".
6. Classify the endpoint as a read or a write
scripts/classify.py template.json
Do this before deciding how to verify, because the two are verified in completely different ways and the wrong one proves nothing.
A read returns data and causes no effect. GET is a read. A GraphQL operation
whose document begins with query is a read even though it travels by POST. A
read is safe to fire repeatedly, which is what makes everything downstream
cheaper.
A write creates, updates, or deletes. Any of POST, PUT, PATCH, or DELETE that is not a GraphQL query is a write until shown otherwise.
The script infers this from the method and, for GraphQL, from the operation type in the document, and prints the evidence it used. Do not assume from the method alone; some POSTs are reads, which is the normal case on any GraphQL target and common on search endpoints everywhere.
It reports unknown rather than guessing when the evidence is not there, which
happens with a persisted query, where the client sends a hash instead of the
document and the operation type is not visible in the request.
unknown is recorded as unknown. It is not folded into write, because
"this writes" and "nobody could tell, so it is being handled as a write" are
different facts, and only the second is something a human should come back to.
The safety gates treat unknown exactly as they treat write, so nothing
destructive fires on the strength of not knowing; the difference is what the map
says afterwards, not what is allowed to happen. It is the same honesty the
ui_only status and an uncertain credential selector already practise: show
what is not known rather than rounding it to the nearest confident answer.
Settle it with --class when you watched the action and know what it did. The
entry then records that a person settled it, and what the request itself said,
so a later reader can see the two separately.
Failure mode: a multi step sequence where one step writes and the others read. The sequence is a write; the script takes the strongest class in the sequence.
7. Verify, in the mode the class calls for
A write is verified by its effect. A read is verified by its shape. Both modes
live in replay.py and are selected by the class from step 6.
Write, effect oracle.
scripts/replay.py template.json --args '{"name":"gamma-three"}' \
--oracle oracle.json --trace-id TRACE
Fires the template headlessly with a fresh input, then checks the oracle. Both the response check and the oracle check must pass.
A 200 is not success. Parse the body. A non empty errors or userErrors array
is a failure regardless of status code. Then confirm independently through the
oracle that the effect exists in the world.
Read, structural equivalence.
scripts/replay.py template.json --args '{"query":"alpha"}' \
--verify structure --baseline runs/a.json --baseline-index 41 --trace-id TRACE
Replays the request with the same inputs it was captured with and confirms the response is structurally equivalent to what was captured: the same keys at the same paths, holding the same types. A read causes no effect, so there is nothing for an oracle to look for and no sandbox needed; firing it again is the whole check.
--baseline is the response that was captured. Pass the capture file with
--baseline-index, using the same index you gave diff.py as a candidate, or a
file holding just that response.
Additive differences do not fail the check. A field that is gone, or a field whose type changed, breaks whatever reads it and is a mismatch. A field that is new cannot break anything built against the older shape, so it is reported as a note. A check that failed every time a backend added a field would fail so often that passing it would stop meaning anything.
Know what this does and does not prove. It proves the endpoint still exists, still accepts these arguments, and still answers in the shape the template was built against. It does not prove the values are correct, and it cannot, because nothing independent of the endpoint knows what the right answer is.
The mode is not a free choice. With neither flag, replay derives it from the
class: a read is verified structurally, a write through its oracle, and an
unknown is refused until you settle it. --verify structure on anything not
classified a read is refused outright, because a mutation that writes nothing
still answers in the shape it always did, so shape must never be what records a
write as verified. If the class is wrong, fix the class with --class, which is
what selects the mode.
Use --dry-run first in either mode to print the resolved request without
sending it. That is how you catch a mis-bound parameter before it writes
anything.
Failure mode: verification fails with a structured reason. Map it to a refusal category, do not retry blindly. A signed body will never replay, no matter how many times you try it.
8. Probe what the endpoint actually requires
scripts/probe.py template.json --args '{"query":"alpha"}' --out auth.json \
--trace-id TRACE
Replays the endpoint three ways and records which of the three the server accepts:
| Variant | What is sent | What it tells you |
|---|---|---|
| full | the live session's credentials, as replay normally sends them | that the endpoint works at all |
| none | every credential header dropped, cookies withheld | whether it is open to anyone |
| stale | credential headers present but holding a malformed token | whether it validates the token or merely requires one |
That yields the entry's auth requirement: public when the unauthenticated
variant succeeds, requires_any_auth when a malformed token is accepted but
nothing at all is not, and requires_fresh_auth when only the live session
works. This is real intelligence about the target and it is a first class field
on the entry, not a footnote.
Probing reuses the same credential handling replay does. Nothing is stored, and the stale variant forges nothing: it sends a value that is visibly not a credential, to see whether the server looks.
The stale variant has a limit, and the record says so. It works by replacing
the value of a credential header, so it has something to replace only when the
request carries one. An endpoint authenticated purely by session cookie carries
none: the cookie rides in the jar, and a browser will not let a script set a
Cookie header, so a well formed but wrong credential cannot be presented from
inside the page at all. The variant is then recorded as not applicable. The
requirement stays requires_fresh_auth, because nothing except the live session
was accepted and that much was tested, but the reason names the half that was
never asked: whether the server validates its credential or would take any well
formed cookie. A refusal that was always going to happen is not evidence that
the server checks anything.
A consented probe can settle an unknown class. If the class is unknown,
you passed --allow-writes, and you supplied --oracle, the probe asks the
oracle afterwards whether the effect appeared. If it did, the endpoint writes,
and the entry records class: write with settled_by: probe. That is evidence
of the same kind the request failed to supply. Three limits, all enforced:
- It only fills an unknown nobody has settled. A class an operator set is never
overruled, including an
unknownthey set deliberately, because a person who watched the action outranks a probe that fired it. - It only moves towards
write. A probe that sees no effect has not shown the endpoint is a read, only that it did not write this time, so the class stays unknown and stays gated. - It cannot happen unnoticed, because
settled_byalways names who settled the class, and the consent gate sits in front of the whole thing.
Writes are gated. Probing fires the endpoint three times. On a read that is
free; on a write it is three creations, three deletions, or three of whatever
the endpoint does. probe.py refuses a write, and refuses an endpoint whose
class is unknown, unless you pass --allow-writes, and you should pass it only
against a sandbox you are willing to see written to three more times.
Failure mode: the full variant fails, which makes the other two
uninterpretable. That is an expired session or a broken template, not a finding
about the endpoint. The requirement is recorded as unknown with the reason.
9. Record the entry
scripts/replay.py template.json --args ... --oracle oracle.json --write-back
scripts/registry.py write DOMAIN ACTION --template template.json \
--status verified --reason "verified against list oracle on 2026-09-15"
scripts/registry.py map add DOMAIN ENDPOINT --template template.json \
--auth auth.json --status verified --reason "..."
--write-back records the verification result into the template. Without it,
nothing downstream has anything to check and verified is refused.
Only a run that passed verification in step 7 may be recorded as verified.
Everything else is ui_only with the reason recorded. There is no third status,
because any hedged status would eventually be treated as usable.
The registry write keeps the per action record the single template workflow has always produced. The map add puts that same record into the domain's map with its class, its auth requirement, and the time it was last verified. Both are built from the same template, so they cannot disagree.
Failure mode: the script refuses to write verified without a verification
record of the right mode for the class. That refusal is deliberate; do not work
around it by hand editing the map file.
10. Keep the map current
scripts/registry.py map check DOMAIN --max-age-days 30
scripts/registry.py map check DOMAIN --against templates/
scripts/registry.py map check DOMAIN --verify [--probe-writes] [--write-back]
Drift detection already runs per entry through the integrity hash. At the map
level, map check walks every entry and reports three separate things.
Moved. An entry whose integrity no longer matches, or that failed to
re-verify. --against DIR compares the stored integrity against templates
freshly derived from new captures, named ENDPOINT.json, and fires nothing.
--verify re-fires each entry in the mode its class calls for: a read against
the response shape recorded when it was verified, a write against its oracle,
using the examples recorded with its parameters. Re-firing a write needs
--probe-writes, and an entry skipped for want of that consent is reported as
unchecked rather than passing quietly. --write-back refreshes the last
verified time on whatever re-verified cleanly.
Stale. A verified entry not re-verified within --max-age-days. An entry
that was never verified is not stale, it is unverified, and it belongs in the
list below; reporting it as both would bury the entries that really did go out
of date.
Needs a human. Every entry with an open gap: a class that is unknown or
that nobody is named as having settled, an auth variant marked
applicable: false, an endpoint never probed or whose requirement came back
unknown, a ui_only status with its reason, and any uncertain_fields.
That last list is the point of keeping the map rather than a pile of templates. It is not a list of failures and it does not change the exit code, which goes non zero for the two things that are events: something moved, or something went stale. The gaps are a standing state, printed every time, because they are what an operator actually acts on next.
Judgment calls
These are the points where the scripts hand the decision back to you.
Which endpoints to map. The one the user asked about, then the reads that make it usable. A create endpoint whose ids you cannot list is hard to use; the list endpoint next to it is usually a GET that costs one capture and no risk. Map the write first, because it is the one that needed the sandbox, then fill in the reads around it.
Choosing run B's input. Vary the one thing you want to parameterize, and
vary it so the change is unmistakable in a diff. Different length, different
character class, different value entirely. alpha-one then beta-two, not
test1 then test2, because a single character difference can hide inside a
field you did not expect to be affected, and two short similar strings make
hashes and derived fields hard to tell apart. Avoid values that collide with
something already in the account, since a uniqueness error changes the response
shape and ruins the diff.
Writing a steps file. This is a human's job, and it is worth doing as soon
as an action will be performed more than once, which is almost always: run B
needs it, run C needs it, and every re-verification after that needs it. The
authoring rules are in step 2 above; the judgment is in what to look at before
writing. Read the page first and take the words off it rather than reaching for
the words you already have in your head, because the description that comes to
mind is usually the product's name for a thing rather than the page's. When a
target comes back ambiguous, fix the description or scope it with within; do
not fix the page and do not reach past the resolver. Keep the step list to the
one action being learned; an extra click in the steps file is an extra candidate
in the diff, exactly as it would be from a human.
Deciding the UI action succeeded. Before you trust a capture, confirm the app visibly did the thing: the row appeared, the toast said saved, the counter moved. A capture of a failed action still contains a request and a response, and it will diff cleanly against another failure. You will have learned how to fail reliably. If the operator is unsure whether it worked, redo the run.
Settling an unknown class. A persisted query hides the operation type, so
classify.py reports unknown rather than reading the operation name and
hoping. You can usually settle it from what you watched happen: if the page
changed nothing and only displayed data, it is a read. If you cannot settle it
from evidence, leave it unknown and let the write gate protect you. Naming an
operation ProductSearch is not evidence; operations get renamed and names lie.
Grouping multi request sequences. Some actions are genuinely several requests: reserve an upload URL, PUT the bytes, then commit the reference. Treat them as one sequence when a later request carries a value that only the earlier response could have produced, and when firing the later one alone would be meaningless. Treat them as separate when each stands on its own. When in doubt, test it: replay only the last request and see whether the effect still happens. If it does, the earlier ones were not part of the action.
Resolving ambiguous fields. Prefer evidence over inference. Run C is the tool for this and it costs one more capture. If a third run is genuinely not possible, use shape: a UUID, an epoch in milliseconds, or a long random hex string is almost certainly generated; a value that reads back as something the operator typed is a parameter. Record the uncertainty in the template rather than quietly guessing, so the next failure is diagnosable.
Refusing to map
Passive capture and replay do not reach every endpoint, and saying so early is
more useful than producing a map entry that fails intermittently. Record the
endpoint as ui_only with the reason, then move on to the next one. A recorded
refusal saves the next person the same investigation, and a map that names its
own gaps is worth more than one that quietly omits them.
Stop on an endpoint when you see:
- Signed request body. An HMAC or signature field computed over the body by client code. You cannot mint it without the signing key or reimplementing the client, and the key rotates.
- Device or app attestation. The server requires a token issued by an attestation service proving a genuine browser or device. It is designed specifically so that replay outside the app fails.
- Aggressive bot defense. A challenge, a proof of work, or a fingerprint the replay context cannot reproduce. Same outcome as attestation.
- Credentials that cannot be re minted. If the auth token is bound to a browser context you cannot reproduce headlessly, every replay dies at the first request.
- The causal request never appears. WebSocket, WebRTC data channel, or a service worker that batches. These captures do not contain the thing you need.
- Opaque or unparseable body. Encrypted, protobuf, or a custom binary encoding with no readable field boundaries. There is nothing to parameterize.
- Verification never passes. After two honest attempts with a correct oracle or a correct baseline, stop. Repeated failure is information, not something to brute force.
- Terms make it off limits. Automation is contractually prohibited on this target, or nobody can say whether it is allowed. Stop regardless of technical feasibility. This is the first precondition arriving late: asking before the captures costs one question, and asking after them costs the captures.
Say plainly what the map does not cover, both per entry and in summary. Do not claim the skill works on every site; it does not, and the entries above are the reasons.
Details on detecting each auth scheme from a capture are in references/AUTH.md.
The two stores
maps/DOMAIN.json holds one entry per endpoint and is the output of this skill.
registry/DOMAIN.json holds one record per learned action and is what a
template becomes when it is recorded. The map entry is that record plus the
three things the registry has no opinion on: the class, the auth requirement,
and when the entry was last shown to still work. The fields of an entry are in
"The unit of output" above.
Two rules about them are enforced rather than documented, so they are worth
knowing before you write either one. An action is always a sequence list even
when it has one step, so a multi step action is never a special case. And
status has exactly two values, verified and ui_only, because any third
value such as probably_works would be read as usable by someone in a hurry.
Full schemas, field by field, with the class and auth semantics and how integrity hashing works:
- references/MAP.md for the map, including a worked example holding a verified write, a verified read, and an endpoint that could not be learned.
- references/REGISTRY.md for the registry, including the sequence step shape, the generated field strategies, and relearning.
Scripts
| Script | Does | You supply |
|---|---|---|
scripts/bootstrap_profile.py |
Opens the capture profile headed so a human can establish a session in it once | The URL to open and who does the signing in |
scripts/capture.py |
Opens an authenticated browser, records a baseline window, then records every request and response during the action. With --steps it also performs the action itself |
The action to perform, its input, and the wait mode |
scripts/steps.py |
Parses a steps file, resolves each described target to exactly one live element through Playwright's locators, performs each step and confirms it took effect. Used by capture.py, not run directly | The steps file, written by hand, and the --arg values |
scripts/reduce.py |
Filters noise and ranks remaining requests by likelihood of having caused the effect | The selection from the ranked list |
scripts/diff.py |
Classifies fields as parameter, constant, generated, or credential and emits a template | Which candidate, and whether a run C is needed |
scripts/classify.py |
Decides from evidence whether an endpoint reads or writes, says which evidence, and answers unknown when the evidence is not in the request |
The call on an unknown, when the operation type is not visible |
scripts/replay.py |
Fires the template headlessly and verifies it, by effect oracle for a write or structural equivalence for a read | The oracle or the baseline, and fresh arguments |
scripts/probe.py |
Replays the endpoint with full, absent, and stale credentials to establish what it actually requires | Permission to probe a write, through --allow-writes |
scripts/registry.py |
Reads and writes the registry and the map, enforces the status rule, |
…(truncated)