# Define Data > Define Data — Phase 4 (of 7) - Skill: `marcellourbani/define-data` (Agent Skill) - Install (CLI): `npx skillmds@latest add marcellourbani/define-data` - Raw SKILL.md: https://api.skillmd.com/api/skills/marcellourbani/define-data/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - Author: marcellourbani (https://skillmd.com/u/marcellourbani) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/marcellourbani/define-data --- # Define Data — Phase 4 (of 7) For a cross-system staged scenario, also load `multi-system-workflows`: declare only pre-existing inputs here. A key produced by an earlier system at run time belongs in the workflow JSON handoff, not a prepared-data requirement. Phase order: analyze-and-plan (1) → explore-ui (2) → design-cases (3) → **define-data (4)** → prepare-data (5) → build-scripts (6) → run-scripts (7). This phase produces `TC-XXX.data.md` requirement specs — the reusable, system-agnostic SHAPE of the data each case needs. It does NOT resolve concrete values (that is Phase 5, `prepare-data`) and does NOT write specs (Phase 6). For bounded, self-contained support work, use `sap-task-helper` with explicit inputs, allowed writes, and an output contract. ## Why this is its own phase Data specs written as an afterthought at the end of a big case-writing session come out as one-line stubs ("cache key + status") that Phase 5 then has to throw away and rewrite from scratch. That is exactly what happened before this was split out. A `.data.md` is a real contract: it decides which of four sources each value comes from, whether an upload fixture is Excel or CSV, and which keys the spec (Phase 6) will reference. Getting the format wrong here means the program silently rejects the upload at run time and the "test" proves nothing. This phase exists so every data requirement gets deliberate attention, once, in the right format. ## Non-negotiable execution gate `prepare-data` (Phase 5), `build-scripts` (Phase 6), and `abapfs_run_playwright_tests` (Phase 7) all reject cases whose `.data.md` is missing, malformed, or references undeclared keys. Author every required sidecar here, correctly, before handing off. ## Tool availability (read this if a tool seems missing) **Naming:** when these docs say *call* `X`, `X` is a **tool** (you invoke it and get a result); *delegate to / invoke* `X` is an **agent** (a subagent you launch); *load / follow* `X` is a **skill** (a procedure you read). A name without a verb: see the overview's "Skills, tools, and agents" list. The editor may hide tools until searched for. Before Step 0, ensure `abapfs_get_test_folder` and `abapfs_build_test_index` are available; if either is missing, search your available tools for it by name. If one cannot be found, tell the user. ## Step 0 — Standalone bootstrap and input gate (mandatory) > **Say before acting:** "Starting Step 0: standalone bootstrap and input gate." 1. Call `abapfs_get_test_folder` **before reading any artifact**. Treat the result as ``; never infer it. 2. If unset, STOP and ask the user to run "ABAP FS: Enable SAP UI Testing Features". If not open in the workspace, STOP and ask them to add it. 3. Resolve `` from the request; if omitted, inspect `/tests/*/test-cases/_index.md`. Auto-select only when exactly one candidate matches; otherwise ask. 4. **Enforce the Phase 3 input gate:** `_index.md`, `_findings.md`, `_units.md`, `_screens.md`, and every `TC-XXX.md` must exist. If `_index.md` is missing, STOP and follow `design-cases`. From `_index.md`, take the authoritative `Data required?` column — that is the list of cases needing a `.data.md`. Do NOT re-derive it by rereading every TC body. 5. Read `_findings.md`'s `## Input file format` — the fixture format, column count, and header names come from there. 6. Read `_units.md` — its "Effective inputs" (DB tables the program reads to validate rows, e.g. MARC/MARA existence) tell you which tables a `source: sql` requirement should query for genuinely valid values, so the data you spec actually passes the program's own validation. > **Say before continuing:** "Step 0 completed. Evidence: test folder, program, `_index.md`, and required upstream artifacts confirmed. Next: Step 1 — enumerate data-required cases." ## Step 1 — Enumerate the cases that need data > **Say before acting:** "Starting Step 1: list every dataRequired: yes case and its data-key references." - From `_index.md`, list every case with `Data required? = yes`. Each needs exactly one `TC-XXX.data.md`. - For each such case, read its `TC-XXX.md` and collect every `` placeholder used in the state table, Steps, Expected Result, and the `## Post-test verification` SQL rows. Every placeholder becomes a `requires` key. - Cases with `Data required? = no` get NO sidecar — creating one is an error `abapfs_build_test_index` rejects. > **Say before continuing:** "Step 1 completed. Evidence: data-required cases and their referenced keys enumerated. Next: Step 2 — author each spec." ## Step 2 — Author each `TC-XXX.data.md` > **Say before acting:** "Starting Step 2: write a complete requirement spec for every data-required case." **Frontmatter placement — the #1 cause of silent data failures. Read this first.** The `requires:` block MUST be a `---`-delimited YAML **frontmatter at the very top of the file** — literally the first bytes. The runtime parses ONLY a leading `---` block; a `requires:` block placed anywhere else parses to nothing, so `resolveTestData` returns zero keys and every `data.` silently comes back `undefined` (surfacing much later as a confusing "value is not a string" in `setField`). When you copy the template below, reproduce the `---` … `---` as the actual start of the file: - ✅ RIGHT — file begins with the frontmatter: ``` --- tcId: TC-001 requires: - key: sample_site ... --- ## Manual override ... ``` - ❌ WRONG — `requires:` under a heading or inside a code fence (parses to zero requirements, `abapfs_build_test_index` and `abapfs_verify_test_data_usage` will now reject it): ```` # TC-001 Test Data ## Requirements ```yaml requires: - key: sample_site ``` ```` Do NOT wrap the frontmatter in a ` ```yaml `/` ```markdown ` code fence, and do NOT put `requires:` under a `## Requirements` heading. Pick the NARROWEST correct `source` per key — don't default to `static` because it's easiest. ```markdown --- tcId: TC-001 requires: - key: sample_something description: source: sql sql: | SELECT ... FROM ... WHERE ... take: first distinctFrom: [sample_other] # declared on BOTH keys of the pair - key: sample_other description: source: sql sql: | SELECT ... FROM ... WHERE ... take: first distinctFrom: [sample_something] # Phase 5 guarantees these resolve to different values - key: some_constant description: source: static staticValue: "FERT" - key: upload_fixture description: source: generated generator: fixture-builder args: format: xlsx # MUST match the program's parser — see Step 3 filename: upload.xlsx columns: [Article, Site, Start Date, End Date, Category, Min Value, Tolerance, Alt Tolerance, Target Rate, Priority, Fallback Rate] rows: # cells may reference other keys ("{{sample_something}}") resolved above, # or relative-date tokens ("today", "+30d", "-5d") resolved against the # CURRENT run time — NEVER write an absolute date, it goes stale. - ["{{sample_something}}", "1000", "+30d", "+31d", "01", "100.00", "5.00", "2.00", "10.00", "1.00", "8.00"] - key: existing_precondition description: source: seeded seed: viaTcId: TC-001 # prepare-data runs this TC's spec once as an approved setup step sql: | SELECT ... FROM ... WHERE ... take: first --- ## Manual override - `TESTDATA_TC_001_QAS_sample_something=X` (system-specific, preferred) - `TESTDATA_TC_001_sample_something=X` (all systems, use sparingly) ``` Rules: - SQL must be portable ABAP SQL (see `abapfs_get_sql_syntax`). Do NOT bake analysis-system values into `.data.md`. - **Never put row-limit syntax in the SQL.** ABAP SQL via ADT rejects `FETCH FIRST n ROWS ONLY`, `LIMIT`, `TOP`, and `ROWNUM`. Choosing among returned rows is what `take: first|last|any` is for; the actual fetch cap is a Phase 5 tool parameter (`abapfs_run_sql_query`'s `rowRange`/`maxRows`), NOT part of the spec. A `.data.md` SQL that uses `FETCH FIRST` fails the moment Phase 5 runs it. - **`take: first`/`last` is NOT a uniqueness guarantee.** A `SELECT` without a deterministic `ORDER BY` has no defined row order, so "first" and "last" can be the SAME row (and with one candidate they always are). If two keys must resolve to DIFFERENT values, do NOT rely on `take:` — declare `distinctFrom` (below). - **Cross-key uniqueness → `distinctFrom: [, …]`** on BOTH keys that must differ (e.g. two article numbers that must not be equal). This is enforced in Phase 5 by `abapfs_check_test_data`, which resolves the whole program and FAILs if two mutually-`distinctFrom` keys landed on the same value — a guarantee `take:` cannot give. - Static constants → `source: static, staticValue: ...`. - A precondition only the application under test can create → `source: seeded`, naming the earlier TC whose spec creates it via `seed.viaTcId`. If NO TC can seed it (it's written by a different program/BAdI/interface), leave `seed.viaTcId` off and instead add `seed.manualSteps` describing how a human seeds it, so Phase 5 can present it. Check `seeded` before ever marking a case `blocked-by-data`. **Ordering note:** a `seed.viaTcId` spec is written in Phase 6, AFTER this phase — that is expected. A `seeded` requirement is DEFERRED, not broken; Phase 5 records it as deferred-until-phase-6 and resolves it in a second pass once the seeding spec exists. Record the dependency in BOTH TCs' `## Notes for automation` so the order is never a hidden surprise. - `dataRequired: yes` in `TC-XXX.md` → a matching `TC-XXX.data.md` is mandatory. `dataRequired: no` → do not create one. - Every `requires` key must be a key the case's `TC-XXX.md` actually references, and every `` in the `.md` must have a `requires` entry here. (Keys used only in `## Post-test verification` SQL or `## Absence preconditions`, or referenced only as a seeding target, are legitimately declared here and consumed outside the spec — that is fine and not an error.) - **A pre-test DB-state snapshot is NOT a `requires` key — never declare one.** A "row count before the run" (an `initial_row_count`-style baseline) is a MEASUREMENT Phase 7 takes immediately before the case, not an input resolved once per system. `requires` values are cached in `data.json` at first-prepare time and reused for every later run, so a baseline stored there freezes forever and every rerun then asserts a stale number. Express the check in the TC's `## Post-test verification` instead — as a relative assertion Phase 7 evaluates with a before/after query pair, or (better) an absolute assertion that needs no baseline. If you catch yourself writing a `requires` key that means "how many rows existed before," stop and move it to verification. - **Absence-precondition keys (E8): declare the VALUE, not the emptiness.** For a case with an `## Absence preconditions` section (its premise is that a value does not exist), declare the candidate value the case types in — usually `source: static` (a dummy value expected to be absent) or `source: generated`. Do NOT try to express "a value that returns zero rows" as `source: sql` — SQL that returns nothing resolves to a missing key and the case is reported missing-data instead of running. The absence SQL itself lives in the TC; Phase 5 runs it and blocks the case if the row actually exists. > **Say before continuing:** "Step 2 completed. Evidence: every data-required case has a `.data.md` whose keys match its `.md`. Next: Step 3 — get the fixture format right." ## Step 3 — Fixture files: match the program's real parser (do NOT default to CSV) > **Say before acting:** "Starting Step 3: set every generated fixture's format from _findings.md's input file format (or confirm no fixtures are needed)." **Step 3 applies ONLY to `source: generated` (upload-fixture) requirements. If you declared none in Step 2 — the program has no file-upload path — say "no generated fixtures required, Step 3 skipped" and go to Step 4.** Do not invent a fixture the program can't consume. Upload fixtures (`source: generated`) are built by the runtime's fixture builder as either `.xlsx` or `.csv`. The program only accepts ONE of these — and a CSV handed to an Excel parser silently fails, so the "test" passes on a broken upload or errors for the wrong reason. Do NOT reach for CSV because it's simpler. - Read `_findings.md`'s `## Input file format`. If the program parses Excel (`ALSM_EXCEL_TO_INTERNAL_TABLE`, `KCD_EXCEL_OLE_TO_INT_CONVERT`, `gui_upload` with an `.xls`/`.xlsx` filter), set `args.format: xlsx` and give `filename` an `.xlsx` extension. Only use `format: csv` when the program genuinely reads delimited text via `GUI_UPLOAD`. - Match the EXACT column count and header names from `_findings.md` — e.g. an 11-column Excel with the exact headers the parser expects. A fixture with the wrong columns fails validation inside the program, not in the test harness. - Each row's cells may reference other resolved keys via `{{key}}` and use relative-date tokens; never hardcode an absolute date. - **Match the EXACT date format the program parses**, taken from `_findings.md`'s recorded format (which is verified from the conversion code — e.g. `MM.DD.YYYY` vs `DD.MM.YYYY`). A fixture whose dates are in the wrong order is silently mis-stored or rejected, and the "test" proves nothing. Set the fixture's `dateFormat` arg to that exact format so relative-date tokens (`today`, `+30d`) render correctly — do NOT rely on the builder's locale default. If `_findings.md` records the format without the proving conversion line, treat it as unverified and confirm against the source before generating. - If the program HAS an upload path but `_findings.md` doesn't state the file format, STOP and follow `analyze-and-plan` to record it (read the upload/parse code) rather than guessing. (If there is no upload path at all, there is nothing to record and nothing to do here — you already skipped this step.) > **Say before continuing:** "Step 3 completed. Evidence: every generated fixture's format, extension, and columns match the program's parser per `_findings.md`. Next: Step 4 — validate." ## Step 4 — Validate that declarations and sidecars agree > **Say before acting:** "Starting Step 4: rebuild the index to confirm every dataRequired declaration has its sidecar." Re-run `abapfs_build_test_index` (`program`, the `sourceSnapshot` from `_findings.md`, and `reviewerConfirmation` = `I called the reviewer agent and it passed all test cases` — the reviewer already passed in Phase 3). This is where the `.data.md` existence check that was deferred in Phase 3 is enforced: the tool WARNS for every `dataRequired: yes` case still missing its `.data.md`, and ERRORS for any `dataRequired: no` case that has one. `abapfs_build_test_index` now also ERRORS if a `.data.md` exists but its `requires:` is not parseable top-of-file frontmatter (e.g. it's in a ` ```yaml ` fence or under a `## Requirements` heading) — a hard stop, because that file resolves to zero keys. Fix the placement, don't work around it. **Completion requirement: there must be ZERO "`.data.md` does not exist yet" warnings.** If any remain, author the missing sidecar (or, if the case genuinely needs no data, change its `TC-XXX.md` to `dataRequired: no` — but that is a case-design decision; if unsure, follow `design-cases`). Do not hand off with outstanding warnings. > **Say before continuing:** "Step 4 completed. Evidence: `abapfs_build_test_index` reports zero missing-`.data.md` warnings and no `dataRequired`/sidecar mismatches. Next: Step 5 — hand off." ## Step 5 — Write the next-chat handoff to Phase 5 (`prepare-data`) > **Say before acting:** "Starting Step 5: write the Phase 4 handoff." Make disk state sufficient for a fresh `prepare-data` chat: - Every `dataRequired: yes` case has a valid `TC-XXX.data.md`; no `dataRequired: no` case has one. - `_index.md`/`_index.docx` are current with zero missing-sidecar warnings. - Your final response names ``, ``, `connectionId`, the cases whose data still needs resolving against a live system, and any `seeded` requirements that will need approval in Phase 5. **Your final message MUST tell the user the exact next step: "Next: Phase 5 — start a new chat, load the `prepare-data` skill, and say: Prepare data for `` on `` using the data specs on disk."** Naming the skill matters — without it the next chat tends to skip loading `prepare-data` and improvise. > **Say after the handoff is complete:** "Step 5 completed. Evidence: every data spec is authored, validated, and handed off. Phase 4 completed. Next phase: Phase 5 — in a new chat, load the `prepare-data` skill and follow it." ## Anti-patterns - ❌ Putting `requires:` in a ` ```yaml ` fenced block or under a `## Requirements` heading instead of top-of-file `---` frontmatter — it parses to nothing, so every `data.` silently resolves to `undefined`. - ❌ One-line stubs (cache key + status) instead of a real `requires` array — the exact failure that forced this phase to exist. - ❌ Defaulting an upload fixture to CSV when the program parses Excel (or vice versa) — read `_findings.md`'s input file format. - ❌ A generated fixture with the wrong column count/headers — it fails inside the program, not in the harness. - ❌ Baking an absolute date or an analysis-system value into `.data.md` — use relative-date tokens and `sql`/`generated` sources. - ❌ Putting `FETCH FIRST`/`LIMIT`/`TOP`/`ROWNUM` in a `requires` SQL — ADT rejects it; cap rows with the Phase 5 tool params and pick with `take:`. - ❌ Declaring a pre-test row-count/baseline as a `requires` key — it's a Phase 7 measurement, not cached input; put it in `## Post-test verification`. - ❌ Relying on `take: first`/`last` to make two keys distinct — use `distinctFrom` on both keys. - ❌ Creating a `.data.md` for a `dataRequired: no` case, or leaving a `dataRequired: yes` case without one. - ❌ Resolving concrete values here — that is Phase 5. This phase writes the SHAPE only. - ❌ Editing `TC-XXX.md` bodies or the index tables — case design is Phase 3.