Shipkit Engineering Goals

shipkit-engineering-goals — Technical Performance Criteria

Purpose: Derive measurable technical performance criteria from the engineering blueprint. Each mechanism, component, and design decision implies criteria for "how do we know this performs?" — this skill makes those criteria explicit, measurable, and trackable.

What it does: Reads the engineering blueprint + stage context, derives technical criteria from cited signals, writes the engineering goal file, and adds engineering criteria to existing stage gates. Most criteria are grounded from engineering-definition.json, stack.json, and stage — only hard performance targets with no signal become questions.

Protocol: This skill follows the canonical elicitation protocol defined in install/shared/references/elicitation-protocol.md (the mechanics — marker, state files, resume).

Calibration: Apply install/shared/references/ground-or-ask-calibration.md (the intelligence — propose vs ask). Ground first: this is a DERIVATION skill — most criteria flow directly from engineering-definition.json (mechanisms → performance/quality/reliability targets), stack.json (stack norms), goals/strategic.json (stage gates), and goals/product.json (outcome alignment). Cite the signal for every proposed threshold and tag it with its source. Flag low-leverage guesses (guessed: true). The only HIGH-LEVERAGE ungrounded fields are hard performance/reliability TARGETS where no signal (stage, stack norms, product goals) implies the number — ask those and only those. Questions are rare (often zero for a well-defined stack); if you're generating more than 2–3, re-ground first.

Output: One JSON file:

• goals/engineering.json — Technical-performance criteria (EM owns)

Strategic goals (stage, constraints, business metrics: S-) are handled by /shipkit-stage, owned by the Visionary agent. Run that skill first — it creates the stage gates that engineering criteria are added to. Product goals (user outcomes: P-) are handled by /shipkit-product-goals, owned by the PM agent.

Modes

When to Invoke

User triggers:

• "Define engineering goals", "Technical performance criteria"
• "SLAs", "Response time targets"
• "Engineering metrics", "Test coverage goals"
• "Evaluate engineering goals"

Workflow position:

• After /shipkit-engineering-definition (reads the engineering blueprint)
• After /shipkit-stage (reads stage and constraints from strategic.json)
• After /shipkit-product-goals (reads product goals for alignment, adds E-* criteria to existing gates)
• Before /shipkit-plan — criteria inform implementation priorities
• Before /shipkit-review-shipping — criteria become verification checks

Prerequisites

Process — Define Mode

Completion Tracking

After loading context, create tasks:

• TaskCreate: "Derive E-* criteria with rubrics from engineering blueprint"
• TaskCreate: "Classify checkability + verificationTool per criterion"
• TaskCreate: "Map E-* to existing gates"
• TaskCreate: "Write goals/engineering.json"
• TaskCreate: "Update strategic.json gates with E-* IDs"

TaskUpdate each task to in_progress when starting it, completed when done.

Writing engineering.json is NOT done — strategic.json gates must also be updated with E-* IDs. Every criterion must include a rubric (3-5 levels).

Step 0: Check for Existing File

1. Check if .shipkit/goals/engineering.json exists
2. If exists: archive current file to .shipkit/.archive/goals-engineering.YYYY-MM-DD.json and regenerate (fork context — no user prompt; the reviewer catches over-eager rewrites)
3. If nothing exists: Skip to Step 1

Step 1: Load Context

Read these files:

.shipkit/engineering-definition.json  → mechanisms, components, design decisions (REQUIRED)
.shipkit/goals/strategic.json         → stage + existing gates (RECOMMENDED)
.shipkit/goals/product.json           → user-outcome targets for alignment (RECOMMENDED)

If engineering-definition.json missing: Route to /shipkit-engineering-definition first.

Step 2: Read Stage

Read stage from goals/strategic.json (if exists) or default to MVP (fork context — no user prompt; dispatch /shipkit-stage first if stage needs to be set explicitly):

Step 3: Derive Criteria from Engineering Blueprint

For each mechanism in engineering-definition.json, derive three types of criteria:

Performance: How fast?

• Generation/response time for pipelines
• Query response time for search/data operations
• Processing throughput for batch operations
• Sync latency for real-time features

Quality: How good is the output?

• Output accuracy/relevance for AI/ML pipelines
• Data accuracy for transformations
• Content appropriateness for generation

Reliability: How often does it work?

• Completion rate / error rate
• Uptime / availability
• Job success rate for background processing

Infrastructure criteria (always included based on stage):

• Build compiles without errors (verifiable → build)
• Test suite passes (verifiable → test)
• Lint runs clean (verifiable → lint, if applicable)

Every threshold MUST include a rubric. A bare number like "< 500ms" is meaningless without defining what each level looks like. For each criterion, generate a rubric with 3-5 level descriptors:

Example rubric for "API response time (p95)":
  > 5s: Unusable — users perceive system as broken
  2-5s: Poor — noticeable delay, users may retry or abandon
  500ms-2s: Acceptable — slight lag but functional
  200-500ms: Good — feels responsive
  < 200ms: Excellent — feels instant
  Target: < 500ms (good — responsive)

The rubric anchors the threshold to observable reality and makes review assessments consistent.

See references/derivation-patterns.md for detailed derivation examples.

Step 3b: Classify Checkability + Verification Tool

For each derived criterion, assign checkability and verificationTool:

When uncertain: default to observable.

Key distinction: "API responds in < 500ms in test" = verifiable. "p95 latency < 500ms under production load" = observable.

Step 4: Ground-or-Ask Calibration + Fork Decision

Apply install/shared/references/ground-or-ask-calibration.md:

Grounded criteria (almost all cases): Performance/quality/reliability thresholds derived from mechanisms in engineering-definition.json, calibrated by stack.json norms and stage context, are grounded — propose them tagged with source (e.g. source: "M-001 + MVP stage"). Proceed directly to Step 5.

High-leverage ungrounded targets (rare): If a hard performance or reliability target has NO signal from stage, stack norms, or product goals that implies the number — and setting it wrong is expensive to discover late (a customer-facing SLA, a contractual guarantee, a hard capacity constraint) — it is a HIGH-LEVERAGE unknown. Do not invent it.

Fork path (ungrounded high-leverage target found):

1. Write .shipkit/elicitation/engineering-goals/questions.md with the specific threshold(s) you need the user to decide (use the questions.md schema from elicitation-protocol.md). Keep it to 1–3 questions — propose your best-grounded estimate alongside each as a starting point.
2. Write/update .shipkit/elicitation/engineering-goals/progress.json.
3. Do NOT write goals/engineering.json. Do NOT invent the threshold.
4. Emit the marker block as the final lines of your output (the NEEDS_ELICITATION: line must be the last non-empty line):

   status=paused
   turn=1
   questions_file=.shipkit/elicitation/engineering-goals/questions.md
   reason=awaiting user decision on ungrounded high-leverage performance target(s)
   NEEDS_ELICITATION:shipkit-engineering-goals
   

Inline path (no ungrounded high-leverage targets): If AskUserQuestion is available and you have a threshold question, you may ask it. Otherwise proceed to Step 5.

If engineering-definition.json is genuinely missing or unusable, return gaps_found in the artifact rather than emitting a marker.

Step 5: Map to Existing Gates

Read gates from goals/strategic.json and assign engineering criteria to them:

ADDING ENGINEERING CRITERIA TO GATES:

MVP Launch Ready:
  Existing: S-001, P-001, P-002
  + E-001 (generation speed)
  + E-002 (build passes)
  + E-003 (tests pass)

Beta Ready:
  Existing: S-002, P-003
  + E-004 (reliability > 99%)

If goals/strategic.json doesn't exist, define gates locally in engineering.json.

Assign gates directly — no user prompt (fork context). The reviewer will flag misalignments in the loop's review cycle.

Step 5.5: Archive Existing Artifact

Artifact strategy: archive — Before writing, if the target file already exists, move it to .shipkit/archive/{filename}.{ISO-date}.json (create the archive/ directory if needed). Then write the new artifact fresh.

Step 6: Generate Engineering Goal File

Write .shipkit/goals/engineering.json.

If goals/strategic.json exists, also update its gates array to include E-* criterion IDs.

Step 7: Save and Suggest Next Steps

Engineering goals saved to .shipkit/goals/engineering.json

  Stage: {stage}
  Engineering criteria: {N}
  Gates updated: {N} (E-* criteria added)

  Status: {N} not-measured

Next:
  1. /shipkit-plan — Create implementation plans
  2. /shipkit-engineering-goals --evaluate — Check criteria after building

Process — Evaluate Mode

When invoked with --evaluate:

Step 1: Load Engineering Goals

Read goals/engineering.json.

Step 2: Load Metrics

Read .shipkit/metrics/latest.json.

If missing: Report "No metrics file found."

Step 3: Compare Actuals to Targets

For each criterion:

• Match metric key to criterion ID
• Compare actual value to threshold
• Update status

Step 4: Output Technical Gap Report

Separate by checkability:

ENGINEERING GOAL EVALUATION — {date}

═══ VERIFIABLE GAPS (run these tools to close) ═══

  ✗ E-001: Generation speed — not measured [semantic-qa]
  ✗ E-002: Build passes — not measured [build]
  ✓ E-003: Tests pass [test]

═══ OBSERVABLE GAPS (need production data) ═══

  ◌ E-004: Reliability — needs sustained production traffic

RECOMMENDATION:
  Priority: E-002 (build) → E-001 (generation speed)
  Run: implement fixes, then /shipkit-semantic-qa

Goal File Schema

goals/engineering.json

{
  "$schema": "shipkit-artifact",
  "type": "goals-engineering",
  "version": "4.0",
  "lastUpdated": "YYYY-MM-DDTHH:MM:SSZ",
  "source": "shipkit-engineering-goals",

  "derivedFrom": {
    "engineeringDefinition": ".shipkit/engineering-definition.json"
  },

  "criteria": [
    {
      "id": "E-001",
      "name": "API response time",
      "category": "technical-performance",
      "metric": "p95 response time",
      "threshold": "< 500ms",
      "rubric": [
        { "range": "> 5s", "meaning": "Unusable — users perceive system as broken" },
        { "range": "2-5s", "meaning": "Poor — noticeable delay, users may retry" },
        { "range": "500ms-2s", "meaning": "Acceptable — slight lag but functional" },
        { "range": "200-500ms", "meaning": "Good — feels responsive" },
        { "range": "< 200ms", "meaning": "Excellent — feels instant" }
      ],
      "currentValue": null,
      "verificationMethod": "automated-test",
      "checkability": "verifiable",
      "verificationTool": "test",
      "gate": "mvp-launch",
      "status": "not-measured",
      "derivedFrom": { "type": "mechanism", "id": "M-001" }
    }
  ],

  "summary": {
    "totalCriteria": 0,
    "byStatus": {},
    "byCheckability": {}
  }
}

Criterion ID Convention

When This Skill Integrates with Others

Before This Skill

• /shipkit-engineering-definition — Engineering blueprint (required)
• /shipkit-stage — Stage context + gates (recommended)
• /shipkit-product-goals — Product goals for alignment (recommended)

After This Skill

• /shipkit-plan — Criteria inform implementation priorities
• /shipkit-review-shipping — Can reference criteria when checking quality
• /shipkit-work-memory — Session continuity and progress tracking

Context Files This Skill Reads

Context Files This Skill Writes

Write Strategy: OVERWRITE

Creates/Updates:

• .shipkit/goals/engineering.json — Technical-performance criteria
• .shipkit/goals/strategic.json — Updates gate criteria arrays to include E-* IDs (if file exists)

Archive location (if replacing):

• .shipkit/.archive/goals-engineering.YYYY-MM-DD.json

After Completion

Guardrails Check: Before moving to next task, verify:

1. Persistence - Has important context been saved to .shipkit/?
2. Prerequisites - Does the next action need a spec or plan first?
3. Session length - Long session? Consider /shipkit-work-memory for continuity.

If NEEDS_ELICITATION:shipkit-engineering-goals was emitted: The skill paused without writing goals/engineering.json. The main session should run /shipkit-engineering-goals inline (where AskUserQuestion is available), answer the questions in .shipkit/elicitation/engineering-goals/questions.md, then re-invoke the original skill or orchestrator to resume. See install/shared/references/elicitation-protocol.md for full handling instructions.

Suggest skill when: User needs plans (/shipkit-plan), specs (/shipkit-spec), or verification (/shipkit-review-shipping).

Success Criteria

Engineering goals artifact is complete when:

• Engineering-definition.json read and mechanisms extracted
• Criteria derived from each mechanism (performance + quality + reliability)
• Infrastructure criteria included (build, test, lint as applicable)
• Each criterion has measurable threshold
• Each threshold has a rubric with 3-5 level descriptors explaining what each range looks like
• Each criterion has verification method
• Each criterion has checkability classification
• Each verifiable criterion has a verificationTool assigned
• Criteria mapped to existing stage gates (or new gates created)
• derivedFrom traceability links are valid
• Summary counts match actual array length
• File saved to .shipkit/goals/engineering.json
• Strategic.json gates updated with E-* criteria (if file exists)
• If marker emitted: NEEDS_ELICITATION:shipkit-engineering-goals is the final output line; goals/engineering.json was NOT written; questions.md written with specific threshold decisions needed

Schema version: 4.0. Split from unified shipkit-goals v3.1 — engineering goals are now owned by the Architect agent.

Backward compatibility: If reading goals files with source: "shipkit-goals", these are from the unified skill. The criteria schema is compatible; only source field and version differ.

Remember: Engineering goals measure technical performance — speed, reliability, quality, and health. These are the EM's responsibility. Strategic goals (S-) are owned by /shipkit-stage. Product outcomes (P-) are owned by /shipkit-product-goals.