Refreshing The Org Vault

Refreshing the org vault

Overview

sfi refresh is the only way the vault learns about new metadata. The pipeline runs five stages in order, and all five have to finish for the vault to be usable:

1. Pull — sf project retrieve start --manifest <pkg.xml> --target-org <alias> --output-dir org-kb/source/. Network step. The slow one for real orgs.
2. Extract — walk org-kb/source/, dispatch each file to the matching extractor (103 component types including Reports, FlexiPages, Permission Set Groups, OmniStudio, CPQ, etc.). Per-file failures are captured, not fatal.
3. Import — load extracted nodes and edges into the DuckDB graph at org-kb/graph/graph.duckdb.
4. Render — write Markdown component files under org-kb/components/<Type>/..., plus Apex/Flow source pages and the vault index.
5. Manifest — recompute the source-tree hash, stamp refreshedAt, write org-kb/meta/manifest.json (including coverage per metadata type when the retrieve result is available).

Optional after a successful refresh: sfi snapshot create --label <name> so sfi.trend / sfi.diff_snapshots can compare structural history later.

A real org takes 5–15 minutes end-to-end, dominated by stage 1. Tell the user this before you start — silence during a long retrieve reads like a hang.

When to fire

Explicit:

• The user typed /sfi-refresh.

Conversational triggers:

• "Refresh the vault" / "refresh the index" / "rebuild the vault".
• "Pull the latest from my org" / "pull latest".
• "My data is stale" / "the vault is stale" / "this is out of date".
• "I just deployed changes" / "I just pushed metadata" / "things changed in the org".
• sfi.health_check returns status: 'degraded' or freshness.stale: true. Tell the user to run /sfi-refresh. status: 'unhealthy' or checks.vaultExists: false → route to /sfi-init.

When NOT to fire:

• The user is asking a question and the vault is already fresh. Answer the question first; only refresh if health_check flags staleness.
• The user wants live record data ("how many Accounts closed today?"). Refresh pulls metadata, not records. Record counts need the opt-in live plane (sfi.live_*) or sf data query.
• There is no org-kb/ directory yet. The user hasn't initialized the vault — point them to /sfi-init, not refresh.

Pre-flight checks

Run these before invoking sfi refresh. Each check that fails stops the flow; do not proceed.

1. sf CLI authentication

Run:

sf org list --json

Parse the JSON for the alias the user is refreshing (or the default). If the alias is not present, or its connectedStatus is not Connected, stop and tell the user:

"The sf CLI isn't authenticated to <alias>. Run sf org login web --alias <alias> (or sf org login web --set-default --alias <alias> if this should be the default), then ask me to refresh again."

If sf itself isn't installed (command not found), point the user to the Salesforce CLI install docs and stop.

2. Vault config exists

Confirm org-kb/meta/config.json exists and parses. The CLI does this too, but checking first lets you surface a clearer message:

"No vault config at org-kb/meta/config.json. Run /sfi-init first."

3. Target alias matches (when the user specified one)

If the user said "refresh from <alias>" and the config's targetOrg is a different alias, ask before overriding:

"Config has targetOrg: <config-alias>. You asked me to refresh from <user-alias>. I'll pass --target-org <user-alias> for this run; want me to update the config too?"

Don't silently override. The config is the persisted intent.

Steps

1. Run the pre-flight checks above. If any fails, stop and surface the actionable message.

2. Set expectations. Before running the command, say something like:

   "Running sfi refresh against <alias>. This pulls metadata from Salesforce, runs nine extractors, rebuilds the graph, and re-renders the vault. Expect 5–15 minutes for a real org; faster for a small fixture org." If the user passed --no-pull (or said "the source tree is already populated, just re-extract"), say so: "Skipping sf project retrieve (you passed --no-pull). Re-extracting from the existing org-kb/source/ tree."

3. Run the command. Forward whatever flags the user provided:

   • --target-org <alias> if specified.
   • --no-pull if the user said the source tree is already there.
   • --types <list> (comma-separated) if the user wants a subset refreshed.

   Example invocations:

   sfi refresh
   sfi refresh --target-org admin@example.com
   sfi refresh --no-pull
   sfi refresh --types CustomObject,CustomField,ValidationRule
   

4. Read the new manifest. After the command exits, read org-kb/meta/manifest.json. Report:

   • refreshedAt
   • sourceOrg
   • components totals by type
   • edges totals by type
   • sourceTreeHash (last 12 chars are fine; full hash is noisy)

   If a previous manifest existed, compare counts and surface the delta ("CustomField: 312 → 318, +6"). Do this honestly — if a count dropped, say so; it usually means a metadata type was removed from the org (or a --types filter excluded it).

5. Handle non-success outcomes. The CLI exits non-zero on partial and failed. See Failure modes below.

Failure modes

When you don't recognize the error, paste the Fatal: line back to the user and ask whether they want to retry or investigate. Do not guess a recovery path.

Common rationalizations

Red flags

Stop and ask the user when:

• Two retries of sfi refresh both fail with the same Fatal: line. The third attempt won't help; surface the message and ask what the user wants to do.
• The new manifest's component counts are drastically lower than the previous manifest (e.g., CustomField: 312 → 8) and the user did not pass --types. Something pulled an empty package; ask before treating this as the new ground truth.
• sf org list shows the target alias as Connected but connectedStatus is followed by a refresh-token expiry warning. Surface the warning; the next retrieve may succeed once before failing.
• The user wants you to "refresh just one object" or "just the fields on Account." The --types filter scopes by metadata type, not by individual component. Tell them: refresh is all-or-nothing within a type.
• The user asks you to commit the regenerated vault. That's their call — don't git add / git commit on their behalf. Mention that org-kb/components/, org-kb/meta/manifest.json, and org-kb/graph/graph.duckdb are the typical commit targets if they want a diff-able snapshot.

Verification

Before telling the user the refresh is done, confirm:

• Did I run sf org list --json (or read prior output) and confirm the target alias is Connected?
• Did I tell the user — before running the command — that this could take 5–15 minutes?
• Did I read org-kb/meta/manifest.json after the run and report the actual refreshedAt, sourceOrg, and counts?
• If status is partial, did I list the per-file errors with their error.kind, instead of summarizing as "mostly worked"?
• If status is failed, did I surface the Fatal: line verbatim and name the recovery path?
• Did I avoid claiming any state about the org's current contents that I didn't read from the new manifest or vault?

Grounding & routing (shared contract). For a vague or broad ask, call sfi.route_question first — in the default hybrid mode it returns a meaning-ranked toolCandidates shortlist (which YOU pick from) plus a suggested plane and a route hint (and whether to sfi.resolve a name first). Default tool profile is core: only the core spine (including sfi.live_consent) is directly invokable. For every other sfi.* analysis, call sfi.run_analysis with { "name": "sfi.<tool>", "args": { … } } (or follow route_question.invoke, which already wraps non-core steps). Optional: sfi.describe_analysis first when args are unclear. Every org fact must come from an sfi.* tool call, cited by its canonical id — never from memory. Build the answer only from what the tools returned, then pass it through sfi.synthesize_answer, which flags any hallucinatedIds (canonical ids no tool produced). Full cascade: using-sf-intelligence.