Refresh Cache

Refresh the data cache

Clears the configured customer's local data lake, then re-pulls a fresh 60-day window so the next report runs against clean data. Before deleting anything, this flow always offers the lighter update-cache alternative – most people who reach for "refresh" actually just want the newest data, not a full wipe.

Two steps once a full refresh is confirmed:

1. Delete (refresh_cache.py) is stdlib-only and offline-safe – it never touches the network, so it can't fail on auth lapses, network errors, or S3 hiccups. It clears the lake root and the catalog; it will exit non-zero if bucket.json is missing (nothing to scope the deletion to); that's the only failure mode.
2. Re-pull (pull_parsely_dpl.py) requires fresh AWS SSO. Splitting the steps means a stale auth at re-pull time doesn't leave the user in a half-state – the lake is cleared either way, and the next report run (or a retry of this flow) will fill it back in. The pull also rebuilds the dpl_events catalog view.

If config itself looks broken (wrong bucket, missing profile), use the clear-configs skill instead and re-run the init flow.

Output discipline

This is housekeeping, with one exception: because deletion is destructive and irreversible, always confirm before running it (see step 2 below). Past that checkpoint, be terse – announce, run, stop. Don't narrate the script output line-by-line – the scripts print their own progress.

Steps

0. Resolve the plugin root. Every bash snippet below assumes $plugin_root is set in that shell invocation; re-run this line whenever you start a new shell:

   plugin_root="${CLAUDE_PLUGIN_ROOT:-${CURSOR_PLUGIN_ROOT:-<plugin-root>}}"
   plugin_python="${AGENTIC_ANALYTICS_VENV:-${AGENTIC_ANALYTICS_DATA_DIR:-$HOME/.local/share/agentic-analytics}/venv}/bin/python"
   [ -x "$plugin_python" ] || plugin_python="python3"
   

   <plugin-root> is the plugin's install directory – the directory two levels above this SKILL.md file. In a dev checkout of the source repo that's plugin/. $plugin_python is the plugin's isolated venv (created by the init flow at $AGENTIC_ANALYTICS_DATA_DIR/venv, default ~/.local/share/agentic-analytics/venv); it falls back to python3 if the venv doesn't exist yet.

1. Load bucket config. Read ${XDG_CONFIG_HOME:-~/.config}/agentic-analytics/bucket.json. It contains:

   • bucket – S3 bucket holding the customer's DPL events.
   • profile – AWS profile to authenticate with.
   • cache_dir – a slug identifying the customer's data (e.g. acme).

   If the file is missing, tell the user to run the init flow first (/agentic-analytics:init on harnesses with slash commands, or "set up agentic analytics" in chat) and stop. Do not guess values.

2. Offer the lighter alternative, every time. Before deleting anything, explain the trade-off and ask which the user wants. Always ask – don't skip this because the user's phrasing sounds decisive ("wipe it", "start over"); the cost of asking is one message, the cost of an unnecessary multi-gigabyte wipe-and-re-pull is real. Say something like:

   Refreshing the cache for '<cache_dir>' deletes everything currently cached and re-pulls the full 60-day window from S3 – can take a while. If you just want the newest data without losing what's already cached, update-cache does that instead and is much faster.

   Full refresh (delete + re-pull) – use this for stale/corrupt data, schema drift, or a bad prior pull. Or: update only (no deletion, just top up).

   • If the user picks update only: follow the update-cache skill's steps in this same turn (announce, then run pull_parsely_dpl.py incrementally with no delete step, then report) using the bucket/profile/cache_dir already loaded in step 1. Stop here – do not continue to step 3.
   • If the user picks full refresh: continue to step 3.

3. Announce. Output verbatim:

   Refreshing the local cache. Config (bucket.json, AWS profile) will be left alone.

4. Delete the cache. Runs the deletion script. Reports which dirs it cleared and the bytes freed, or (nothing to clear):

   "$plugin_python" "$plugin_root/scripts/refresh_cache.py"
   

5. Re-pull a fresh 60 days. Same window the staircase report targets. pull_parsely_dpl.py is incremental and idempotent; after a wipe it pulls the full window once and also rebuilds the dpl_events catalog view.

   read target_start target_end < <(python3 -c "
   from datetime import date, timedelta
   start = date.today() - timedelta(days=60)
   end = date.today() - timedelta(days=1)
   print(start.strftime('%Y-%m-%d'), end.strftime('%Y-%m-%d'))
   ")
   "$plugin_python" "$plugin_root/scripts/db/pull_parsely_dpl.py" \
     --bucket <bucket> [--site <site>] \
     --start "$target_start" --end "$target_end" \
     [--profile <profile>]
   

   Substitute <bucket> and <profile> with the values held from step 1. Omit --site to refresh all sites; pass --site <site> to limit the pull to one site. Do NOT add || true here – unlike the report path, a failed re-pull is the headline result of this flow; surface it.

6. Report done. One short line: "Cache rebuilt. Next report run uses fresh data."

   If the re-pull failed (auth lapse, S3 error, no events for the window), say so plainly and tell the user the lake is now empty – the next report run will try to pull again, or they can re-run this flow once auth is fresh.

Notes

• Only the currently configured customer's DPL data is affected. The lake root holds all sources/sites pooled under $AGENTIC_ANALYTICS_DATA_DIR (default ~/.local/share/agentic-analytics); deletion is scoped to the dpl/events/ subtree and the catalog, so other data you may later land in the lake is untouched. To work with a different customer, switch bucket.json first.
• Deletion is not scoped to a single site even when the re-pull afterward is – refresh_cache.py always clears the whole dpl/events/ subtree (every site/apikey under the configured bucket). Passing --site <site> to step 5 only limits what gets re-pulled, not what got deleted. If you only want to top up or repair one site's data without touching the rest, use update-cache instead – it never deletes.
• The DuckDB catalog (catalog.duckdb and its .wal sidecar) is cleared too. It's derived state any report can regenerate by re-running pull_parsely_dpl.py.
• Legacy gz cache sweep: if you're upgrading from a pre-lake version, this flow also clears the old gz-JSON cache at ~/.cache/agentic-analytics/dpl/<cache_dir>/ and the duckdb-tmp/ working dir (scoped to your cache_dir). The lake code never reads them; sweeping just reclaims the orphaned disk. No-op for fresh installs.