Update Cache

Update the data cache

Pulls a fresh 60-day window into the local data lake, incrementally – it only fetches days that aren't already on disk. Nothing is deleted. Safe to run any time; already-synced customers pay milliseconds, a fresh or partial lake pays for whatever's missing.

If the data looks wrong rather than just behind (stale numbers that don't move, a bad prior pull, schema drift), use the refresh-cache skill instead – this flow won't fix corrupted or malformed local data, only fill gaps.

Output discipline

This is housekeeping. Be terse. Announce, run, stop. Don't narrate the script output line-by-line – the script prints its own progress.

Steps

0. Resolve the plugin root. Every bash snippet below assumes $plugin_root is set in that shell invocation; re-run this line whenever you start a new shell:

   plugin_root="${CLAUDE_PLUGIN_ROOT:-${CURSOR_PLUGIN_ROOT:-<plugin-root>}}"
   plugin_python="${AGENTIC_ANALYTICS_VENV:-${AGENTIC_ANALYTICS_DATA_DIR:-$HOME/.local/share/agentic-analytics}/venv}/bin/python"
   [ -x "$plugin_python" ] || plugin_python="python3"
   

   <plugin-root> is the plugin's install directory – the directory two levels above this SKILL.md file. In a dev checkout of the source repo that's plugin/. $plugin_python is the plugin's isolated venv (created by the init flow at $AGENTIC_ANALYTICS_DATA_DIR/venv, default ~/.local/share/agentic-analytics/venv); it falls back to python3 if the venv doesn't exist yet.

1. Announce. Output verbatim:

   Updating the local cache. Nothing will be deleted – only missing days are pulled.

2. Load bucket config. Read ${XDG_CONFIG_HOME:-~/.config}/agentic-analytics/bucket.json. It contains:

   • bucket – S3 bucket holding the customer's DPL events.
   • profile – AWS profile to authenticate with.
   • cache_dir – a slug identifying the customer's data (e.g. acme).

   If the file is missing, tell the user to run the init flow first (/agentic-analytics:init on harnesses with slash commands, or "set up agentic analytics" in chat) and stop. Do not guess values.

3. Pull a fresh 60 days. Same window the staircase report targets. pull_parsely_dpl.py is incremental and idempotent: it checks the lake for each day's parquet partition and only fetches what's missing. It also rebuilds the dpl_events catalog view.

   read target_start target_end < <(python3 -c "
   from datetime import date, timedelta
   start = date.today() - timedelta(days=60)
   end = date.today() - timedelta(days=1)
   print(start.strftime('%Y-%m-%d'), end.strftime('%Y-%m-%d'))
   ")
   "$plugin_python" "$plugin_root/scripts/db/pull_parsely_dpl.py" \
     --bucket <bucket> [--site <site>] \
     --start "$target_start" --end "$target_end" \
     [--profile <profile>]
   

   Substitute <bucket> and <profile> with the values held from step 2. Omit --site to update all sites; pass --site <site> to limit the pull to one site. Do NOT add || true here – unlike the report path, a failed pull is the headline result of this flow; surface it.

4. Report done. One short line: "Cache updated. Next report run uses the latest data."

   If the pull failed (auth lapse, S3 error), say so plainly. Nothing was deleted, so whatever was already cached is still there and usable – the failure only means new data wasn't added this time.

Notes

• Only the currently configured customer's DPL data is affected.
• The DuckDB catalog view (dpl_events) is rebuilt after the pull so it reflects any newly-synced partitions.
• This flow never deletes local files. If you suspect the local cache itself is corrupted or inconsistent (not just behind), use refresh-cache instead.