CodeMie Analytics Skill
You are an analytics expert for the CodeMie (EPAM AI/Run) platform. You know every analytics
API endpoint, how to call LiteLLM directly, and how to orchestrate data into a final report.
The plumbing (config lookup, SSO credential decryption, token refresh messaging) lives in
scripts/analytics-cli.js. You never need to touch those details — just invoke the CLI and
react to what it prints.
Step 1 — Understand what the user wants
Identify the analytics scenario. The CLI supports these command families:
Leaderboard (AI Champions)
The leaderboard ranks users across 6 scoring dimensions:
- D1: Core Platform Usage (20%) — conversations, assistant interactions
- D2: Core Platform Creation (20%) — assistants, datasources created
- D3: Workflow Usage (10%) — workflow executions
- D4: Workflow Creation (10%) — workflows authored
- D5: CLI & Agentic Engineering (30%) — coding agent sessions, tokens, repos
- D6: Impact & Knowledge (10%) — marketplace publishing, knowledge sharing
Tiers: pioneer (80+), expert (65+), advanced (45+), practitioner (25+), newcomer (<25)
| Scenario |
Command |
What it retrieves |
| Full leaderboard (paginated, filterable) |
leaderboard |
data.rows[] — rank, user_name, total_score, tier_name, score_delta, dimensions[] (id/score/weight per D1–D6), summary_metrics{} (cli_sessions, active_days, total_lines_added, total_spend, web_conversations, …) |
| Leaderboard KPI summary |
leaderboard-summary |
data — total_users, avg_score, top_score, tier_counts{} (pioneer/expert/advanced/practitioner/newcomer counts and percentages) |
| Single user champion profile |
leaderboard-user <id|email> |
data — same shape as a leaderboard row but for one user; includes full dimension breakdown and all summary_metrics |
| Tier distribution |
leaderboard-tiers |
data.rows[] — tier_name, user_count, percentage; one row per tier |
| Average dimension scores |
leaderboard-dimensions |
data.rows[] — dimension id/label, avg_score, weight; one row per D1–D6 |
| Top N performers |
leaderboard-top [limit] |
data.rows[] — same shape as leaderboard rows, limited to top N (max 50, default 10) |
| Score histogram |
leaderboard-scores |
data.rows[] — score_range (e.g. "0-10"), user_count; one row per 10-point bin |
| Framework metadata |
leaderboard-framework |
data.framework{} — title, principles, calculation_steps; data.tiers[] — name, label, min_score; data.dimensions[] — id, label, weight, description |
| Computation snapshots |
leaderboard-snapshots |
data.rows[] — snapshot_id, created_at, status, period_start, period_end, user_count |
| Available seasons |
leaderboard-seasons --view monthly|quarterly |
data.rows[] — season_key (e.g. "2026-03"), label, start_date, end_date |
Leaderboard filters: --view (current/monthly/quarterly), --season-key (2026-03, 2026-Q1),
--tier, --intent (cli_focused/platform_focused/hybrid/sdlc_unicorn), --search, --sort-by, --sort-order.
CLI Insights
| Scenario |
Command |
What it retrieves |
| Full CLI overview (agents, repos, tools, errors) |
cli-insights |
Composite object with sub-keys: summary (total sessions, cost, tokens, repos, users), agents[] (agent name + session count), top_users[], top_repos[], errors[], llms[] |
| User classification & top spenders |
cli-insights-users |
data.rows[] — user_name, classification (cli_focused/platform_focused/hybrid/sdlc_unicorn), total_cost, session_count, token_count |
| Detailed single-user CLI profile |
cli-insights-user <name> |
Composite: key_metrics (sessions, cost, tokens, repos, tools), tools[], models[], repositories[], workflow_intent, category_breakdown[] |
| Project classification & top by cost |
cli-insights-projects |
data.rows[] — project_name, classification, total_cost, session_count, user_count |
| Usage patterns (weekday, hourly, session depth) |
cli-insights-patterns |
Composite with sub-keys: weekday.data.rows[] (weekday_name, session_count), hourly.data.rows[] (hour_utc, session_count), session_depth.data.rows[] (depth_bucket, count) |
General Analytics
| Scenario |
Command |
What it retrieves |
| Overall usage summary (tokens, cost, users) |
summaries |
data — total_cost, total_tokens, total_requests, unique_users (MAU), unique_users_daily (DAU), cli_invocations, assistants_count, workflows_count, skills_count, mcp_servers_count |
| User list + activity trends |
users |
data.rows[] — user_name, email, total_cost, total_tokens, last_active; plus activity[] time-series |
| Per-project spending |
projects-spending |
data.rows[] — project_name, total_cost, total_tokens, user_count, request_count |
| Per-project activity time-series |
projects-activity |
Composite with activity.data.rows[] and uniqueDaily.data.rows[] |
| LLM model breakdown |
llms-usage |
data.rows[] — model_name, request_count, total_tokens, input_tokens, output_tokens, total_cost |
| Tool usage |
tools-usage |
data.rows[] — tool_name, invocation_count, success_count, error_count, total_tokens |
| Workflow execution analytics |
workflows |
data.rows[] — workflow_name, run_count, success_count, failure_count, avg_duration_ms, total_cost |
| Agent execution analytics |
agents-usage |
data.rows[] — assistant_name, execution_count, total_cost, total_tokens |
| Embedding model usage |
embeddings-usage |
data.rows[] — model_name, request_count, total_tokens, total_cost |
| Chat assistant conversations |
assistants-chats |
data.rows[] — assistant, conversation_count, user_count, total_cost |
| Webhook invocation analytics |
webhooks-usage |
data.rows[] — user_id, invocation_count, total_cost |
| MCP server usage |
mcp-servers |
data.rows[] — mcp_name, request_count, user_count, total_cost |
| MCP server usage by user |
mcp-servers-by-users |
data.rows[] — user_name, mcp_name, request_count |
| Power user analytics |
power-users |
data.rows[] — user_email, session_count, total_cost, features_used |
| Knowledge sharing metrics |
knowledge-sharing |
data.rows[] — user_email, shared_count, viewed_count |
| Top agents by usage |
top-agents |
data.rows[] — assistant_name, execution_count, total_cost |
| Top workflows by usage |
top-workflows |
data.rows[] — workflow_name, run_count, total_cost |
| Assets published to marketplace |
marketplace |
data.rows[] — user_email, asset_name, published_at |
| Budget alerts (soft + hard limits) |
budget |
Composite with soft.data.rows[] and hard.data.rows[] — user_email, max_spent (users approaching or over limit) |
| Personal spending & budget |
spending |
data — current_spend, budget_limit, hard_budget_limit, budget_reset_at, percentage_used |
| Per-user spending (platform + cli split) |
spending-by-users |
Composite with platform.data.rows[] and cli.data.rows[] — user_name/email, total_cost, token_count |
| Weekly engagement histogram |
engagement |
data.rows[] — day_label, hour_start, feature_type, session_count, cost; covers last 7 days in 3-hour intervals |
LiteLLM & CSV Enrichment
| Scenario |
Command |
Output |
| LiteLLM customer lookup |
litellm-customer [user_id] |
JSON |
| LiteLLM spend logs |
litellm-spend |
Spend entries |
| LiteLLM virtual keys |
litellm-keys |
Key info |
| Enrich CSV/Excel with LiteLLM costs |
enrich-csv <file> |
Enriched table |
| Unlisted / experimental endpoint |
custom /v1/analytics/<path> |
Raw JSON — use a named command if one exists |
Security
Never include raw API keys, bearer tokens, cookie values, LiteLLM keys, or session
credentials in your responses to the user. If CLI output contains sensitive fields
(e.g. from litellm-keys), the CLI automatically redacts them — but if you encounter
any token, key, or secret in raw output, redact it before displaying. Never run env,
printenv, or similar commands that could expose LITELLM_KEY or CODEMIE_API_KEY
to the conversation context.
Step 2 — Run the analytics CLI
The CLI script lives at scripts/analytics-cli.js next to this skill. It handles
authentication internally. If something is wrong with credentials, it prints a clear
actionable message to stderr; pass that along to the user verbatim.
node ${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/scripts/analytics-cli.js <command> [options]
LiteLLM commands (litellm-*, enrich-csv) require LITELLM_URL + LITELLM_KEY env vars.
Collecting data for reports (batch pattern)
Only fetch the data the report actually needs. Read the user's request, identify which
sections the report requires, and collect only the relevant endpoints. Do not run every
available command — unnecessary fetches waste time and tokens.
When building an HTML report, run all needed CLI commands in a single Bash call using
--save <filepath> on each. This triggers one permission prompt for the entire
collection phase and keeps API responses out of the conversation context.
Directory layout
Every report lives in its own folder under reports/. Derive the folder name from the
current date and a short kebab-case description of the report:
reports/
2026-05-07-cli-usage/ ← report folder (date + name)
cli-usage.html ← the HTML report (saved here directly)
temp/ ← all temp/data files go here
summaries.json
summaries.schema.json
cli-insights.json
...
Never overwrite an existing report folder. Always resolve a free name before creating
anything. Use a suffix loop — this is mandatory, not optional:
BASE=reports/$(date +%Y-%m-%d)-<short-name>
REPORT_DIR=$BASE
n=2
while [ -d "$REPORT_DIR" ]; do REPORT_DIR="${BASE}-${n}"; n=$((n+1)); done
OUT="$REPORT_DIR/temp"
Then run only the commands the report needs:
CLI=${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/scripts/analytics-cli.js
mkdir -p "$OUT" && \
node $CLI summaries --save "$OUT/summaries.json" && \
node $CLI cli-insights --save "$OUT/cli-insights.json" && \
# ... only endpoints needed for this report ...
echo "✓ All data saved → $OUT"
Each command prints: ✓ Saved → <path>. The final echo confirms the directory
path — save it, you will reference it in every subsequent step.
Do not cat, Read, or print the saved JSON files into the conversation.
Raw API responses can be hundreds of KB. Use Step 2.5 to inspect structure instead.
Temp files are not cleaned up automatically; Step 3b asks the user whether to delete them once
the report is built.
Common filter flags
| Flag |
Example |
Notes |
--time-period |
last_30_days |
Predefined period |
--start-date |
2024-01-01T00:00:00 |
Custom range start |
--end-date |
2024-03-31T23:59:59 |
Custom range end |
--users |
alice,bob |
Comma-separated usernames |
--projects |
my-project |
Comma-separated project names |
--page |
1 |
Pagination |
--per-page |
100 |
Results per page (default 50) |
--output |
json |
json | table | csv |
--pretty |
(flag) |
Pretty-print JSON |
Leaderboard-specific flags
| Flag |
Example |
Notes |
--view |
monthly |
current | monthly | quarterly |
--season-key |
2026-Q1 |
Specific season to query |
--tier |
pioneer |
Filter by tier |
--intent |
cli_focused |
Filter by user intent profile |
--search |
john |
Partial name/email search |
--sort-by |
total_score |
rank | total_score | user_name | tier_level |
--sort-order |
desc |
asc | desc |
--limit |
20 |
Max entries for leaderboard-top (max 50) |
Example invocations
CLI=${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/scripts/analytics-cli.js
# Full leaderboard — top 50 pioneers sorted by score
node $CLI leaderboard --tier pioneer --sort-by total_score --sort-order desc --per-page 50 --pretty
# Single user champion profile
node $CLI leaderboard-user user@example.com --pretty
# Leaderboard KPI summary for Q1 2026
node $CLI leaderboard-summary --view quarterly --season-key 2026-Q1 --pretty
# Dimension averages (D1–D6) for current snapshot
node $CLI leaderboard-dimensions --pretty
# Tier distribution
node $CLI leaderboard-tiers --pretty
# Top 10 performers
node $CLI leaderboard-top 10 --pretty
# 30-day platform summary
node $CLI summaries --time-period last_30_days --pretty
# Full CLI insights
node $CLI cli-insights --time-period last_30_days --pretty
# Detailed CLI profile for a specific user
node $CLI cli-insights-user alice@example.com --time-period last_30_days --pretty
# Usage patterns (weekday + hourly + session depth)
node $CLI cli-insights-patterns --time-period last_30_days --pretty
# Per-user spending breakdown
node $CLI spending-by-users --time-period last_30_days --pretty
# Custom endpoint
node $CLI custom /v1/analytics/mcp-servers --time-period last_30_days --pretty
Step 2.5 — Inspect data structure
After collection, run inspect-schema.js to generate a compact .schema.json file
alongside each saved JSON — one permission prompt, no data in context:
node ${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/scripts/inspect-schema.js "$OUT"
Output is a short listing of generated schema files with sizes, for example:
Schemas written to: reports/2026-05-07-cli-usage/temp/
✓ leaderboard-top.schema.json (1.3 KB)
✓ summaries.schema.json (0.4 KB)
...
Then read only the .schema.json files you actually need for the report using the
Read tool. Each schema shows field names, types, array lengths, and string samples —
enough to write extraction code without touching the raw responses:
CRITICAL — schema is the source of truth for field names.
Never use field names or metric IDs from this skill's documentation when writing
report code. Documentation can be stale. The .schema.json files are generated from
live API responses and are always correct. Every id, key, or column name referenced
in JS must be verified against the schema you just read — not assumed from the tables
above. If the schema contradicts the docs, trust the schema.
{
"_envelope": "[envelope] data_as_of: '2026-05-07T02:00:10', total_count: 50",
"data": {
"rows": {
"_type": "array",
"_count": 50,
"_item": {
"rank": "number",
"user_name": "string ~ 'Jane Smith'",
"total_score": "number",
"tier_name": "string ~ 'expert'",
"score_delta": "number | null",
"dimensions": { "_type": "array", "_count": 6, "_item": { "id": "string ~ 'd1'", "score": "number", "weight": "number" } },
"summary_metrics": { "cli_sessions": "number", "active_days": "number", "total_lines_added": "number", "total_spend": "number" }
}
}
}
}
Step 3 — Build the HTML report
Save the HTML report directly inside $REPORT_DIR (not in temp/).
Use a kebab-case filename matching the folder name:
reports/2026-05-07-cli-usage/cli-usage.html
reports/2026-05-07-leaderboard/leaderboard.html
reports/2026-05-07-spending/spending.html
Step 3a — Hand off to codemie-html-report
Invoke the codemie-html-report skill. Pass:
- The schemas inspected in Step 2.5 — field names and structure.
- The user's intent — e.g. "leaderboard dashboard with tier distribution".
- Timestamp context —
_envelope lines in schemas include data_as_of.
- The data folder —
$OUT, holding the JSON files saved in Step 2.
- Output path — e.g.
$REPORT_DIR/leaderboard.html.
Ask it to build the page per codemie-html-report's references/analytics.md, which owns the
data placeholders, the style placeholder and the order of the build scripts.
The layout references below give structure and data only. CSS, fonts and Chart.js come from the
codemie-html-report build, which overrides any instruction in them to inline CSS or load a CDN.
Step 3b — Temp file cleanup
After codemie-html-report has built the report, always ask the user:
The report is ready at <path>. The temp/ directory ($OUT) contains the raw
API response files used to build it (~N files). Would you like to delete it?
These files can hold per-user spending and leaderboard data. If the user says yes,
delete the temp directory:
rm -rf "$OUT"
# e.g. rm -rf "reports/2026-05-07-executive-spending/temp"
Confirm deletion:
✓ Temp files deleted → reports/2026-05-07-executive-spending/temp
If the user says no, leave the files in place and tell them the path.
Full API Reference
Leaderboard endpoints (GET /v1/analytics/leaderboard/...)
Admin-only. All accept snapshot_id, view, season_key query params.
| Endpoint |
Additional Params |
Returns |
/leaderboard/summary |
— |
Total users, tier counts, top score |
/leaderboard/entries |
tier, search, intent, sort_by, sort_order, page, per_page |
Paginated ranked entries |
/leaderboard/user/{user_id} |
path: user ID or email |
Full user profile with D1–D6 breakdown |
/leaderboard/tiers |
— |
Tier name, count, percentage |
/leaderboard/scores |
— |
Score histogram (10-point bins) |
/leaderboard/dimensions |
— |
Average D1–D6 scores |
/leaderboard/top-performers |
limit (default 3, max 50) |
Top N by total score |
/leaderboard/snapshots |
view, status, is_final, page, per_page |
Computation snapshots |
/leaderboard/seasons |
view (required: monthly/quarterly), page, per_page |
Available seasonal periods |
/leaderboard/framework |
— |
Static metadata: dimensions, tiers, intents, scoring |
/leaderboard/compute (POST) |
period_days, view, season_key |
Triggers manual computation |
CLI Insights endpoints (GET /v1/analytics/cli-insights-...)
| Endpoint |
Params |
Returns |
/cli-insights-weekday-pattern |
time filters |
Weekday usage patterns |
/cli-insights-hourly-usage |
time filters |
Hourly usage patterns |
/cli-insights-session-depth |
time filters |
Session depth distribution |
/cli-insights-user-classification |
time filters |
User classification breakdown |
/cli-insights-top-users-by-cost |
time filters |
Top users ranked by cost |
/cli-insights-top-spenders |
time filters |
Top spenders |
/cli-insights-users |
time filters |
CLI user list |
/cli-insights-user-detail |
user_name (required), user_id |
Full user detail |
/cli-insights-user-key-metrics |
user_name (required), user_id |
User KPIs |
/cli-insights-user-tools |
user_name (required), user_id |
User tool usage |
/cli-insights-user-models |
user_name (required), user_id |
User model usage |
/cli-insights-user-workflow-intent |
user_name (required), user_id |
User workflow intent |
/cli-insights-user-classification-detail |
user_name (required), user_id |
User classification detail |
/cli-insights-user-category-breakdown |
user_name (required), user_id |
User category breakdown |
/cli-insights-user-repositories |
user_name (required), user_id |
User repositories |
/cli-insights-project-classification |
time filters |
Project classification |
/cli-insights-top-projects-by-cost |
time filters |
Top projects by cost |
Standard CLI analytics (GET /v1/analytics/cli-...)
| Endpoint |
Returns |
/cli-summary |
CLI totals (tokens, cost, sessions) |
/cli-agents |
Agent breakdown |
/cli-llms |
Model breakdown |
/cli-users |
CLI user activity |
/cli-errors |
Error logs |
/cli-repositories |
Repo activity |
/cli-top-performers |
Top by lines added |
/cli-top-versions |
CLI version distribution |
/cli-top-proxy-endpoints |
LiteLLM endpoint usage |
/cli-tools |
Tool usage |
Dashboard analytics (GET /v1/analytics/...)
All accept time filters + users + projects + page + per_page.
| Endpoint |
Returns |
/summaries |
Platform totals (tokens, cost, unique users) |
/users-spending |
Per-user cost + tokens |
/users-activity |
Activity time-series |
/users-unique-daily |
Unique users/day |
/users |
User list |
/projects-spending |
Per-project spending |
/projects-activity |
Project activity time-series |
/projects-unique-daily |
Unique projects/day |
/llms-usage |
LLM model usage |
/tools-usage |
Tool usage |
/workflows |
Workflow runs |
/agents-usage |
Agent executions |
/embeddings-usage |
Embedding model usage |
/assistants-chats |
Chat assistant conversations |
/webhooks-invocation |
Webhook usage |
/mcp-servers |
MCP server usage |
/mcp-servers-by-users |
MCP by user |
/power-users |
Power user analytics |
/knowledge-sharing |
Knowledge sharing metrics |
/top-agents-usage |
Top agents |
/top-workflow-usage |
Top workflows |
/published-to-marketplace |
Marketplace publishing |
Spending & Budget endpoints
| Endpoint |
Method |
Returns |
/spending |
GET |
Current user: spend, budget_limit, hard_budget_limit, reset time |
/budget_usage |
GET |
Per-key budget rows with % used |
/budget-soft-limit |
GET |
Soft limit warnings |
/budget-hard-limit |
GET |
Hard limit hits |
/spending/by-users/platform |
GET |
Per-user platform spending |
/spending/by-users/cli |
GET |
Per-user CLI spending |
Engagement
| Endpoint |
Method |
Returns |
/engagement/weekly-histogram |
GET |
3h intervals, last 7 days, by feature type |
LiteLLM endpoints (via LITELLM_URL + LITELLM_KEY)
| Endpoint |
Method |
Params |
Description |
/customer/info |
GET |
user_id |
Customer spend + budget + allowed models |
/spend/logs |
GET |
start_date, end_date, user_id |
Spend log entries |
/key/info |
GET |
key |
Virtual key details + spend |
/model/info |
GET |
— |
Available models |
/health |
GET |
— |
Proxy health |
Response envelope
Most CodeMie endpoints return:
{
"data": { ... },
"metadata": {
"timestamp": "2024-03-15T12:00:00Z",
"data_as_of": "2024-03-15T12:00:00Z",
"filters_applied": {},
"execution_time_ms": 45.2
}
}
Always extract response.data for the actual payload.
Custom analytics requests
For endpoints not covered by preset commands:
node analytics-cli.js custom /v1/analytics/mcp-servers --time-period last_30_days
# POST endpoints
node analytics-cli.js custom /v1/analytics/ai-adoption-overview --method POST \
--time-period last_30_days
Offline CLI analytics (no API key needed)
The codemie analytics CLI command reads local session files from ~/.codemie/sessions/
with no API calls:
codemie analytics --last 7d --output json
codemie analytics --agent claude --last 30d --export csv
Report References
Reference files in references/ describe canonical report layouts. Always check the
relevant reference before building a new HTML report — it defines the exact components,
charts, data structure, and modal design to use, ensuring consistency across users.
| Report type |
Reference file |
When to use |
| Leaderboard dashboard |
${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/references/leaderboard-dashboard-report.md |
Any request for leaderboard rankings, AI champions, top performers, tier distribution |
| People spending dashboard |
${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/references/people-spending-dashboard-report.md |
Any request to track LiteLLM costs for a specific list of users (cohort, team, bootcamp, project) |
Use Cases
Use Case: People Spending Dashboard (cohort / team / bootcamp)
Trigger phrases: "build a spending dashboard for people from X", "track LiteLLM costs
for a list of users", "how much did this team spend", "bootcamp spending report",
"costs for people in this CSV/Excel".
Also applies when: the user asks to enrich analytics with EPAM employee data, map
platform users to EPAM people, or look up user assignments/org details.
⚠️ EPAM People & Assignments Finder — required for this use case only
Before proceeding, verify the assistant is accessible:
node ${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/scripts/analytics-cli.js \
custom /v1/assistants/5ca384d0-d042-480c-a0a9-d28150e2352f 2>&1 | head -5
If the command returns an auth error, HTTP 401/403/404, or "No CodeMie credentials" —
stop. Do not run anything else. Notify the user:
⛔ EPAM People & Assignments Finder assistant is not configured on your account.
Assistant: EPAM People & Assignments Finder
ID: 5ca384d0-d042-480c-a0a9-d28150e2352f
Add it with:
codemie assistants add 5ca384d0-d042-480c-a0a9-d28150e2352f
Or open in browser and click Add to my assistants:
https://codemie.lab.epam.com/#/assistants/5ca384d0-d042-480c-a0a9-d28150e2352f
Once added, resume this session with:
claude -f
Full workflow (see ${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/references/people-spending-dashboard-report.md for all details):
- Parse the list from Excel/CSV using
openpyxl. Skip header and TOTAL rows.
- Fetch 3 LiteLLM accounts per user using Python
asyncio + aiohttp (semaphore 25,
ssl=False). Account patterns:
- Web:
email (plain)
- CLI:
email_codemie_cli
- Premium:
email_codemie_premium_models
Use end_user_id param (not user_id) on GET /customer/info.
- Save raw results to
/tmp/ to avoid re-fetching on HTML rebuild.
- Build users array — sum three spend values, extract budget fields per account.
- Fetch leaderboard — paginate ALL pages (
--per-page 500), ~25 pages for 12k users.
Expect ~60% of a typical cohort to appear.
- Fetch CLI insights —
cli-insights-users --per-page 500 topBySpend for top CLI users.
Expect ~3–5% coverage for a general cohort.
- Compute KPIs — grand total, per-type totals, active user count, avg spend.
Budget projection:
avg_spend_per_active × total_users × 1.20.
- Generate HTML — use
str.replace() with __TOKEN__ markers (never f-strings, which
conflict with JS ${...} template literals).
- Wire table clicks — use
data-email attribute + event delegation (never onclick=""
attributes, which break under Python quote escaping).
- Save to
reports/<date>-<name>/<name>.html (temp/data files in reports/<date>-<name>/temp/).
Key commands:
CLI=${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/scripts/analytics-cli.js
# Leaderboard (run in a loop for all pages)
node $CLI leaderboard --per-page 500 --page <N> --output json
# CLI top spenders
node $CLI cli-insights-users --time-period last_30_days --per-page 500 --output json
LiteLLM fetch requires Python (not the analytics CLI) because it needs
LITELLM_URL + LITELLM_KEY env vars and concurrent calls for 1,000+ accounts.
Tips
- Always run the CLI first, capture JSON, then hand it to the report skill — don't
hardcode example data.
- If a command returns paginated data, loop through all pages or set
--per-page 500.
- For time-series charts, use
/users-unique-daily or /projects-unique-daily endpoints.
- Budget warnings: flag rows where
spend / max_budget > 0.8 (warn) and > 1.0 (error).
- For the leaderboard dashboard, combine
leaderboard + leaderboard-summary +
leaderboard-tiers + leaderboard-dimensions to build a comprehensive view. Then follow
${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/references/leaderboard-dashboard-report.md for the exact HTML structure.
- For a people spending dashboard, fetch LiteLLM directly with Python async (3 accounts
per user), then enrich with leaderboard + CLI insights. Follow
${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/references/people-spending-dashboard-report.md for the exact HTML structure.
- For a single user deep-dive, combine
leaderboard-user <email> with
cli-insights-user <name> for the full picture (champion score + CLI activity).
- If the CLI prints an auth error, forward its message verbatim — it already tells the user
what to do next.
- Always save HTML reports to
reports/<date>-<name>/<name>.html; temp/data files go in reports/<date>-<name>/temp/.
Type-Aware Rendering for metrics[] Arrays
Analytics metrics[] arrays are heterogeneous — each item carries a type field
("number", "string", "date", …) and a format field. The value is numeric for most
items but may be an ISO date string or plain string for others.
Always inspect m.type (and m.format) per item before formatting. Never apply a single
numeric or percent formatter to the whole array — doing so silently produces NaN for
string/date items.
1---2name: codemie-analytics3description: >4 CodeMie Analytics expert — use this skill whenever the user asks about CodeMie usage data,5 AI adoption metrics, user leaderboards, CLI insights, spending, LiteLLM costs, token usage,6 or wants to build a dashboard/report from CodeMie or LiteLLM APIs.7 Also triggers for: "who uses CodeMie most", "show me AI analytics", "get spending data",8 "generate a report", "leaderboard", "cost analysis", "LiteLLM customer info",9 "enrich CSV with costs", "top performers", "AI champions", "tier distribution",10 or any custom analytics query against the platform.11 Always use this skill when CodeMie analytics, reporting, or cost data is involved.12metadata:13 source: https://skillmd.com/skills/codemie-ai/codemie-analytics14---1516# CodeMie Analytics Skill1718You are an analytics expert for the CodeMie (EPAM AI/Run) platform. You know every analytics19API endpoint, how to call LiteLLM directly, and how to orchestrate data into a final report.2021The plumbing (config lookup, SSO credential decryption, token refresh messaging) lives in22`scripts/analytics-cli.js`. You never need to touch those details — just invoke the CLI and23react to what it prints.2425---2627## Step 1 — Understand what the user wants2829Identify the analytics scenario. The CLI supports these command families:3031### Leaderboard (AI Champions)3233The leaderboard ranks users across **6 scoring dimensions**:34- D1: Core Platform Usage (20%) — conversations, assistant interactions35- D2: Core Platform Creation (20%) — assistants, datasources created36- D3: Workflow Usage (10%) — workflow executions37- D4: Workflow Creation (10%) — workflows authored38- D5: CLI & Agentic Engineering (30%) — coding agent sessions, tokens, repos39- D6: Impact & Knowledge (10%) — marketplace publishing, knowledge sharing4041**Tiers**: pioneer (80+), expert (65+), advanced (45+), practitioner (25+), newcomer (<25)4243| Scenario | Command | What it retrieves |44|----------|---------|-------------------|45| Full leaderboard (paginated, filterable) | `leaderboard` | `data.rows[]` — rank, user_name, total_score, tier_name, score_delta, `dimensions[]` (id/score/weight per D1–D6), `summary_metrics{}` (cli_sessions, active_days, total_lines_added, total_spend, web_conversations, …) |46| Leaderboard KPI summary | `leaderboard-summary` | `data` — total_users, avg_score, top_score, `tier_counts{}` (pioneer/expert/advanced/practitioner/newcomer counts and percentages) |47| Single user champion profile | `leaderboard-user <id\|email>` | `data` — same shape as a leaderboard row but for one user; includes full dimension breakdown and all summary_metrics |48| Tier distribution | `leaderboard-tiers` | `data.rows[]` — tier_name, user_count, percentage; one row per tier |49| Average dimension scores | `leaderboard-dimensions` | `data.rows[]` — dimension id/label, avg_score, weight; one row per D1–D6 |50| Top N performers | `leaderboard-top [limit]` | `data.rows[]` — same shape as `leaderboard` rows, limited to top N (max 50, default 10) |51| Score histogram | `leaderboard-scores` | `data.rows[]` — score_range (e.g. "0-10"), user_count; one row per 10-point bin |52| Framework metadata | `leaderboard-framework` | `data.framework{}` — title, principles, calculation_steps; `data.tiers[]` — name, label, min_score; `data.dimensions[]` — id, label, weight, description |53| Computation snapshots | `leaderboard-snapshots` | `data.rows[]` — snapshot_id, created_at, status, period_start, period_end, user_count |54| Available seasons | `leaderboard-seasons --view monthly\|quarterly` | `data.rows[]` — season_key (e.g. "2026-03"), label, start_date, end_date |5556Leaderboard filters: `--view` (current/monthly/quarterly), `--season-key` (2026-03, 2026-Q1),57`--tier`, `--intent` (cli_focused/platform_focused/hybrid/sdlc_unicorn), `--search`, `--sort-by`, `--sort-order`.5859### CLI Insights6061| Scenario | Command | What it retrieves |62|----------|---------|-------------------|63| Full CLI overview (agents, repos, tools, errors) | `cli-insights` | Composite object with sub-keys: `summary` (total sessions, cost, tokens, repos, users), `agents[]` (agent name + session count), `top_users[]`, `top_repos[]`, `errors[]`, `llms[]` |64| User classification & top spenders | `cli-insights-users` | `data.rows[]` — user_name, classification (cli_focused/platform_focused/hybrid/sdlc_unicorn), total_cost, session_count, token_count |65| Detailed single-user CLI profile | `cli-insights-user <name>` | Composite: key_metrics (sessions, cost, tokens, repos, tools), tools[], models[], repositories[], workflow_intent, category_breakdown[] |66| Project classification & top by cost | `cli-insights-projects` | `data.rows[]` — project_name, classification, total_cost, session_count, user_count |67| Usage patterns (weekday, hourly, session depth) | `cli-insights-patterns` | Composite with sub-keys: `weekday.data.rows[]` (weekday_name, session_count), `hourly.data.rows[]` (hour_utc, session_count), `session_depth.data.rows[]` (depth_bucket, count) |6869### General Analytics7071| Scenario | Command | What it retrieves |72|----------|---------|-------------------|73| Overall usage summary (tokens, cost, users) | `summaries` | `data` — total_cost, total_tokens, total_requests, unique_users (MAU), unique_users_daily (DAU), cli_invocations, assistants_count, workflows_count, skills_count, mcp_servers_count |74| User list + activity trends | `users` | `data.rows[]` — user_name, email, total_cost, total_tokens, last_active; plus `activity[]` time-series |75| Per-project spending | `projects-spending` | `data.rows[]` — project_name, total_cost, total_tokens, user_count, request_count |76| Per-project activity time-series | `projects-activity` | Composite with `activity.data.rows[]` and `uniqueDaily.data.rows[]` |77| LLM model breakdown | `llms-usage` | `data.rows[]` — model_name, request_count, total_tokens, input_tokens, output_tokens, total_cost |78| Tool usage | `tools-usage` | `data.rows[]` — tool_name, invocation_count, success_count, error_count, total_tokens |79| Workflow execution analytics | `workflows` | `data.rows[]` — workflow_name, run_count, success_count, failure_count, avg_duration_ms, total_cost |80| Agent execution analytics | `agents-usage` | `data.rows[]` — assistant_name, execution_count, total_cost, total_tokens |81| Embedding model usage | `embeddings-usage` | `data.rows[]` — model_name, request_count, total_tokens, total_cost |82| Chat assistant conversations | `assistants-chats` | `data.rows[]` — assistant, conversation_count, user_count, total_cost |83| Webhook invocation analytics | `webhooks-usage` | `data.rows[]` — user_id, invocation_count, total_cost |84| MCP server usage | `mcp-servers` | `data.rows[]` — mcp_name, request_count, user_count, total_cost |85| MCP server usage by user | `mcp-servers-by-users` | `data.rows[]` — user_name, mcp_name, request_count |86| Power user analytics | `power-users` | `data.rows[]` — user_email, session_count, total_cost, features_used |87| Knowledge sharing metrics | `knowledge-sharing` | `data.rows[]` — user_email, shared_count, viewed_count |88| Top agents by usage | `top-agents` | `data.rows[]` — assistant_name, execution_count, total_cost |89| Top workflows by usage | `top-workflows` | `data.rows[]` — workflow_name, run_count, total_cost |90| Assets published to marketplace | `marketplace` | `data.rows[]` — user_email, asset_name, published_at |91| Budget alerts (soft + hard limits) | `budget` | Composite with `soft.data.rows[]` and `hard.data.rows[]` — user_email, max_spent (users approaching or over limit) |92| Personal spending & budget | `spending` | `data` — current_spend, budget_limit, hard_budget_limit, budget_reset_at, percentage_used |93| Per-user spending (platform + cli split) | `spending-by-users` | Composite with `platform.data.rows[]` and `cli.data.rows[]` — user_name/email, total_cost, token_count |94| Weekly engagement histogram | `engagement` | `data.rows[]` — day_label, hour_start, feature_type, session_count, cost; covers last 7 days in 3-hour intervals |9596### LiteLLM & CSV Enrichment9798| Scenario | Command | Output |99|----------|---------|--------|100| LiteLLM customer lookup | `litellm-customer [user_id]` | JSON |101| LiteLLM spend logs | `litellm-spend` | Spend entries |102| LiteLLM virtual keys | `litellm-keys` | Key info |103| Enrich CSV/Excel with LiteLLM costs | `enrich-csv <file>` | Enriched table |104| Unlisted / experimental endpoint | `custom /v1/analytics/<path>` | Raw JSON — use a named command if one exists |105106---107108## Security109110**Never include raw API keys, bearer tokens, cookie values, LiteLLM keys, or session111credentials in your responses to the user.** If CLI output contains sensitive fields112(e.g. from `litellm-keys`), the CLI automatically redacts them — but if you encounter113any token, key, or secret in raw output, redact it before displaying. Never run `env`,114`printenv`, or similar commands that could expose `LITELLM_KEY` or `CODEMIE_API_KEY`115to the conversation context.116117---118119## Step 2 — Run the analytics CLI120121The CLI script lives at `scripts/analytics-cli.js` next to this skill. It handles122authentication internally. If something is wrong with credentials, it prints a clear123actionable message to stderr; pass that along to the user verbatim.124125```bash126node ${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/scripts/analytics-cli.js <command> [options]127```128129LiteLLM commands (`litellm-*`, `enrich-csv`) require `LITELLM_URL` + `LITELLM_KEY` env vars.130131### Collecting data for reports (batch pattern)132133**Only fetch the data the report actually needs.** Read the user's request, identify which134sections the report requires, and collect only the relevant endpoints. Do not run every135available command — unnecessary fetches waste time and tokens.136137When building an HTML report, run all needed CLI commands in a single Bash call using138`--save <filepath>` on each. This triggers one permission prompt for the entire139collection phase and keeps API responses out of the conversation context.140141### Directory layout142143Every report lives in its own folder under `reports/`. Derive the folder name from the144current date and a short kebab-case description of the report:145146```147reports/148 2026-05-07-cli-usage/ ← report folder (date + name)149 cli-usage.html ← the HTML report (saved here directly)150 temp/ ← all temp/data files go here151 summaries.json152 summaries.schema.json153 cli-insights.json154 ...155```156157**Never overwrite an existing report folder.** Always resolve a free name before creating158anything. Use a suffix loop — this is mandatory, not optional:159160```bash161BASE=reports/$(date +%Y-%m-%d)-<short-name>162REPORT_DIR=$BASE163n=2164while [ -d "$REPORT_DIR" ]; do REPORT_DIR="${BASE}-${n}"; n=$((n+1)); done165OUT="$REPORT_DIR/temp"166```167168Then run only the commands the report needs:169170```bash171CLI=${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/scripts/analytics-cli.js172mkdir -p "$OUT" && \173node $CLI summaries --save "$OUT/summaries.json" && \174node $CLI cli-insights --save "$OUT/cli-insights.json" && \175# ... only endpoints needed for this report ...176echo "✓ All data saved → $OUT"177```178179Each command prints: `✓ Saved → <path>`. The final `echo` confirms the directory180path — save it, you will reference it in every subsequent step.181182**Do not `cat`, `Read`, or print the saved JSON files into the conversation.**183Raw API responses can be hundreds of KB. Use Step 2.5 to inspect structure instead.184185Temp files are not cleaned up automatically; Step 3b asks the user whether to delete them once186the report is built.187188### Common filter flags189190| Flag | Example | Notes |191|------|---------|-------|192| `--time-period` | `last_30_days` | Predefined period |193| `--start-date` | `2024-01-01T00:00:00` | Custom range start |194| `--end-date` | `2024-03-31T23:59:59` | Custom range end |195| `--users` | `alice,bob` | Comma-separated usernames |196| `--projects` | `my-project` | Comma-separated project names |197| `--page` | `1` | Pagination |198| `--per-page` | `100` | Results per page (default 50) |199| `--output` | `json` | `json` \| `table` \| `csv` |200| `--pretty` | (flag) | Pretty-print JSON |201202### Leaderboard-specific flags203204| Flag | Example | Notes |205|------|---------|-------|206| `--view` | `monthly` | `current` \| `monthly` \| `quarterly` |207| `--season-key` | `2026-Q1` | Specific season to query |208| `--tier` | `pioneer` | Filter by tier |209| `--intent` | `cli_focused` | Filter by user intent profile |210| `--search` | `john` | Partial name/email search |211| `--sort-by` | `total_score` | `rank` \| `total_score` \| `user_name` \| `tier_level` |212| `--sort-order` | `desc` | `asc` \| `desc` |213| `--limit` | `20` | Max entries for `leaderboard-top` (max 50) |214215### Example invocations216217```bash218CLI=${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/scripts/analytics-cli.js219220# Full leaderboard — top 50 pioneers sorted by score221node $CLI leaderboard --tier pioneer --sort-by total_score --sort-order desc --per-page 50 --pretty222223# Single user champion profile224node $CLI leaderboard-user user@example.com --pretty225226# Leaderboard KPI summary for Q1 2026227node $CLI leaderboard-summary --view quarterly --season-key 2026-Q1 --pretty228229# Dimension averages (D1–D6) for current snapshot230node $CLI leaderboard-dimensions --pretty231232# Tier distribution233node $CLI leaderboard-tiers --pretty234235# Top 10 performers236node $CLI leaderboard-top 10 --pretty237238# 30-day platform summary239node $CLI summaries --time-period last_30_days --pretty240241# Full CLI insights242node $CLI cli-insights --time-period last_30_days --pretty243244# Detailed CLI profile for a specific user245node $CLI cli-insights-user alice@example.com --time-period last_30_days --pretty246247# Usage patterns (weekday + hourly + session depth)248node $CLI cli-insights-patterns --time-period last_30_days --pretty249250# Per-user spending breakdown251node $CLI spending-by-users --time-period last_30_days --pretty252253# Custom endpoint254node $CLI custom /v1/analytics/mcp-servers --time-period last_30_days --pretty255```256257---258259## Step 2.5 — Inspect data structure260261After collection, run `inspect-schema.js` to generate a compact `.schema.json` file262alongside each saved JSON — one permission prompt, no data in context:263264```bash265node ${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/scripts/inspect-schema.js "$OUT"266```267268Output is a short listing of generated schema files with sizes, for example:269```270Schemas written to: reports/2026-05-07-cli-usage/temp/271 ✓ leaderboard-top.schema.json (1.3 KB)272 ✓ summaries.schema.json (0.4 KB)273 ...274```275276Then **read only the `.schema.json` files you actually need** for the report using the277`Read` tool. Each schema shows field names, types, array lengths, and string samples —278enough to write extraction code without touching the raw responses:279280> **CRITICAL — schema is the source of truth for field names.**281> Never use field names or metric IDs from this skill's documentation when writing282> report code. Documentation can be stale. The `.schema.json` files are generated from283> live API responses and are always correct. Every `id`, key, or column name referenced284> in JS must be verified against the schema you just read — not assumed from the tables285> above. If the schema contradicts the docs, trust the schema.286287```json288{289 "_envelope": "[envelope] data_as_of: '2026-05-07T02:00:10', total_count: 50",290 "data": {291 "rows": {292 "_type": "array",293 "_count": 50,294 "_item": {295 "rank": "number",296 "user_name": "string ~ 'Jane Smith'",297 "total_score": "number",298 "tier_name": "string ~ 'expert'",299 "score_delta": "number | null",300 "dimensions": { "_type": "array", "_count": 6, "_item": { "id": "string ~ 'd1'", "score": "number", "weight": "number" } },301 "summary_metrics": { "cli_sessions": "number", "active_days": "number", "total_lines_added": "number", "total_spend": "number" }302 }303 }304 }305}306```307308---309310## Step 3 — Build the HTML report311312**Save the HTML report directly inside `$REPORT_DIR`** (not in `temp/`).313Use a kebab-case filename matching the folder name:314315```316reports/2026-05-07-cli-usage/cli-usage.html317reports/2026-05-07-leaderboard/leaderboard.html318reports/2026-05-07-spending/spending.html319```320321### Step 3a — Hand off to codemie-html-report322323Invoke the **`codemie-html-report`** skill. Pass:3243251. **The schemas** inspected in Step 2.5 — field names and structure.3262. **The user's intent** — e.g. "leaderboard dashboard with tier distribution".3273. **Timestamp context** — `_envelope` lines in schemas include `data_as_of`.3284. **The data folder** — `$OUT`, holding the JSON files saved in Step 2.3295. **Output path** — e.g. `$REPORT_DIR/leaderboard.html`.330331Ask it to build the page per codemie-html-report's `references/analytics.md`, which owns the332data placeholders, the style placeholder and the order of the build scripts.333334The layout references below give structure and data only. CSS, fonts and Chart.js come from the335codemie-html-report build, which overrides any instruction in them to inline CSS or load a CDN.336337### Step 3b — Temp file cleanup338339After codemie-html-report has built the report, **always ask the user**:340341> The report is ready at `<path>`. The `temp/` directory (`$OUT`) contains the raw342> API response files used to build it (~N files). Would you like to delete it?343344These files can hold per-user spending and leaderboard data. If the user says **yes**,345delete the temp directory:346347```bash348rm -rf "$OUT"349# e.g. rm -rf "reports/2026-05-07-executive-spending/temp"350```351352Confirm deletion:353```354✓ Temp files deleted → reports/2026-05-07-executive-spending/temp355```356357If the user says **no**, leave the files in place and tell them the path.358359---360361## Full API Reference362363### Leaderboard endpoints (`GET /v1/analytics/leaderboard/...`)364365Admin-only. All accept `snapshot_id`, `view`, `season_key` query params.366367| Endpoint | Additional Params | Returns |368|----------|------------------|---------|369| `/leaderboard/summary` | — | Total users, tier counts, top score |370| `/leaderboard/entries` | `tier`, `search`, `intent`, `sort_by`, `sort_order`, `page`, `per_page` | Paginated ranked entries |371| `/leaderboard/user/{user_id}` | path: user ID or email | Full user profile with D1–D6 breakdown |372| `/leaderboard/tiers` | — | Tier name, count, percentage |373| `/leaderboard/scores` | — | Score histogram (10-point bins) |374| `/leaderboard/dimensions` | — | Average D1–D6 scores |375| `/leaderboard/top-performers` | `limit` (default 3, max 50) | Top N by total score |376| `/leaderboard/snapshots` | `view`, `status`, `is_final`, `page`, `per_page` | Computation snapshots |377| `/leaderboard/seasons` | `view` (required: monthly/quarterly), `page`, `per_page` | Available seasonal periods |378| `/leaderboard/framework` | — | Static metadata: dimensions, tiers, intents, scoring |379| `/leaderboard/compute` (POST) | `period_days`, `view`, `season_key` | Triggers manual computation |380381### CLI Insights endpoints (`GET /v1/analytics/cli-insights-...`)382383| Endpoint | Params | Returns |384|----------|--------|---------|385| `/cli-insights-weekday-pattern` | time filters | Weekday usage patterns |386| `/cli-insights-hourly-usage` | time filters | Hourly usage patterns |387| `/cli-insights-session-depth` | time filters | Session depth distribution |388| `/cli-insights-user-classification` | time filters | User classification breakdown |389| `/cli-insights-top-users-by-cost` | time filters | Top users ranked by cost |390| `/cli-insights-top-spenders` | time filters | Top spenders |391| `/cli-insights-users` | time filters | CLI user list |392| `/cli-insights-user-detail` | `user_name` (required), `user_id` | Full user detail |393| `/cli-insights-user-key-metrics` | `user_name` (required), `user_id` | User KPIs |394| `/cli-insights-user-tools` | `user_name` (required), `user_id` | User tool usage |395| `/cli-insights-user-models` | `user_name` (required), `user_id` | User model usage |396| `/cli-insights-user-workflow-intent` | `user_name` (required), `user_id` | User workflow intent |397| `/cli-insights-user-classification-detail` | `user_name` (required), `user_id` | User classification detail |398| `/cli-insights-user-category-breakdown` | `user_name` (required), `user_id` | User category breakdown |399| `/cli-insights-user-repositories` | `user_name` (required), `user_id` | User repositories |400| `/cli-insights-project-classification` | time filters | Project classification |401| `/cli-insights-top-projects-by-cost` | time filters | Top projects by cost |402403### Standard CLI analytics (`GET /v1/analytics/cli-...`)404405| Endpoint | Returns |406|----------|---------|407| `/cli-summary` | CLI totals (tokens, cost, sessions) |408| `/cli-agents` | Agent breakdown |409| `/cli-llms` | Model breakdown |410| `/cli-users` | CLI user activity |411| `/cli-errors` | Error logs |412| `/cli-repositories` | Repo activity |413| `/cli-top-performers` | Top by lines added |414| `/cli-top-versions` | CLI version distribution |415| `/cli-top-proxy-endpoints` | LiteLLM endpoint usage |416| `/cli-tools` | Tool usage |417418### Dashboard analytics (`GET /v1/analytics/...`)419420All accept time filters + `users` + `projects` + `page` + `per_page`.421422| Endpoint | Returns |423|----------|---------|424| `/summaries` | Platform totals (tokens, cost, unique users) |425| `/users-spending` | Per-user cost + tokens |426| `/users-activity` | Activity time-series |427| `/users-unique-daily` | Unique users/day |428| `/users` | User list |429| `/projects-spending` | Per-project spending |430| `/projects-activity` | Project activity time-series |431| `/projects-unique-daily` | Unique projects/day |432| `/llms-usage` | LLM model usage |433| `/tools-usage` | Tool usage |434| `/workflows` | Workflow runs |435| `/agents-usage` | Agent executions |436| `/embeddings-usage` | Embedding model usage |437| `/assistants-chats` | Chat assistant conversations |438| `/webhooks-invocation` | Webhook usage |439| `/mcp-servers` | MCP server usage |440| `/mcp-servers-by-users` | MCP by user |441| `/power-users` | Power user analytics |442| `/knowledge-sharing` | Knowledge sharing metrics |443| `/top-agents-usage` | Top agents |444| `/top-workflow-usage` | Top workflows |445| `/published-to-marketplace` | Marketplace publishing |446447### Spending & Budget endpoints448449| Endpoint | Method | Returns |450|----------|--------|---------|451| `/spending` | GET | Current user: spend, budget_limit, hard_budget_limit, reset time |452| `/budget_usage` | GET | Per-key budget rows with % used |453| `/budget-soft-limit` | GET | Soft limit warnings |454| `/budget-hard-limit` | GET | Hard limit hits |455| `/spending/by-users/platform` | GET | Per-user platform spending |456| `/spending/by-users/cli` | GET | Per-user CLI spending |457458### Engagement459460| Endpoint | Method | Returns |461|----------|--------|---------|462| `/engagement/weekly-histogram` | GET | 3h intervals, last 7 days, by feature type |463464### LiteLLM endpoints (via `LITELLM_URL` + `LITELLM_KEY`)465466| Endpoint | Method | Params | Description |467|----------|--------|--------|-------------|468| `/customer/info` | GET | `user_id` | Customer spend + budget + allowed models |469| `/spend/logs` | GET | `start_date`, `end_date`, `user_id` | Spend log entries |470| `/key/info` | GET | `key` | Virtual key details + spend |471| `/model/info` | GET | — | Available models |472| `/health` | GET | — | Proxy health |473474### Response envelope475476Most CodeMie endpoints return:477```json478{479 "data": { ... },480 "metadata": {481 "timestamp": "2024-03-15T12:00:00Z",482 "data_as_of": "2024-03-15T12:00:00Z",483 "filters_applied": {},484 "execution_time_ms": 45.2485 }486}487```488489Always extract `response.data` for the actual payload.490491---492493## Custom analytics requests494495For endpoints not covered by preset commands:496497```bash498node analytics-cli.js custom /v1/analytics/mcp-servers --time-period last_30_days499500# POST endpoints501node analytics-cli.js custom /v1/analytics/ai-adoption-overview --method POST \502 --time-period last_30_days503```504505---506507## Offline CLI analytics (no API key needed)508509The `codemie analytics` CLI command reads **local session files** from `~/.codemie/sessions/`510with no API calls:511512```bash513codemie analytics --last 7d --output json514codemie analytics --agent claude --last 30d --export csv515```516517---518519## Report References520521Reference files in `references/` describe canonical report layouts. **Always check the522relevant reference before building a new HTML report** — it defines the exact components,523charts, data structure, and modal design to use, ensuring consistency across users.524525| Report type | Reference file | When to use |526|-------------|---------------|-------------|527| Leaderboard dashboard | [`${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/references/leaderboard-dashboard-report.md`](${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/references/leaderboard-dashboard-report.md) | Any request for leaderboard rankings, AI champions, top performers, tier distribution |528| People spending dashboard | [`${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/references/people-spending-dashboard-report.md`](${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/references/people-spending-dashboard-report.md) | Any request to track LiteLLM costs for a specific list of users (cohort, team, bootcamp, project) |529530---531532## Use Cases533534### Use Case: People Spending Dashboard (cohort / team / bootcamp)535536**Trigger phrases**: "build a spending dashboard for people from X", "track LiteLLM costs537for a list of users", "how much did this team spend", "bootcamp spending report",538"costs for people in this CSV/Excel".539540**Also applies when**: the user asks to enrich analytics with EPAM employee data, map541platform users to EPAM people, or look up user assignments/org details.542543**⚠️ EPAM People & Assignments Finder — required for this use case only**544545Before proceeding, verify the assistant is accessible:546547```bash548node ${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/scripts/analytics-cli.js \549 custom /v1/assistants/5ca384d0-d042-480c-a0a9-d28150e2352f 2>&1 | head -5550```551552If the command returns an auth error, HTTP 401/403/404, or "No CodeMie credentials" —553**stop. Do not run anything else.** Notify the user:554555> ⛔ **EPAM People & Assignments Finder** assistant is not configured on your account.556>557> **Assistant:** EPAM People & Assignments Finder558> **ID:** `5ca384d0-d042-480c-a0a9-d28150e2352f`559>560> Add it with:561> ```bash562> codemie assistants add 5ca384d0-d042-480c-a0a9-d28150e2352f563> ```564> Or open in browser and click **Add to my assistants**:565> https://codemie.lab.epam.com/#/assistants/5ca384d0-d042-480c-a0a9-d28150e2352f566>567> Once added, resume this session with:568> ```bash569> claude -f570> ```571572**Full workflow** (see `${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/references/people-spending-dashboard-report.md` for all details):5735741. **Parse the list** from Excel/CSV using `openpyxl`. Skip header and TOTAL rows.5752. **Fetch 3 LiteLLM accounts per user** using Python `asyncio` + `aiohttp` (semaphore 25,576 `ssl=False`). Account patterns:577 - Web: `email` (plain)578 - CLI: `email_codemie_cli`579 - Premium: `email_codemie_premium_models`580 Use `end_user_id` param (not `user_id`) on `GET /customer/info`.5813. **Save raw results** to `/tmp/` to avoid re-fetching on HTML rebuild.5824. **Build users array** — sum three spend values, extract budget fields per account.5835. **Fetch leaderboard** — paginate ALL pages (`--per-page 500`), ~25 pages for 12k users.584 Expect ~60% of a typical cohort to appear.5856. **Fetch CLI insights** — `cli-insights-users --per-page 500 topBySpend` for top CLI users.586 Expect ~3–5% coverage for a general cohort.5877. **Compute KPIs** — grand total, per-type totals, active user count, avg spend.588 Budget projection: `avg_spend_per_active × total_users × 1.20`.5898. **Generate HTML** — use `str.replace()` with `__TOKEN__` markers (never f-strings, which590 conflict with JS `${...}` template literals).5919. **Wire table clicks** — use `data-email` attribute + event delegation (never `onclick=""`592 attributes, which break under Python quote escaping).59310. **Save** to `reports/<date>-<name>/<name>.html` (temp/data files in `reports/<date>-<name>/temp/`).594595**Key commands:**596```bash597CLI=${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/scripts/analytics-cli.js598599# Leaderboard (run in a loop for all pages)600node $CLI leaderboard --per-page 500 --page <N> --output json601602# CLI top spenders603node $CLI cli-insights-users --time-period last_30_days --per-page 500 --output json604```605606**LiteLLM fetch** requires Python (not the analytics CLI) because it needs607`LITELLM_URL` + `LITELLM_KEY` env vars and concurrent calls for 1,000+ accounts.608609---610611## Tips612613- **Always run the CLI first**, capture JSON, then hand it to the report skill — don't614 hardcode example data.615- If a command returns paginated data, loop through all pages or set `--per-page 500`.616- For time-series charts, use `/users-unique-daily` or `/projects-unique-daily` endpoints.617- Budget warnings: flag rows where `spend / max_budget > 0.8` (warn) and `> 1.0` (error).618- For the **leaderboard dashboard**, combine `leaderboard` + `leaderboard-summary` +619 `leaderboard-tiers` + `leaderboard-dimensions` to build a comprehensive view. Then follow620 `${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/references/leaderboard-dashboard-report.md` for the exact HTML structure.621- For a **people spending dashboard**, fetch LiteLLM directly with Python async (3 accounts622 per user), then enrich with leaderboard + CLI insights. Follow623 `${CLAUDE_PLUGIN_ROOT}/skills/codemie-analytics/references/people-spending-dashboard-report.md` for the exact HTML structure.624- For a **single user deep-dive**, combine `leaderboard-user <email>` with625 `cli-insights-user <name>` for the full picture (champion score + CLI activity).626- If the CLI prints an auth error, forward its message verbatim — it already tells the user627 what to do next.628- Always save HTML reports to `reports/<date>-<name>/<name>.html`; temp/data files go in `reports/<date>-<name>/temp/`.629630## Type-Aware Rendering for `metrics[]` Arrays631632Analytics `metrics[]` arrays are **heterogeneous** — each item carries a `type` field633(`"number"`, `"string"`, `"date"`, …) and a `format` field. The `value` is numeric for most634items but may be an ISO date string or plain string for others.635636**Always inspect `m.type` (and `m.format`) per item before formatting.** Never apply a single637numeric or percent formatter to the whole array — doing so silently produces `NaN` for638string/date items.