Store
Help the user choose storage formats, locations, and publishing methods for their football data.
Accuracy
Read and follow docs/accuracy-guardrail.md before answering any question about provider-specific facts (IDs, endpoints, schemas, coordinates, rate limits). Always use search_docs — never guess from training data.
First: check profile
Read .nutmeg.user.md. If it doesn't exist, tell the user to run /nutmeg first.
Storage format decision tree
Small projects (< 100MB, single user)
| Format |
Best for |
Tools |
| JSON |
Raw event data, API responses |
Any language |
| CSV |
Tabular stats, easy to share |
Spreadsheets, pandas, R |
| Parquet |
Columnar analytics, fast queries |
polars, pandas, DuckDB, Arrow |
| SQLite |
Relational queries, multiple tables |
Any language, DB browser tools |
Recommendation: Start with JSON for raw data, Parquet for processed data.
Medium projects (100MB - 10GB)
| Format |
Best for |
Notes |
| Parquet files |
Analytics workloads |
5-10x smaller than JSON, fast columnar reads |
| DuckDB |
SQL analytics on local files |
Queries Parquet/CSV directly, no server needed |
| SQLite |
Relational data with joins |
Single file, portable, ACID compliant |
Recommendation: Parquet for storage, DuckDB for querying.
Large projects (> 10GB, multiple users)
| Solution |
Best for |
Cost |
| PostgreSQL |
Production apps, complex queries |
Free (self-hosted) or ~$7/mo (Railway, Supabase) |
| BigQuery |
Massive analytical queries |
Free tier: 1TB/mo queries |
| Cloudflare R2 |
Object storage (raw files) |
Free tier: 10GB storage |
| S3 / GCS |
Object storage at scale |
~$0.023/GB/mo |
Directory structure
Recommend this structure for football data projects:
project/
data/
raw/ # Untouched API/scrape responses
statsbomb/
events/
matches.json
fbref/
2024/
processed/ # Cleaned, transformed data
events.parquet
shots.parquet
passes.parquet
derived/ # Computed metrics
xg_model.parquet
passing_networks/
notebooks/ # Analysis notebooks
scripts/ # Data pipeline scripts
outputs/ # Charts, reports, exports
.env # API keys (gitignored)
.nutmeg.user.md # Nutmeg profile
Publishing and sharing
Interactive dashboards
| Platform |
Language |
Cost |
Notes |
| Streamlit |
Python |
Free (community cloud) |
Most popular for football analytics. Deploy from GitHub |
| Observable |
JavaScript |
Free tier |
Great for D3.js visualisations. Notebooks + Framework |
| Shiny |
R |
Free (shinyapps.io, 25 hrs/mo) |
R ecosystem integration |
| Gradio |
Python |
Free (HuggingFace Spaces) |
Quick ML model demos |
Static sites
| Platform |
Notes |
| GitHub Pages |
Free. Good for static charts (D3, matplotlib exports) |
| Cloudflare Pages |
Free. Faster, more features than GH Pages |
| Vercel |
Free tier. Good for Next.js/Astro sites |
Sharing data
| Method |
Best for |
| GitHub repo |
Small datasets (< 100MB), code + data together |
| GitHub Releases |
Larger files (up to 2GB per release) |
| Kaggle Datasets |
Community sharing, discoverable, free |
| HuggingFace Datasets |
ML-focused, versioned, free |
Social media / content
| Output |
Tool |
Notes |
| Static charts |
matplotlib, ggplot2, D3.js |
Export as PNG/SVG |
| Animated charts |
matplotlib.animation, D3 transitions |
Export as GIF/MP4 |
| Twitter/X threads |
Chart images + alt text |
Accessibility matters |
| Blog posts |
Markdown + embedded charts |
GitHub Pages, Medium, Substack |
Cost awareness
Based on the user's .nutmeg.user.md goals, flag costs:
- Exploration/learning: Everything can be free. StatsBomb open data + Jupyter/Colab + GitHub Pages.
- Content creation: Streamlit Community Cloud is free. Cloudflare Pages is free.
- Professional: Budget for API access ($100-1000+/mo for Opta/StatsBomb commercial).
- Product: Database hosting ($7-50/mo), consider data licensing costs separately.
1---2name: nutmeg-store3description: Choose how and where to store football data. Use when the user asks about database choices, file formats, cloud storage, data pipelines, or how to organise their football data project. Also covers publishing and sharing outputs (Streamlit, Observable, GitHub Pages).4---56# Store78Help the user choose storage formats, locations, and publishing methods for their football data.910## Accuracy1112Read and follow `docs/accuracy-guardrail.md` before answering any question about provider-specific facts (IDs, endpoints, schemas, coordinates, rate limits). Always use `search_docs` — never guess from training data.13## First: check profile1415Read `.nutmeg.user.md`. If it doesn't exist, tell the user to run `/nutmeg` first.1617## Storage format decision tree1819### Small projects (< 100MB, single user)2021| Format | Best for | Tools |22|--------|---------|-------|23| JSON | Raw event data, API responses | Any language |24| CSV | Tabular stats, easy to share | Spreadsheets, pandas, R |25| Parquet | Columnar analytics, fast queries | polars, pandas, DuckDB, Arrow |26| SQLite | Relational queries, multiple tables | Any language, DB browser tools |2728**Recommendation:** Start with JSON for raw data, Parquet for processed data.2930### Medium projects (100MB - 10GB)3132| Format | Best for | Notes |33|--------|---------|-------|34| Parquet files | Analytics workloads | 5-10x smaller than JSON, fast columnar reads |35| DuckDB | SQL analytics on local files | Queries Parquet/CSV directly, no server needed |36| SQLite | Relational data with joins | Single file, portable, ACID compliant |3738**Recommendation:** Parquet for storage, DuckDB for querying.3940### Large projects (> 10GB, multiple users)4142| Solution | Best for | Cost |43|----------|---------|------|44| PostgreSQL | Production apps, complex queries | Free (self-hosted) or ~$7/mo (Railway, Supabase) |45| BigQuery | Massive analytical queries | Free tier: 1TB/mo queries |46| Cloudflare R2 | Object storage (raw files) | Free tier: 10GB storage |47| S3 / GCS | Object storage at scale | ~$0.023/GB/mo |4849## Directory structure5051Recommend this structure for football data projects:5253```54project/55 data/56 raw/ # Untouched API/scrape responses57 statsbomb/58 events/59 matches.json60 fbref/61 2024/62 processed/ # Cleaned, transformed data63 events.parquet64 shots.parquet65 passes.parquet66 derived/ # Computed metrics67 xg_model.parquet68 passing_networks/69 notebooks/ # Analysis notebooks70 scripts/ # Data pipeline scripts71 outputs/ # Charts, reports, exports72 .env # API keys (gitignored)73 .nutmeg.user.md # Nutmeg profile74```7576## Publishing and sharing7778### Interactive dashboards7980| Platform | Language | Cost | Notes |81|----------|---------|------|-------|82| Streamlit | Python | Free (community cloud) | Most popular for football analytics. Deploy from GitHub |83| Observable | JavaScript | Free tier | Great for D3.js visualisations. Notebooks + Framework |84| Shiny | R | Free (shinyapps.io, 25 hrs/mo) | R ecosystem integration |85| Gradio | Python | Free (HuggingFace Spaces) | Quick ML model demos |8687### Static sites8889| Platform | Notes |90|----------|-------|91| GitHub Pages | Free. Good for static charts (D3, matplotlib exports) |92| Cloudflare Pages | Free. Faster, more features than GH Pages |93| Vercel | Free tier. Good for Next.js/Astro sites |9495### Sharing data9697| Method | Best for |98|--------|---------|99| GitHub repo | Small datasets (< 100MB), code + data together |100| GitHub Releases | Larger files (up to 2GB per release) |101| Kaggle Datasets | Community sharing, discoverable, free |102| HuggingFace Datasets | ML-focused, versioned, free |103104### Social media / content105106| Output | Tool | Notes |107|--------|------|-------|108| Static charts | matplotlib, ggplot2, D3.js | Export as PNG/SVG |109| Animated charts | matplotlib.animation, D3 transitions | Export as GIF/MP4 |110| Twitter/X threads | Chart images + alt text | Accessibility matters |111| Blog posts | Markdown + embedded charts | GitHub Pages, Medium, Substack |112113## Cost awareness114115Based on the user's `.nutmeg.user.md` goals, flag costs:116117- **Exploration/learning:** Everything can be free. StatsBomb open data + Jupyter/Colab + GitHub Pages.118- **Content creation:** Streamlit Community Cloud is free. Cloudflare Pages is free.119- **Professional:** Budget for API access ($100-1000+/mo for Opta/StatsBomb commercial).120- **Product:** Database hosting ($7-50/mo), consider data licensing costs separately.
withqwerty