LLM Tracker Maintainer
Quick Reference
- Repo:
Xplore0114/Xplore0114.github.io
- Data path:
llm-tracker/ (papers.json, company-papers.json, timeline-data.json)
- Script:
scripts/fetch_papers.py (root level)
- Workflow:
.github/workflows/update-papers.yml
- Auth:
GITHUB_PAT env var (already configured in ~/.bashrc)
- Schedule: Daily at 01:00 UTC (09:00 Beijing)
Common Issues & Fixes
Workflow won't trigger via API (HTTP 422)
Usually caused by corrupted YAML or duplicate workflow files.
- Delete duplicate workflow files (e.g. both
daily-papers.yml and update-papers.yml)
- Ensure
workflow_dispatch: is present under on:
- If still failing, delete and recreate the workflow file
- Python code inside
run: | blocks must use heredoc (<< 'PYEOF') for inline scripts
- Never put raw
\n in YAML string values — use heredoc instead
arXiv API returns nothing (HTTP 301 redirect)
arxiv.org now requires HTTPS. Ensure the script uses:
url = "https://export.arxiv.org/api/query?" + params # NOT http://
Timeline not updating
The timeline page reads timeline-data.json. This file must be generated by the fetch script.
See scripts/sync_timeline.py.
Manual Update Workflow
Via API (recommended)
curl -s -X POST \
-H "Authorization: Bearer ${GITHUB_PAT}" \
-H "Accept: application/vnd.github+json" \
"https://api.github.com/repos/Xplore0114/Xplore0114.github.io/actions/workflows/update-papers.yml/dispatches" \
-d '{"ref":"master"}'
Via page button
The llm-tracker page has a "⚡ 更新数据" button that calls the same API.
Full regeneration (local)
cd /tmp
# Download current papers.json from repo
# Run scripts/fetch_papers.py locally
# Upload results via GitHub API
Data Files
| File |
Description |
Generated by |
papers.json |
All papers (topic queries) |
fetch_papers.py |
company-papers.json |
Company-specific papers |
fetch_papers.py |
timeline-data.json |
Individual papers list for heatmap calendar |
fetch_papers.py (generate_timeline) |
⚠️ timeline-data.json format: Must be an array of individual paper objects with date field (NOT aggregated monthly counts). The timeline page uses this for a GitHub-style heatmap calendar.
Updating the Workflow YAML
Always use heredoc for Python code in run: blocks:
- name: Run script
run: |
python3 << 'PYEOF'
import json
print("hello")
PYEOF
Never inline Python with raw newlines in YAML string values.
Company Keywords Reference
| Company |
arXiv Keywords (author/affiliation) |
Models |
| OpenAI |
openai |
GPT-4o, o1, o3, Sora, Codex |
| Google |
google, deepmind |
Gemini, Gemma, PaLM |
| Anthropic |
anthropic |
Claude |
| Meta |
meta ai, meta platforms, fair, facebook ai |
LLaMA, fairseq |
| DeepSeek |
deepseek |
DeepSeek-V, DeepSeek-R1 |
| Qwen |
alibaba, qwen, tongyi, damo |
Qwen, Tongyi Qianwen |
| Mistral |
mistral |
Mistral, Mixtral, Pixtral |
| Baidu |
baidu |
ERNIE, Wenxin |
| Xiaomi |
xiaomi |
MiMo |
| MiniMax |
minimax |
MiniMax-01, MiniMax-V |
| Zhipu |
zhipu, tsinghua, chatglm |
GLM-4, ChatGLM, CogVLM |
1---2name: llm-tracker-maintainer3description: Maintain the LLM Papers Tracker (xplore0114.github.io/llm-tracker). Handles paper fetching from arXiv, data sync, GitHub Actions workflow fixes, and page updates. Use when user mentions llm-tracker, LLM论文追踪, paper updates, arxiv fetch issues, or the tracker website.4---56# LLM Tracker Maintainer78## Quick Reference910- **Repo**: `Xplore0114/Xplore0114.github.io`11- **Data path**: `llm-tracker/` (papers.json, company-papers.json, timeline-data.json)12- **Script**: `scripts/fetch_papers.py` (root level)13- **Workflow**: `.github/workflows/update-papers.yml`14- **Auth**: `GITHUB_PAT` env var (already configured in `~/.bashrc`)15- **Schedule**: Daily at 01:00 UTC (09:00 Beijing)1617## Common Issues & Fixes1819### Workflow won't trigger via API (HTTP 422)2021Usually caused by corrupted YAML or duplicate workflow files.22231. Delete duplicate workflow files (e.g. both `daily-papers.yml` and `update-papers.yml`)242. Ensure `workflow_dispatch:` is present under `on:`253. If still failing, delete and recreate the workflow file264. Python code inside `run: |` blocks must use heredoc (`<< 'PYEOF'`) for inline scripts275. Never put raw `\n` in YAML string values — use heredoc instead2829### arXiv API returns nothing (HTTP 301 redirect)3031arxiv.org now requires HTTPS. Ensure the script uses:32```python33url = "https://export.arxiv.org/api/query?" + params # NOT http://34```3536### Timeline not updating3738The timeline page reads `timeline-data.json`. This file must be generated by the fetch script.39See `scripts/sync_timeline.py`.4041## Manual Update Workflow4243### Via API (recommended)44```bash45curl -s -X POST \46 -H "Authorization: Bearer ${GITHUB_PAT}" \47 -H "Accept: application/vnd.github+json" \48 "https://api.github.com/repos/Xplore0114/Xplore0114.github.io/actions/workflows/update-papers.yml/dispatches" \49 -d '{"ref":"master"}'50```5152### Via page button53The llm-tracker page has a "⚡ 更新数据" button that calls the same API.5455### Full regeneration (local)56```bash57cd /tmp58# Download current papers.json from repo59# Run scripts/fetch_papers.py locally60# Upload results via GitHub API61```6263## Data Files6465| File | Description | Generated by |66|------|-------------|-------------|67| `papers.json` | All papers (topic queries) | fetch_papers.py |68| `company-papers.json` | Company-specific papers | fetch_papers.py |69| `timeline-data.json` | Individual papers list for heatmap calendar | fetch_papers.py (generate_timeline) |7071**⚠️ timeline-data.json format**: Must be an array of individual paper objects with `date` field (NOT aggregated monthly counts). The timeline page uses this for a GitHub-style heatmap calendar.7273## Updating the Workflow YAML7475Always use heredoc for Python code in `run:` blocks:7677```yaml78- name: Run script79 run: |80 python3 << 'PYEOF'81 import json82 print("hello")83 PYEOF84```8586Never inline Python with raw newlines in YAML string values.8788## Company Keywords Reference8990| Company | arXiv Keywords (author/affiliation) | Models |91|---------|--------------------------------------|--------|92| OpenAI | openai | GPT-4o, o1, o3, Sora, Codex |93| Google | google, deepmind | Gemini, Gemma, PaLM |94| Anthropic | anthropic | Claude |95| Meta | meta ai, meta platforms, fair, facebook ai | LLaMA, fairseq |96| DeepSeek | deepseek | DeepSeek-V, DeepSeek-R1 |97| Qwen | alibaba, qwen, tongyi, damo | Qwen, Tongyi Qianwen |98| Mistral | mistral | Mistral, Mixtral, Pixtral |99| Baidu | baidu | ERNIE, Wenxin |100| Xiaomi | xiaomi | MiMo |101| MiniMax | minimax | MiniMax-01, MiniMax-V |102| Zhipu | zhipu, tsinghua, chatglm | GLM-4, ChatGLM, CogVLM |
Xplore-LAB