# AI Training Platform

> Build and iterate on the AI learning flashcard platform (品牌名「知渔」/「KnowHow知渔」) — project-based 6-stage learning paths, data flywheel crowd wisdom injection, progressive depth, TTS voice, content generation, Doubao image workflow. Use when working on /Users/hua/6-产品研发/ok-KnowHow知渔/ (主目录) or legacy /Users/hua/6-产品研发/23-ai培训教程/ / /Users/hua/Desktop/渔芯科技/workspace/AI培训平台工程/.

- Skill: `openclaw-cn-dev/ai-training-platform` (Agent Skill, multi-file: 6 files)
- Install (CLI): `npx get-skill add openclaw-cn-dev/ai-training-platform`
- Raw SKILL.md: https://api.skillmd.com/api/skills/openclaw-cn-dev/ai-training-platform/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- Author: openclaw-cn-dev (https://skillmd.com/u/openclaw-cn-dev)
- Updated: 2026-11-14
- Page: https://skillmd.com/skills/openclaw-cn-dev/ai-training-platform

---


# AI Training Platform

## LLM Backend

- **v5.0+**: 公司 LLM Gateway (`http://127.0.0.1:18888/openai/v1`) → DeepSeek V4 Pro
- **v4.x**: 火山引擎直连 (`https://ark.cn-beijing.volces.com/api/v3`) → ***SECRET***
- Gateway 不需要真实 API key，占位符 `gateway-local-no-key-required` 即可
- config.py: `ark_base_url` / `ark_model_id` / `ark_api_key` 均指向 Gateway
- timeout 提升到 120s 适配推理模型

### Pitfall: DeepSeek V4 Pro 推理模型会截断 content（2026-08-16 验证）

DeepSeek V4 Pro（及其他 reasoning 模型）先输出 `reasoning_content`（思考过程）再输出 `content`（最终答案），两者**共用 `max_tokens`**。`max_tokens=1200` 时推理过程会吃掉大部分 token，导致最终正文被截成半句话（如结尾停在"……的关键"）。

- **症状**：LLM 返回的正文明显没写完，`finish_reason=stop` 但 content 不完整
- **诊断**：看返回的 `message` 有 `content` + `reasoning_content` 两个字段；`usage.completion_tokens_details.reasoning_tokens` 就是被推理吃掉的 token 数（实测 67/90 都给了推理）
- **修复**：批量生成长文（教程正文、路径介绍等）时 `max_tokens` 调到 **6000**，并加"content 长度 < 200 则判失败重试"的校验，防止把半截内容写进库

## Quick Start

> **项目别名**: 用户口中的「知渔」=「KnowHow知渔」= 本平台（品牌名 v5.x 起为「KnowHow知渔 · ai学习平台」）。
> **当前主目录**: `/Users/hua/6-产品研发/渔芯社区/知渔/` — **2026-10-30 实测**：旧主目录 `ok-KnowHow知渔/` 与 `23-ai培训教程/` 均已不存在，项目已迁入渔芯社区工作区并升到 v5.5.0（Docker/Makefile 启动、pytest 560 passed，见该目录 CLAUDE.md；`make run` 默认 127.0.0.1:8520）。本 skill 以下正文的旧路径均为历史口径，动手前一律以磁盘现状为准（沿下方 DB 多候选纪律 find 重定位）。

```bash
cd /Users/hua/6-产品研发/ok-KnowHow知渔

# 启动（推荐：项目自带脚本，后台运行，PID 写 logs/server.pid，日志 logs/server.stdout.log）
./start.sh --daemon          # 后台 → http://127.0.0.1:8520
./start.sh                    # 前台 uvicorn
./start.sh --status           # 查看运行状态
./start.sh --stop             # 停止

# 健康检查
curl -s http://127.0.0.1:8520/api/ops/health   # → {"ok":true,"version":"5.3.0",...}

# 直接跑后端（不推荐，绕过脚本）
python3 server/app.py  # port 8520

# Re-seed DB
python3 server/seed.py
# Re-seed content
python3 server/seed_content.py
```

**给用户打开前端测试**: 后台启动后用 `open http://127.0.0.1:8520` 在默认浏览器打开 + `browser_navigate` 自验页面加载（首页导航 + 登录/注册界面）。

## LLM Provider Configuration (v5.0+)

The platform routes all LLM calls through the company LLM Gateway at `http://127.0.0.1:18888`, not directly to volcengine.

**Config locations that matter for provider switches:**

| File | Field | Gateway Value |
|---|---|---|
| `server/config.py` | `ark_base_url` | `http://127.0.0.1:18888/openai/v1` |
| `server/config.py` | `ark_model_id` | `deepseek-v4-pro` |
| `server/config.py` | `ark_api_key` | `"gateway-local-no-key-required"` (placeholder — Gateway doesn't validate) |
| `server/config.py` | `ark_timeout` | `120` (reasoning models need more time) |
| `server/services/llm.py` | (reads from settings) | No hardcoded values — uses `settings.*` dynamically |

**Pitfall**: The `LLMClient.available` property checks `bool(self.api_key)`. Always set a non-empty placeholder even when using Gateway (it doesn't validate keys but the client won't send requests without one).

**Codex timeout warning**: When using `codex exec` to work on this project, Codex may time out trying to fetch model metadata from the Gateway. Fall back to direct edits (terminal + patch tools) if Codex hangs.

See `references/gateway-config.md` for full Gateway behavior notes (reasoning model SSE format, model name aliases, switch script).

## Architecture

```
FastAPI backend (server/app.py, port 8520)
  ├── LLM Gateway (127.0.0.1:18888 → DeepSeek V4 Pro)
  ├── SQLite (db/ai_learning.db)
  │   ├── terms (206 cards, depth/parent_id for sub-cards)
  │   ├── term_content (detail_md/examples/common_mistakes/related_terms)
  │   ├── career_paths → path_phases → path_terms
  │   ├── learning_log (SM-2 spaced repetition)
  │   └── ai_figures, ai_industry_tree, ai_creators
  └── Static files (web/index.html)
      └── Single-page vanilla JS app, no framework
```

## UI Layout — 左侧 Sidebar（渔芯前端参考标杆）

知渔前端是**左侧 sidebar 布局**，不是顶部横排导航栏（华哥 2026-08-16 明确纠正过）：

- `<aside class="sidebar">` 左侧栏：`sidebar-logo` + `sidebar-nav`（闪卡复习/学习卡片/阶段测验等入口，用 `data-p` 属性标识）+ `sidebar-footer`
- 移动端才收成 ☰ 汉堡按钮（`menu-btn` → `toggleSidebar()`）+ 底部 `bottom-nav`
- 内容区在右侧，靠 JS 切换页面视图

**华哥设计规范**：任何"参考知渔框架"做的渔芯产品（如初创者 33-初创者），布局**必须用左侧导航栏，不要做顶部横排导航**。做前端改造前先确认标杆产品的真实布局（`grep -nE "sidebar|<aside|<nav" web/index.html`），别凭印象猜。

**"路径 + 卡片"形式可复用**：知渔的内容组织 = 学习路径（`career_paths → path_phases → path_terms`，即"阶段(phase) → 卡片链(cards)"）+ 名词卡片（`terms`，统一卡片含 icon/名称/描述/难度/标签）。其他渔芯产品的结构化内容可套用同一形式，实测映射（初创者 2026-08-16）：
- "案例" → 用**路径式**展示：案例详情 = 阶段时间线（原场景 → AI方案 → 可复制SOP → 效果数据），SOP 每步 = 一张带编号圆圈的卡片
- "行业方案/新360行" → 用**卡片网格**展示：每张卡 = icon + 名称 + 摘要 + 痛点 + 关联案例数，`grid-template-columns: repeat(auto-fill, minmax(240px,1fr))`

## Key APIs

- `GET /api/terms` — list with depth + child_count
- `GET /api/terms/{id}` — full card with content + children + parent
- `GET /api/terms/{id}/children` — sub-card list
- `GET /api/terms/{id}/depth-tree` — recursive tree
- `POST /api/terms/{id}/content` — update detail content
- `GET /api/paths/{name}` — learning path with phases
- `POST /api/learn` — SM-2 spaced repetition
- `GET /api/dashboard` — learning dashboard (progress, mastery, streak, weak areas)
- `GET /api/recommend?limit=10` — smart recommendations (due review + weak terms + new cards)
- `GET /api/wrong-book` — weak area analysis based on low quiz scores
- `GET /api/quiz/{career}/{phase}?mode=normal` — generate quiz
  - `mode=normal` — 5 questions (default)
  - `mode=review` — questions from weak (score < 60) phases
  - `mode=intense` — 10 questions for intensive practice

### v4.6 Data Flywheel Effect APIs

Crowd wisdom injection and trending topic leaderboard:

- `POST /api/generate/matrix` (UPDATED in v4.6) — generates with crowd-sourced reference injection
  - Before generating, queries existing terms table for related cards (LIKE match on topic)
  - Injects up to 15 existing cards into the LLM prompt as "crowd wisdom reference"
  - LLM is instructed to: (1) fill blind spots (2) deepen existing content (3) fill in missing details
  - Flywheel effect: more users → more reference cards → better generated content → more users
  - **Result**: Quality increases non-linearly with user base
- `GET /api/generate/trending` — get hot topic leaderboard
  - Returns: `{trending: [{name, card_count, total_learns, learner_count}], total_cards, total_groups}`
  - Grouped by `group_name`, ordered by `total_learns DESC`
  - Card count shown in UI as `(${card_count})` next to each topic tag
  - Click handler calls `quickGen(topic)` which auto-fills the input and triggers generation

### v4.5.1 Project-Based Learning Framework

Six-stage progressive learning path (strict order):

| Stage | Focus | Content Type |
|-------|-------|-------------|
| 1. 理论基础 | What/why | Core concepts, definitions, problems solved |
| 2. 环境搭建 | Setup | Installation, config, hello-world, verification |
| 3. 核心能力 | 80% use case | Main APIs, basic workflows, essential operations |
| 4. 进阶提升 | Advanced | Edge cases, best practices, pitfall avoidance |
| 5. 性能优化 | Tuning | Performance analysis, bottlenecks, optimization |
| 6. 高效实战 | Real project | Automation, ecosystem, team collaboration |

**Matrix generation prompt pattern**:

```python
system_prompt = f"""你是专业的技术学习路径设计师。
用户想学习「{topic}」，请生成一个完整的分阶段项目学习方案，包含 {card_count} 张高质量学习卡片。

## 学习阶段划分（必须严格按这个顺序）
1. 理论基础（2-3张）
2. 环境搭建（1-2张）
3. 核心能力（3-4张）
4. 进阶提升（2-3张）
5. 性能优化（1-2张）
6. 高效实战（1-2张）

每张卡必须包含：action_item（学完后要做的具体动作）
难度分布：60% 入门，30% 进阶，10% 深入
实操导向：至少 40% 的卡片是 is_practical: true

返回 JSON：{{topic, learning_phases, path_description, cards: [...]}}
"""
```

**Card preview UI pattern**:
- Group cards by learning phase with section headers
- Visual distinction: 📖 theoretical, ✋ practical
- Each card shows `action_item` in a highlighted callout box
- Preview-first workflow: generate → review → batch import

**Hot topics quick-start pattern**:
- Show clickable tags below main input: "🔥 大家都在学: React (15), Agent (12)..."
- Tags styled with background color, rounded corners, hover effect
- Click fills input and triggers generation directly
- Builds social proof and reduces user effort (zero-type generation)

### Data Flywheel Growth Stages

| Stage | User Scale | Effect |
|-------|-----------|--------|
| Cold start | 1-10 | First users build foundation cards across domains |
| Acceleration | 10-100 | Hot topics emerge, generation quality visibly improves |
| Self-growth | 100+ | Platform content rich enough that generation quality surpasses any single expert |

## Quiz Question System

All 213 terms have pre-generated multiple-choice questions stored in the `terms` table:
- `quiz_q` — the question text
- `quiz_a` — correct answer text
- `quiz_options` — JSON array of 4 options

**Generation script**: Run once to populate all terms with 4-choice questions:
```python
import sqlite3, random
conn = sqlite3.connect('db/ai_learning.db')

for term_id, name, en, desc, depth in all_terms:
    # Get 3 distractor terms from same depth
    distractors = random.sample([t for t in terms if t[4]==depth and t[0]!=term_id], 3)
    options = [name] + [d[1] for d in distractors]
    random.shuffle(options)
    
    question = f"关于 '{name}' 的正确描述是？" if depth > 0 else f"'{en}' 的中文术语名称是什么？"
    
    conn.execute("""UPDATE terms SET quiz_q=?, quiz_a=?, quiz_options=? WHERE id=?""",
        (question, name, str(options).replace("'", '"'), term_id))
```

**Fallback**: If no pre-generated question exists, dynamically generate true/false questions with keyword swapping (AI ↔ 数据库, 模型 ↔ 服务器, etc.).

## Mastery Distribution Visualization

Dashboard API computes mastery levels from SM-2 `ef` (easiness factor):
- `ef >= 3.0` → 精通 (Master)
- `ef >= 2.5` → 熟练 (Proficient)
- `ef >= 2.0` → 掌握 (Competent)
- `ef >= 1.5` → 了解 (Familiar)
- `ef < 1.5` → 生疏 (Rusty)

## Quiz Submit API (`POST /api/quiz/submit`)

Schema note: The `quiz_log` table only tracks per-quiz results, not per-answer correctness. Fields available:
- `id, career_name, phase_order, total_questions, correct_answers, score_pct, taken_at, user_id`
- ❌ **No** `term_id` or per-answer `correct` columns — cannot track which specific questions were wrong
- ❌ **No** `created_at` — use `taken_at` instead

**Workaround**: Track weak areas by `phase_order` instead — phases with `score_pct < 70` indicate weak areas. Update all terms in that phase as needing review.

## Progressive Depth System

Three-tier learning: `state.learnDepth` (0/1/2)

| Level | Content | Trigger |
|-------|---------|---------|
| L1 | Card name + description | Default |
| L2 | + Sub-cards | depth >= 1 |
| L3 | + Full detail (collapsible sections) | depth >= 2 |

`setLearnDepth(d)` switches depth, clears `richCards` cache, re-renders.

## TTS Voice System

Browser SpeechSynthesis API (free, no API key). Voice: zh-CN "婷婷".

Two modes:
- `tts.mode='lecture'` — 讲稿式分段朗读 with highlight sync
- `tts.mode='quick'` — 单段速读

`tts.buildLecture(card, rich)` generates natural lecture sections:
1. "你好，我们来学习「X」这个概念。"
2. English name
3. Simple definition
4. Detail paragraphs (split at 80 chars)
5. Examples
6. Common mistakes
7. "总结一下...你学会了吗？"

Visual sync: `tts.onHighlight(idx)` callbacks highlight `.detail-section` elements with `.tts-highlight` class.

## Content Generation Template

Use `POST /api/terms/{id}/content` with:

```json
{
  "detail_md": "**加粗标题**\n\n### 小标题\n- 要点\n\n| 表格 | 数据 |",
  "examples": "案例1 | 案例2 | 案例3",
  "common_mistakes": "❌ 错误认知 → ✅ 正确理解",
  "related_terms": "关联名词1 T001 | 关联名词2 T002"
}
```

Markdown rendering: `simpleMD()` function converts `**bold**`→`<b>`, `### heading`→`<h4>`, `` `code` ``→`<code>`, `\n`→`<br>`.

## Patch Tool Escape-Drift Workaround

The Hermes `patch` tool double-escapes backslashes when replacing JavaScript regex patterns in HTML files. For any replacement involving `\\` in JS (regexes, template literals, `replace()`), use a Python script via `terminal` instead:

```python
with open('index.html', 'r') as f: content = f.read()
content = content.replace(old_bytes, new_bytes)
with open('index.html', 'w') as f: f.write(content)
```

Also: when using `patch` in JS template literals with `${...}`, the tool may escape-quote `\\\\\"` sequences incorrectly. Fall back to Python.

**CSS comment syntax bug**: The `patch` tool sometimes silently drops the closing `*/` of CSS comments, causing broken styles. Always verify CSS comment syntax after patching, and add missing `*/` manually if needed.

## Feature Iteration Workflow

For each version increment:

1. **Implement feature**: Backend routes first, then frontend UI
2. **Update CHANGELOG.md**: Summarize new features, optimizations, fixes
3. **Update README.md**: Bump version + new tagline
4. **Run full test suite**: `python -m pytest tests/` — target 100% green
5. **Cleanup**: Move deprecated scripts to `_archive/` (don't delete — preserves git history)
6. **Git commit**: `git add -A && git commit -m "feat: vX.Y ..."`
7. **Git tag**: `git tag vX.Y`
8. **Report**: 72 tests all green is the standard baseline (v4.4+)

**Release checklist template**:
```
✅ vX.Y 版本发布完成！

🎯 新增功能:
  • [功能1] - 描述
  • [功能2] - 描述

📊 版本信息:
  • Git 标签: vX.Y
  • 测试结果: 72 / 72 ✅ 全绿
  • 改动文件: N 个
  • CHANGELOG: 已同步
```

## DB Seeds

- `server/seed.py` — terms + paths from JSON files
- `server/seed_content.py` — detailed content for terms
- Terms JSON: `01-名词卡片/terms-full.json`
- Paths JSON: `career-paths.json`

## Adding a Learning Path (新增学习路径)

新增一条职业/学习路径 = 写 3 张表（`career_paths` / `path_phases` / `path_terms`）+ 按需新建卡片。核心数据源是 SQLite `db/ai_learning.db`；`career-paths.json` 只是 seed 源，运行时 API 只读数据库。

**不要用 `seed.py` 加单条路径**——它的 `seed_terms`/`seed_paths` 会 `DELETE FROM` 后全量重导（清空用户学习进度）。单条新增必须增量 INSERT。

### 表结构速查

| 表 | 关键字段 |
|---|---|
| `career_paths` | `name`(UNIQUE), `description`, `total_cards`, `level`, `level_label` |
| `path_phases` | `career_name`, `phase_order`(从1), `phase_name`, `phase_desc`, `phase_outcome` |
| `path_terms` | `career_name`, `phase_order`, `term_id`, `term_order`(从1) |

卡片表 `terms`：`id`(T001~), `name`, `en`, `description`, `difficulty`(⭐~⭐⭐⭐⭐), `layout`(hub/pipe/stack/comp/cycle), `labels`(用`|`分隔), `group_name`, `sort_order`, `content_md`(教程正文), `quiz_q/quiz_a/quiz_options`(JSON 数组字符串)。

### 新卡片编号与分组

- 最大编号：`SELECT id FROM terms ORDER BY CAST(substr(id,2) AS INTEGER) DESC LIMIT 1`（2026-08 为 220，从 T221 继续）
- `group_name` 用现有分类：`AI开发工具` / `AI Agent` / `基础概念` / `前沿方向` / `RAG与知识` / `部署与MLOps` 等

### 幂等写入模式（可直接套用）

- 卡片：`INSERT OR IGNORE INTO terms (...)`，或先 `SELECT id` 判断再插
- 路径：`INSERT INTO career_paths ... ON CONFLICT(name) DO UPDATE SET ...`
- phases / terms：先 `DELETE FROM path_phases WHERE career_name=?`（和 `path_terms`）再插入，保证干净
- terms 插入自动触发 FTS 触发器同步全文搜索，无需手动 rebuild

### 步骤顺序

1. **先备份**（这个库有损坏前科，见下文 DB Corruption Recovery）：`cp db/ai_learning.db /tmp/ai_learning.db.bak_$(date +%s)`
2. 幂等写入卡片 + 路径 + phases + terms（建议 `write_file` 写一个临时 `.py` 脚本再 `python3` 执行，避免 heredoc 中文+emoji 触发安全扫描）
3. **API 验证**：`GET /api/paths?search=<关键词>` 能搜到；`GET /api/paths/{name}` 返回 phases/cards 完整（注意 `list_paths` 有 `@cached(ttl=300)`，新路径可能延迟可见，用 get 详情验证最准）
4. **预热介绍页**：`GET /api/paths/{name}/intro` 调 LLM 生成 `easy_explain/goal/prerequisites/deliverables/work_ready_combo` 落库 `path_intro`；不预热则用户第一次打开路径时前端触发（LLM 不可用会 503）。批量预热多条路径时 LLM 网关易限流，单条返回 503 时 `sleep 5` 后重试即可成功

### Pitfall: 卡片脚本 content_md 转义坑（2026-08-16 验证）

批量写卡片脚本时，`content_md` 里的代码块/多行内容**必须用 `\n` 转义成单行字符串**，不能写真实换行——Python 字符串字面量不支持跨行，会报 `unterminated string literal`。另外 `quiz_options` 用单引号包裹时，值里含单引号（如 `grep ':80 '`）会提前终止字符串，报 `{ was never closed`。

**防坑**：写完脚本先 `python3 -c "import py_compile; py_compile.compile('脚本', doraise=True)"` 做语法检查再执行。发现转义问题用 `str.replace` 精确替换修复（`execute_code` 在 cron 模式被阻止时，用 `write_file` 写修复脚本 + `terminal` 执行）。

### Pitfall: 批量操作 terms 别加 `depth=0` 过滤（2026-08-16 验证）

`terms.depth` 字段：0=主卡，1/2=子卡片。批量补 content_md/description 等时，若在 WHERE 里加 `AND depth=0`，会把占多数的子卡片（depth≥1）排除——实测 83 张空卡里 depth=0 只有 15 张，depth=1/2 有 68 张，脚本"正常跑完"其实只处理了主卡。

**防坑**：
- 批量生成/更新内容时**不要按 depth 过滤**，除非明确只处理主卡
- 生成完用 `SELECT COUNT(*) FROM terms WHERE content_md IS NULL OR content_md=''` 验证是否真补全，别只看脚本退出码（`exit_code=0` ≠ 跑完了目标数量）

### Pitfall: path_intro 缺 easy_explain 列（2026-08-16 修复）

`schema.py` 迁移 22 建 `path_intro` 表时漏了 `easy_explain` 列，但 `routes/paths.py` 的 `_persist_intro` / `_load_intro` 一直在读写它 → **所有路径介绍页 500**（报 `table path_intro has no column named easy_explain`）。

修复：`MIGRATIONS` 追加
```python
(39, "ALTER TABLE path_intro ADD COLUMN easy_explain TEXT DEFAULT ''", "path_intro.easy_explain"),
```
运行中的库立即生效（WAL 支持并发写，不必重启服务）：
```bash
sqlite3 db/ai_learning.db "ALTER TABLE path_intro ADD COLUMN easy_explain TEXT DEFAULT '';"
sqlite3 db/ai_learning.db "INSERT OR IGNORE INTO schema_version (version,note) VALUES (39,'path_intro.easy_explain');"
```

## 批量内容补齐工作流（content_md + phase_outcome）

"内容丰富"类任务 = 批量调用 LLM Gateway 给空字段补内容。先审计缺口，再批量生成。

### 字段完整度审计（先查再补）

```bash
cd /Users/hua/6-产品研发/ok-KnowHow知渔
sqlite3 db/ai_learning.db "SELECT COUNT(*) FROM terms WHERE content_md IS NULL OR content_md='';"            # 教程正文缺口
sqlite3 db/ai_learning.db "SELECT COUNT(*) FROM path_phases WHERE phase_outcome IS NULL OR phase_outcome='';"  # 阶段产出缺口
sqlite3 db/ai_learning.db "SELECT COUNT(*) FROM path_phases;"   # 阶段总数（218 个阶段里 165 空 = 76% 属常态）
```

**Pitfall — 表名是 `path_phases` 不是 `phases`**：查阶段数据时 `SELECT ... FROM phases` 报 `no such table: phases`。阶段数据在 `path_phases`（字段 `career_name, phase_order, phase_name, phase_desc, phase_outcome`），卡片链在 `path_terms`（`term_id` 关联 `terms`，用 `term_order` 排序）。

### 批量生成脚本（可复用）

脚本：`scripts/enrich_content_md.py`（补 `terms.content_md`）+ `scripts/enrich_phase_outcomes.py`（补 `path_phases.phase_outcome`）。核心要点：

- 走 LLM Gateway `http://127.0.0.1:18888/openai/v1/chat/completions`，模型 `deepseek-v4-pro`，key 占位 `gateway-local-no-key-required`
- **`max_tokens` 必须 6000**（推理模型 reasoning_content 会吃掉 token，见上文 DeepSeek 截断 pitfall），并加"content 长度 < 阈值则判失败"校验
- phase_outcome 脚本用 `path_phases LEFT JOIN path_terms LEFT JOIN terms ... GROUP_CONCAT(t.name)` 把阶段内卡片名喂给 LLM，产出"学完能独立做什么"一句 20~40 字
- 逐条 UPDATE + commit，单条失败只影响那一行，脚本继续
- 165 个阶段约 40~60 分钟（约 4 张/分钟），用 `terminal background=true + notify_on_complete=true` 跑，别前台阻塞；用 `process(action=poll)` + 查库计数确认进度（脚本 stdout 可能因缓冲延迟显示）

### 跑完验证

```bash
sqlite3 db/ai_learning.db "SELECT COUNT(*) FROM terms WHERE content_md IS NULL OR content_md='';"            # 应 = 0
sqlite3 db/ai_learning.db "SELECT COUNT(*) FROM path_phases WHERE phase_outcome IS NULL OR phase_outcome='';"  # 应 = 0
```

别只看脚本退出码/打印——`exit_code=0` ≠ 目标数量补全（见上文 depth=0 过滤坑）。

## Job Database (`ai_jobs`)

The `ai_jobs` table tracks real AI job postings for the career guidance section. Schema and update workflow in `references/job-database.md`.

### Weekly Cron Update

Run as a cron job every Monday. The workflow:

1. Query existing jobs: `SELECT id, title, company, term_ids FROM ai_jobs ORDER BY id`
2. Search for new AI jobs (see anti-bot note below)
3. Insert 5-10 new jobs with all required fields
4. Expire stale jobs (update status + updated_at) — either ~5 hand-picked, or a deterministic month-sweep of any whole batch whose month-level `posted_date` is >1 month old (see `references/job-database.md` → "Expiring Jobs")
5. Report: new count, expired count, active count, category breakdown

Reusable script: `scripts/update_ai_jobs.py` (parameterized INSERT + expire + report; copy, fill `new_jobs`/`expire_ids`, run). The `term_ids` map in `references/job-database.md` is the authoritative one — verify IDs against `terms` before writing.

#### Pitfall: `sqlite3 db < file.sql` does NOT stop on parse errors — retrying the fixed file creates DUPLICATE rows (2026-09-14 incident)

The sqlite3 CLI prints parse errors but keeps executing subsequent statements. The weekly file (1 UPDATE-expire + N INSERTs) that fails on a mid-file quote bug has ALREADY committed everything before the error; re-running the corrected file then (a) re-expires the prior run's still-active inserts, (b) re-inserts every job as a duplicate with fresh autoincrement ids. Verified arithmetic from 2026-09-14: 3 attempts (3 quote bugs fixed one at a time) → 11 ghost rows (ids 48–58, all status='expired') duplicating the 7 real jobs (ids 59–65). The active set stays correct — each retry's UPDATE expires the prior copies — so the UI is unaffected and the corruption hides in expired-row counts only. **Tell-tale symptom: an "expired count larger than expected" in the weekly report.**

**Prevention** — run the weekly file atomically (file starts with `BEGIN;` and ends with `COMMIT;`):
```bash
sqlite3 -bail "/path/to/ai_learning.db" < /tmp/ai_jobs_weekly_update.sql
```
`-bail` stops at the first error; BEGIN/COMMIT makes the run all-or-nothing so a failed attempt leaves zero partial state and can simply be re-run after the fix.

**Recovery after an already-duplicated run**:
```sql
SELECT title, company, posted_date, COUNT(*) c FROM ai_jobs
 GROUP BY title, company, posted_date HAVING c > 1;   -- find ghosts
DELETE FROM ai_jobs WHERE id BETWEEN 48 AND 58;       -- 2026-09-14 leftover ghosts (expired dupes of the 7 live jobs)
```

**Root cause of the failed parses — ASCII apostrophes inside Chinese JD prose**: quoting a term inside requirements text as '数据元年' / '超级个体' / '大脑' with ASCII single quotes terminates the SQL string literal early (`Parse error near "数据元年"`). Use Chinese quotes “ ” for any quoted term in Chinese prose, and lint the file before executing (BSD grep lacks `-P`, use python):
```bash
python3 -c "import re,sys; s=open(sys.argv[1],encoding='utf-8').read(); bad=[l for l in s.splitlines() if re.search('[\u4e00-\u9fff]'[\u4e00-\u9fff]',l)]; print('BAD:',bad[:3]) if bad else print('OK')" /tmp/ai_jobs_weekly_update.sql
```

### Anti-Bot Challenges (verified 2026-08-03)

Status of major Chinese job/news sites when scraped by Hermes browser tools in default local mode (no residential proxy):

| Site | Status | Notes |
|------|--------|-------|
| **猎聘 liepin.com** | ✅ **WORKS** | Job cards render to real DOM (`a[data-nick="job-detail-job-info"]`), full JD text visible on detail pages (`/job/<id>.shtml`), update date stamped in body. First-choice source. |
| **36kr.com** | ✅ **WORKS** | Article search `/search/articles/<kw>?sort=date` returns dated cards; article detail pages expose full text via `document.body.innerText`. Best for market-trend intelligence (e.g., "具身智能招聘市场热"). |
| **BOSS直聘 zhipin.com** | ❌ **BLOCKED** | Canvas rendering, security-check overlay, encrypts job IDs. No DOM access without residential proxy + cookie replay. Skip entirely in cron mode. |
| **百度搜索 baidu.com** | ❌ **BLOCKED** | Returns CAPTCHA wall (`wappass.baidu.com/static/captcha/...`). Even simple site-restricted searches fail. |
| **Google 搜索** | ❌ **BLOCKED** | DNS / connection timeout from current network. Not reachable from cron environment. |

**Strategy**: 猎聘 (real jobs w/ salaries) + 36kr (market trends & company expansion news) is sufficient for one weekly update. Do NOT spend tool budget on blocked sites — confirm block once, then move on.

**Web-search fallback (verified 2026-08-17)**: When browser scraping is blocked/unavailable, `web_search` alone is enough to compile the weekly update. Search snippets from BOSS直聘/猎聘 carry `title | company | location | salary | experience` inline, and news articles (腾讯/小米 2027 校招, DeepSeek Harness 扩编, 21财经 AI训练师薪资) supply concrete roles + salary bands + requirements you can write 4-6 bullets from. Note: the default `web_extract` backend is DuckDuckGo which is **search-only** — it cannot extract URL content and returns `"DuckDuckGo (ddgs) is a search-only backend"`. Don't waste calls on `web_extract`; rely on `web_search` snippets + report body instead. `web_search` may intermittently time out (`ConnectTimeout`) — retry the same query, or rephrase, rather than abandoning.

**Pitfall — tool budget**: A full weekly update must complete within ~25-30 browser tool calls. Budget breakdown that fits:
- 1 call: search/listings page → extract job URLs from DOM in one `browser_console` call
- 3-4 calls: open detail pages for the top candidates (one `browser_navigate` + one `browser_console` extract per job)
- 1 call: final DB write via `terminal python3 -c "..."` heredoc
- Reserve 2-3 calls for verification + retries
If you find yourself doing more than 5 sequential `browser_navigate` calls into detail pages, you are over-budget — write the rows from listing-page data only and note which need JD enrichment next week.

See `references/job-scraping-recipes.md` for concrete selectors and extraction patterns.

### Cron-Mode Constraints

- `execute_code` is BLOCKED in cron mode — use `terminal` with `python3 -c` or write-file-then-execute instead
- **Heredoc pitfall**: `python3 << 'PYEOF' ... PYEOF` in terminal may be flagged by the security scanner (confusable Unicode detection on CJK text mixed with ASCII). Workaround: use `write_file` to create a `.py` file, then execute it with `python3 /path/to/script.py`. This avoids the scanner entirely and produces a re-runnable artifact.
- **`skill_view` name-resolution failure (2026-09-14 verified)**: `skills_list` can show a skill while `skill_view(name)` returns `not found` — registry names may differ from the on-disk directory (example from 2026-09: registered `cron-mode-tool-constraints` lived at `skills/devops/***SECRET***/`; **2026-10-30 curator move: that directory now lives inside `cron-self-evolution-toolkit/references/***SECRET***/`**; retrying the same name loops forever). Workaround that always works: locate the file with `search_files` (`pattern: **/<name>/SKILL.md`, `path: /Users/hua/.hermes`) and `read_file` it directly. Same fallback for reading a sibling-profile skill you must consult but not modify.
- Pipe-to-interpreter (`curl | python3`) triggers security approval — split into two steps: download to temp file, then process
- `terminal` network requests may time out after 30s — use `--max-time` flag

### Required Job Fields

Every job MUST have: `title`, `company`, `location`, `salary`, `experience`, `education`, `requirements`, `skills` (pipe-separated), `term_ids` (pipe-separated T-IDs from `terms` table), `url`, `source`, `category`, `posted_date`, `updated_at`, `status` (active/expired).

**Pitfall — `terms` column is `name`, not `term`**: `SELECT id, term FROM terms` fails with `no such column: term`. The card name column is `name` (`SELECT id, name, group_name FROM terms ORDER BY id`). Verify term IDs exist before writing them: `SELECT id FROM terms WHERE id IN ('T061',...)`. The authoritative term→ID map (accurate as of 2026-08, T001–T287) lives in `references/job-database.md` — DO NOT trust memory or the legacy table in old INSERT examples (they carry wrong mappings like T147=Agent开发, which is actually "AI for Science").

See `references/job-database.md` for full schema and example INSERT.

### Multiple DB Locations & Schema Drift (re-verified 2026-09-07)

There are now MULTIPLE "知渔" SQLite DBs — always confirm which path the task/cron actually points to before writing (do NOT assume the skill's documented 主目录), and re-locate when the named path is missing. **The cron prompt itself can carry a stale path**: on 2026-09-07 it still hard-coded 渔芯独角兽/知渔/db, which no longer exists. Discovery that worked: `find /Users/hua -name "ai_learning.db" -not -path "*/Library/*" -not -path "*/.Trash/*"`, then confirm the candidate holds data (`SELECT COUNT(*) FROM ai_jobs; SELECT COUNT(*) FROM terms;`) before writing.

| Path | State |
|---|---|
| `/Users/hua/Documents/New project/ai-learning/db/ai_learning.db` | **LIVE** — docker compose + sqlite platform in the "New project" workspace (own AGENTS.md); 220 terms (T001–T220); ai_jobs at 65 rows (7 active / 58 expired) after the 2026-09-14 weekly run (ids 48–58 are a known leftover duplicate batch — see the `-bail` pitfall above) |
| `/Users/hua/6-产品研发/渔芯独角兽/知渔/db/ai_learning.db` | **GONE** — `知渔/` dir no longer exists under 渔芯独角兽/ (cron prompts keep carrying this stale path — re-verify every run) |
| `/Users/hua/6-产品研发/渔芯独角兽/00-基本完成/知渔/db/ai_learning.db` | parallel snapshot (287 terms incl. T227–T287; not re-verified since 2026-09-07) |
| `/Users/hua/6-产品研发/ok-KnowHow知渔/db/ai_learning.db` | older snapshot (not re-verified since 2026-09-07) |

Terms count drifts per copy (206 / 220 / 287) — a T-ID valid in one copy may not exist in another. Always validate term_ids against the LIVE terms table of the DB being written.

**Pitfall — the empty target DB may carry a SIMPLIFIED `ai_jobs` schema.** The fresh `知渔/db` table had only `id,title,company,category,salary,location,skills,requirements,term_ids,updated_at` — missing `source`, `status`, `experience`, `education`, `url`, `posted_date`. The task requires `source` (来源) and status tracking (标记新岗位/下线过期岗位), so you MUST migrate the schema before INSERT, never assume it matches production:

```python
cols = {r[1] for r in cur.execute("PRAGMA table_info(ai_jobs)")}
missing = [("experience","TEXT DEFAULT ''"), ("education","TEXT DEFAULT '本科及以上'"),
           ("url","TEXT DEFAULT ''"), ("source","TEXT DEFAULT '公开招聘'"),
           ("posted_date","TEXT"), ("status","TEXT DEFAULT 'active'")]
for name, ddl in missing:
    if name not in cols:
        cur.execute(f"ALTER TABLE ai_jobs ADD COLUMN {name} {ddl}")
```

When the table is empty (0 rows), there are no stale jobs to expire — report `expired=0` instead of hunting for old rows. **Canonical reference script**: `00-基本完成/知渔/db/update_ai_jobs_20260817.py` shows the full workflow (backup → INSERT new jobs → expire stale → report). The `terms` table in the production DB has grown to **T287** with new `group_name`s 网络安全(T227–T241) / 爬虫技术(T242–T257) / 网络技术(T258–T287) — always re-verify `term_ids` against the LIVE `terms` table (`SELECT id,name,group_name FROM terms`), never trust memory.

### Database Corruption Recovery

The `ai_learning.db` SQLite file can become corrupted while the server is running (e.g. `sqlite_autoindex_users_1` index damage). Symptoms: `SELECT` works but `INSERT`/`UPDATE` fails with `sqlite3.DatabaseError: database disk image is malformed`. Integrity check (`PRAGMA integrity_check`) reports specific index corruption.

**Recovery procedure** (works even when `.dump`/`.recover` fail):
```bash
# 1. Clone to a new file — this filters out corruption
sqlite3 "/path/to/ai_learning.db" ".clone /tmp/ai_learning_clone.db"

# 2. Verify the clone is clean
sqlite3 /tmp/ai_learning_clone.db "PRAGMA integrity_check;"  # should return "ok"

# 3. Backup the corrupted original (keep for forensics)
cp /path/to/ai_learning.db "/path/to/ai_learning.db.corrupted_bak_$(date +%Y%m%d_%H%M%S)"

# 4. Remove WAL/SHM files (they're tied to the old DB)
rm -f /path/to/ai_learning.db-shm /path/to/ai_learning.db-wal

# 5. Deploy the clean clone
cp /tmp/ai_learning_clone.db /path/to/ai_learning.db
```

The `.clone` approach preserves all tables and data while dropping the corrupted index structures. FTS5 virtual tables may emit benign warnings during clone (they're rebuilt on access) — this is harmless. Run all DB writes against the clone, then deploy it to production.

**When to suspect corruption**: if `SELECT COUNT(*)` works but any write fails with "malformed", the DB is corrupted. Don't try `REINDEX` or `PRAGMA writable_schema` — clone is the safest recovery path.

See `references/db-corruption-recovery.md` for detailed diagnosis steps.

## Doubao Image Generation Workflow (豆包生图)

For generating visual flashcard illustrations using Doubao (豆包) AI image generator.

### Directory Structure

Two project locations are supported (note lowercase 'ai'):
- Primary: `/Users/hua/6-产品研发/23-ai培训教程/`
- Secondary: `/Users/hua/Desktop/渔芯科技/workspace/AI培训平台工程/`

```
02-豆包生图Prompt/     # Prompt files (ready for generation = ok- prefix)
  ├── 00_生图总览.md     # Progress tracking
  ├── ok-领域_*.md       # Completed prompt files (marked as ready)
  └── 领域_*.md          # Draft prompt files (not yet ready)
03-豆包生成图/          # Renamed images (standard naming convention)
  └── T001-AI（人工智能）_v1.png, T002-*_v2.png, etc.
04-最终精选图/          # Best images selected for platform
05-学习平台/            # Final platform integration
```

### Image Naming Pattern

**Standard format**: `T{XXX}-{中文术语}_v{N}.png`

Example: `T001-AI（人工智能）_v1.png`, `T002-机器学习_v4.png`

### Batch Renaming Workflow

**First step**: ALWAYS search for existing Doubao-generated images before creating new ones. The user typically generates images manually first.

```bash
# Search for recently downloaded PNGs/JPEGs
find ~ -name "*.png" -mtime -1 2>/dev/null | head -30
find ~ -name "*.jpg" -mtime -1 2>/dev/null | head -30
ls -lt ~/Downloads/ 2>/dev/null | head -30
ls -lt ~/Desktop/ 2>/dev/null | head -30
```

**When execute_code is blocked (cron mode)**: Use sequential terminal commands. This is more reliable and avoids security restrictions:

```bash
cd "/Users/hua/6-产品研发/23-ai培训教程/02-豆包生图Prompt"

# T001 - AI（人工智能）- first 4 images (no t-prefix)
cp "AI名词知识卡片风格分析.png" "../03-豆包生成图/T001-AI（人工智能）_v1.png"
cp "AI名词知识卡片风格分析 (1).png" "../03-豆包生成图/T001-AI（人工智能）_v2.png"
cp "AI名词知识卡片风格分析 (2).png" "../03-豆包生成图/T001-AI（人工智能）_v3.png"
cp "AI名词知识卡片风格分析 (3).png" "../03-豆包生成图/T001-AI（人工智能）_v4.png"

# T002 - 机器学习 (t002 prefix)
cp "t002AI名词知识卡片风格分析.png" "../03-豆包生成图/T002-机器学习_v1.png"
cp "t002AI名词知识卡片风格分析 (1).png" "../03-豆包生成图/T002-机器学习_v2.png"
cp "t002AI名词知识卡片风格分析 (2).png" "../03-豆包生成图/T002-机器学习_v3.png"
cp "t002AI名词知识卡片风格分析 (3).png" "../03-豆包生成图/T002-机器学习_v4.png"

# T003-T012 follow similar pattern with continuous numbering
# T005 starts at (4), T006 at (8), T007 at (12), T008 at (16), etc.
```

**When execute_code is available**: Use the Python pattern for larger batches:

```python
import os
import re
import shutil

source_dir = "02-豆包生图Prompt"
target_dir = "03-豆包生成图"
os.makedirs(target_dir, exist_ok=True)

# T-ID to Chinese name mapping (from prompt files)
t_names = {
    "T001": "AI（人工智能）",
    "T002": "机器学习",
    # ... add all terms
}

# Group and sort images
for filename in sorted(os.listdir(source_dir)):
    if not filename.endswith('.png'):
        continue
    
    # Detect T-ID: match t002 prefix or bare numbers for T001
    match = re.match(r'^t(\d{3})', filename, re.IGNORECASE)
    if match:
        t_id = f"T{match.group(1).upper()}"
    else:
        t_id = "T001"  # Files without t-prefix are T001 (first term)
    
    # Sort within group: no parentheses = v1, then by number in parentheses
    # Rename to standard format and copy to target_dir
```

### User Mandate for Image Renaming

**NON-NEGOTIABLE: User-established rule (2026-07-13)**
> "必须每一张都识别一次再编号。"
> "Must identify EVERY SINGLE image before renaming."

You MAY NOT:
- Assume pattern holds based on first few samples
- Trust manifest file numbers without verification (manifest said T001 had 36 images; actual pattern is 4 per term)
- Proceed with bulk renaming if you cannot reliably identify images

**When vision tools fail (they DO fail, often):**
1. Be honest immediately — "My vision tools cannot reliably identify these images"
2. Propose clear options to the user:
   - Option A: Proceed with timestamp-based ordering + manifest (user spot-checks)
   - Option B: User verifies 2-3 key points to confirm pattern, then proceed
   - Option C: Wait for user to provide verified mapping
3. Never pretend you identified images when you couldn't

### Key Pitfalls

1. **T001 has no prefix**: The first batch (T001) generates with no t-prefix in the filename
4. **T001 may have MANY images**: The first term (T001) may have 36+ images from multiple generation runs, not just 4 — verify actual count, don't trust manifest
5. **Vision tools hallucinate**: `browser_vision` and `vision_analyze` are unreliable. They may return unrelated content (e.g., medical terminology instead of AI labels) or claim "cannot see image". Verify every result, don't trust blindly.
5. **Mark `ok-` prefix**: When a prompt file is ready for generation, prefix with `ok-` (e.g., `ok-领域_基础概念.md`)
6. **Don't generate fallback images first**: The user often generates images manually in Doubao first and needs them renamed/organized. ALWAYS search for existing images in Downloads/Desktop first before generating fallback PIL images. Search paths: `~/Downloads/`, `~/Desktop/`, and Doubao app containers. Use `find ~ -name "*.png" -mtime -1` to locate recently generated images.
7. **Clarify before generating**: If the user says "rename images" or "change filenames", they mean their existing Doubao-generated images, not new generated ones. Only generate PIL images as a last resort when no existing images can be found.
8. **execute_code blocked in cron mode**: When running as a cron job, `execute_code` is blocked for security. Use sequential `terminal` `cp` commands instead for batch file operations:
   ```bash
   cp "source-file.png" "../03-豆包生成图/T001-AI（人工智能）_v1.png"
   cp "source-file (1).png" "../03-豆包生成图/T001-AI（人工智能）_v2.png"
   # Continue with explicit commands
   ```
9. **Directory path case sensitivity**: The project directory may be `23-ai培训教程` (lowercase 'ai') not `23-AI培训教程`. Always verify the actual path first.

### Progress Tracking

Update `00_生图总览.md` with:
- Table of domains with status (⏳待处理 / ✅待生成 / 已完成)
- Per-term image count
- Total image count in `03-豆包生成图/`
- Links to the directory

**Script**: See `scripts/rename_images.py` for the ready-to-use batch renaming utility.

### Fallback Image Generation (PIL/Python)

When Doubao web login or API authentication is unavailable, generate concept images programmatically with Python PIL. This produces neural-network style visuals suitable for educational flashcards.

**Script location**: `scripts/generate_concept_images.py`

**Usage**:
```python
python3 scripts/generate_concept_images.py \
  --term-id T001 \
  --term-name "AI（人工智能）" \
  --output-dir "03-豆包生成图/" \
  --styles 4
```

**What it generates** (4 style variations):
- **科技蓝** (Tech Blue): Dark blue background with glowing blue nodes
- **未来紫** (Future Purple): Purple theme for futuristic look  
- **活力橙** (Vibrant Orange): Warm colors for energy concepts
- **清新绿** (Fresh Green): Green theme for growth/nature-related AI

**Visual elements included**:
- 5-layer concentric neural network node layout
- 150+ random connecting lines (data flow)
- Center brain-shape halo (5 circles expanding outward)
- Chip icon (with pins) for hardware reference
- 20+ diagonal data stream particles

**Image specs**: 1024x1024 PNG, ~70KB each.

**When to use**:
- Doubao login required but no session available
- API key authentication failures
- Rate limiting or service downtime
- Need consistent visual style across all cards
- Generating placeholder images before AI generation

## 图片-卡片映射机制与图文核对（2026-08-16 实战）

### 图片靠文件名匹配卡片，映射存数据库 `card_images` 表

图片本身**不在 terms 表里**（terms 无 image 字段）。映射链路：

```
04-最终精选图/t{编号}-{中文名}-v01.png   ← 文件名以 term_id 开头（大小写不敏感）
        ↓ scan_card_images.py (POST /api/cards/sync)
card_images 表 (term_id, style_key, url, variant, is_primary)
        ↓ GET /api/styles/{style}/primary
前端 IMG_MAP[card.id] → 图片路径
```

- `card_images` 表字段：`term_id, style_key(ai/comic/humor), url(/styles/ai/xxx.png), variant, is_primary`
- 扫描器 `scripts/scan_card_images.py`：**只新增/更新，不删除**已不存在的文件记录；同 `term_id+style_key+variant` 冲突时 `ON CONFLICT DO UPDATE`
- 前端 primary = 每个 term_id 里 `variant` 最小 + `is_primary=1` 的那条

### 图文不符检查流程（华哥要求"图片内容与文字内容不符"时）

1. **文件名层面比对**（最快，先做）：提取图片文件名里的中文名，和 `terms.name` 比对，抓出"文件名概念 ≠ 卡片名概念"的
2. **vision 抽查坐实**：对嫌疑图 `vision_analyze`，确认图片左上角印的编号/标题和卡片名对不上（注意：豆包生图会把编号 TXXX 印进图里，可作铁证）
3. **修复要改两处**：
   - 文件系统：错位图重命名归位 / 孤儿图移到 `_orphan_bak_日期/`（可逆，不直接删）
   - 数据库：`card_images` 表 `UPDATE` 归位图的 url、`DELETE` 孤儿图记录——**只改文件系统不够**，前端读库

### 常见错位模式（渔芯实测）

- **旧卡片图残留**：一批卡片（如 Agent 框架 OpenHands/SWE-agent/AutoGPT）被删/重构后，旧图占着编号，挂到新卡片上（T205 挂 OpenHands 图但卡片名是"MCP协议"）
- **编号笔误孤儿图**：`t312-llama.cpp`（应为 T132）、`t447`（应为 T147）、`t503`（应为 T053）——多打一位数字，图成了孤儿
- **乱码编号**：`t0277-System Prompt`（应为 T054）、`t0056-流式输出`（应为 T056）——图内容对但编号错，应归位而非删除

## 培训内容方法论（原 ai-training-content）

零基础员工 AI 培训的内容设计方法论。核心是**需求驱动**（demand-driven，反向设计）：从学员目标出发（"我要做 X → 需要学 Y"），不是从知识树出发。

**7 原则**：① 从需求出发（支持 39 职业路径 + 任务关键词匹配）② 先学最需要的（UbD 反向设计，`--minimal` 把 52 卡路径砍到 ~20 卡

…(truncated)
