jerryliu0103
- 6 skills
- 0 followers
- 1 day ago last updated
- ▌ Curation Pipeline 2 · jerryliu0103AI 輔助資料策展的 staging 閘門紀律——代理只寫 staging、唯讀的 validator 檢查、人核准，然後才有東西進正本。當你要建立或檢討一條「由 LLM 代理供稿到資料庫、知識庫或策展資料集」的流水線、要決定哪個代理可以寫哪裡、或是一份策展資料集長出了重複、無來源的列或安靜的缺口時使用。
- ▌ Provenance And Dedup 2 · jerryliu0103怎麼記錄策展資料的來源、以及怎麼偵測一份來源已經被匯入過——用來源座標而非文字比對、對代理萃取結果做雙向核對、用刻意丟棄清單讓代理的判斷可被審查。當同一份來源可能被送兩次、代理的產出需要檢查有沒有漏、要決定 schema 需要哪些來源欄位、或一份策展資料集長出了重複時使用。
- ▌ Silent Failure Hunting 2 · jerryliu0103一份不會產生錯誤訊息的資料流水線失敗清單——什麼都沒配對到的 JOIN、其實是真寫入的試跑、把列數放大的 fan-out、因編碼錯誤而死掉的 guard、寫好卻沒合併的工具。當流水線回報成功但資料是錯的、要檢討匯入或同步腳本、要決定批次寫入後該驗什麼、或要設計一個「runtime 不會替你抓」的檢查時使用。
- ▌ Curation Pipeline · jerryliu0103The staging-gate discipline for AI-assisted data curation — agents collect into staging, a read-only validator checks, a human approves, and only then does anything reach the canonical store. Use when building or reviewing a pipeline where an LLM agent contributes data to a database, knowledge base, or curated dataset, when deciding which agent may write where, or when a curated dataset has developed duplicates, unattributed rows, or silent gaps.
- ▌ Provenance And Dedup · jerryliu0103How to record where curated data came from and how to detect that a source was already imported — provenance coordinates instead of text comparison, bidirectional reconciliation of what an agent extracted, and the deliberate-discard list that makes an agent's judgment reviewable. Use when the same source may be submitted twice, when an agent's output must be checked for omissions, when deciding what provenance fields a schema needs, or when a curated dataset has developed duplicates.
- ▌ Silent Failure Hunting · jerryliu0103A catalogue of data-pipeline failures that produce no error message — joins that match nothing, dry runs that were real writes, fan-out that duplicates rows, guards that die on an encoding error, and tools that were written but never merged. Use when a pipeline reports success but the data is wrong, when reviewing an import or sync script, when deciding what to verify after a batch write, or when designing a check that has to catch something the runtime will not.