# Pdh Android Collector

> Build or debug a Personal-Data-Hub collector that pulls a Chinese Android app's data off a rooted phone over ADB. Encodes the real-device rule — prefer on-device plaintext SQLite over fragile web APIs (cookies/signing/endpoints drift; local DBs are stable + sign-free).

- Skill: `chainlesschain/pdh-android-collector` (Agent Skill)
- Install (CLI): `npx skillmds@latest add chainlesschain/pdh-android-collector`
- Raw SKILL.md: https://api.skillmd.com/api/skills/chainlesschain/pdh-android-collector/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Web & Frontend
- Author: chainlesschain (https://skillmd.com/u/chainlesschain)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/chainlesschain/pdh-android-collector

---


# PDH 安卓本地采集器构建 (pdh-android-collector)

The durable way to collect a Chinese app's data is **reading its on-device plaintext
SQLite DBs over ADB+root**, NOT calling its web API. Web APIs drift constantly (cookie
schemas, signing salts, endpoint paths, permission codes); the local DBs are stable and
need no signing. Reach for a web fetcher only when the data genuinely isn't on-device.

Real-device provenance (device 5lhyaqu8lbwstc6x / Xiaomi chopin, 2026-06-11): this
recipe cracked Toutiao (uid from `account_db.login_info` when the web profile returned
`error_code 16`) and Douyin (`video_record.db` `record_<uid>` = 900 plaintext watch
records, sidestepping the SQLCipher IM db). Reference memory `pdh_real_device_findings_2026_06_11`
and `pdh_local_db_over_web_api_strategy`.

## Phase 0 — Map the device (always first)

```bash
adb devices -l                  # device + transport
adb shell "su -c id"            # root (uid=0) — release APKs aren't debuggable
adb shell pm list packages -3   # which target apps are installed
```

Map installed packages → adapters in `packages/personal-data-hub/lib/adapters/`. Only
installed apps verify. The app must be **logged in AND have generated the data** — most
"0 rows" results are login/usage state, not collector bugs.

## Phase 1 — Try the existing collector, read the reason code

`cc hub <platform>-adb-sync --json` (weibo/douyin/kuaishou/toutiao/xhs/bilibili exist):

- `*_COOKIES_INCOMPLETE` / `*_NO_WEBVIEW_COOKIES` → not logged into the in-app WebView.
- `*_IM_DB_ENCRYPTED` → SQLCipher; needs frida key-hook (hard — separate infra).
- web `error_code` / HTTP 404 → endpoint/permission drift (Phase 4).

`cc hub stats --json` runs under the **global `cc`** (host-ABI native sqlite); the
workspace CLI under plain `node` usually can't load `bs3mc` (Phase 5).

## Phase 2 — Inspect the app's local DBs (highest ROI)

```bash
D=/data/data/<pkg>/databases
adb shell "su -c 'ls -S $D/'"                 # by size; ignore -wal/-shm/-journal
adb shell "su -c 'head -c 16 $D/<file>'" | od -A n -t x1
#   53514c697465... = "SQLite format 3" = PLAINTEXT;  else SQLCipher/other
```

Pull plaintext DBs (base64 stream — MIUI-safe) and inspect with Node's built-in
`node:sqlite` (no native dep, no device):

```bash
adb shell "su -c 'base64 $D/<file>'" | tr -d '\r\n' | base64 -d > "$TEMP/x.db"
node --experimental-sqlite -e '…'   # ⚠️ Node-on-Windows /tmp = C:\tmp; use process.env.TEMP
```

Dump `sqlite_master` tables + counts + `PRAGMA table_info`, then sample rows. **The uid is
usually in a filename or table name** (`<uid>_im.db`, `record_<uid>`) and/or a plaintext
`account_db.login_info` row — that's how you recover the uid the web API won't give.

## Phase 3 — Build the collector (mirror the established pattern)

References: `social-toutiao-adb/account-reader.js`, `social-douyin-adb/watch-history-reader.js`.
A reader module exports:

- `pull<Db>ViaSu(adb, serial, opts)` — `ls … || echo NOT_FOUND` (on NOT_FOUND run
  `pm list packages <pkg>` to distinguish "not installed" from "installed-but-no-data"),
  `id -u` root check, `base64 <path> | tr -d '\n\r'` stream, validate `SQLite format 3`
  magic, write tmp file.
- `read<Data>(dbPath, opts)` — open via
  `require("../social-bilibili-adb/chromium-cookies-reader")._internals.loadDatabaseClass()`
  (dual-load: bs3mc under Electron / better-sqlite3 under test), `{readonly:true}`.
- `create<X>Extension(factoryOpts)` — bridge handler `(params, ctx)=>{…}`, requiring
  `ctx.adb` + `ctx.pickDevice`.

Normalize into the **existing** adapter's snapshot kinds (no second adapter) — e.g. douyin
watch history reuses `social-douyin`'s `KIND_HISTORY`. Then wire the extension in **BOTH**
`packages/cli/src/lib/personal-data-hub-wiring.js` AND
`desktop-app-vue/src/main/personal-data-hub/wiring.js` (the `extensions:{…}` object), add a
`hub.<x>Sync` facade method, and a `cc hub <x>-sync` command in `packages/cli/src/commands/hub.js`.

## Phase 4 — Only if the data isn't local (web API)

Pull the real cookie/credential off the device first (local DBs give the uid). Capture the
**actual** response before coding — endpoints drift. ByteDance passport uses `{message,data}`
envelopes with `err_no`/`error_code` (NOT `code`/`status_code`) and returns HTTP 200 +
`data:[]` on error — surface `err_no != 0` or it masks as "empty". Signed endpoints
(X-Bogus / `_signature` / X-S / WBI / DS / weapi) only work via the desktop SignBridge — the
CLI short-circuits them.

## Phase 5 — Verify on the real device (Electron harness)

The native sqlite reader (`bs3mc`) and the SignBridge (`WebContentsView`) load only under
**Electron**, not plain `node`. Write a throwaway `_harness.cjs` and run
`desktop-app-vue/node_modules/.bin/electron _harness.cjs`:

```js
const { app } = require("electron");
app.disableHardwareAcceleration();
app.commandLine.appendSwitch("no-sandbox");
app.whenReady().then(async () => {
  // require reader+collector by ABSOLUTE path; build a bridge whose invoke()
  // routes to createXExtension({})(params,{adb,pickDevice}); call collect…(bridge,{…});
  // console.log the result; app.exit(0);
});
```

Verifies the whole pipeline (pull → read → normalize, + signing) without driving the GUI.
**Delete the harness + any pulled DBs after; never commit a harness or device data.**

## Testing + landing

Inject `adb` (fake canned `su` output) and `_databaseClass` (fake Database with
`prepare().all()/.get()`) — no device, no native driver. Cover: pull diagnosis branches
(NOT_FOUND→installed/not, NO_ROOT, NOT_SQLITE, EMPTY), reader row parsing, and a
collect→normalize round-trip asserting a valid `partitionBatch`. Run the full pdh suite for
regressions.

**Release gate:** any `pdh/lib` change needs pdh version bump + `npm publish` + Android
`USR_VERSION` bump before it reaches the published CLI / Android (traps #27/#28). In the
desktop dev app (`npm run dev`, symlinks workspace pdh) it's live immediately.

## Git hygiene

Parallel Claude sessions share one `.git`. Commit with `git commit --only -- <paths>` (stage
new files with `git add -- <path>` first). The post-commit hook auto-pushes to github + gitee.