Trigger
User asks to create fine-tuning data from PDF for thinkingmachines/Inkling using Tinker.
Requirements
- Zero-Binary / Zero-Batch enforcement: execute via
sys.executable, no compiled wrappers (see HermesAGENTS.md: plugins/skills extend at edges, core stays pure Python). - Tokenizer:
training_client.get_tokenizer()(provided by Tinker SDK; requires fulltinkerenv;tml_tokenizersis SDK-internal, not on PyPI). - Tinker SDK:
pip install tinker(v0.23.0 verified).types.Datum,types.ModelInput.from_ints,cross_entropyloss. - Tinker
base_modelIDs verified from docs (tinker/models/):thinkingmachines/Inkling(64K),thinkingmachines/Inkling:peft:262144(256K). SDK400withthinkingmachines/inklingindicates case-sensitive ID; use exact docs string.
Tinker Datum format (verified from tinker-docs quickstart)
datum = types.Datum(
model_input=types.ModelInput.from_ints(tokens=input_tokens),
loss_fn_inputs=dict(
weights=weights, # 0 = prompt, 1 = completion
target_tokens=target_tokens # shift by 1 from input
)
)
Used with training_client.forward_backward_async(data=[datum], loss_fn="cross_entropy") via create_lora_training_client.
Chunking strategy
- By headings when detectable; fall back to page-based with configurable overlap.
- Each chunk → instruction (heading/summary) + response (body text).
- Command/tool extraction: many books include terminal commands in images (screenshots), not text. For text commands, scan chunks for backticks /
$patterns. For image-embedded commands: extract images viafitzpage.get_images(), apply OCR (e.g.,tesseractor vision model) to recover terminal text. Extracted commands become supervised pairs (instruction = "How to use ?", response = usage + context).
Final Workflow (verified — generic, no hard-coded paths)
- Extract any PDF (
--pdf <path>) → filter → chunk - Generate SL pairs →
<output_path>(.jsonl, default:<pdf_basename>_training_full.jsonl) - Train:
scripts/train_lora.py(LoRA +cross_entropy) withcreate_lora_training_client - Format:
tinker_datum_supervised_lora(loss: cross_entropy,weights: ..., LoRA) - Image commands:
fitz.get_images()+ OCR
Verification
Run: python chunk_pdf.py --pdf <path>