# Model Architect

> Design transformer architectures by target scale and emit Hugging Face Transformers-compatible model configs. Use when this capability is needed.

- Skill: `tomevault-io/model-architect` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add tomevault-io/model-architect`
- Raw SKILL.md: https://api.skillmd.com/api/skills/tomevault-io/model-architect/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: tomevault-io (https://skillmd.com/u/tomevault-io)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/tomevault-io/model-architect

---


# Model Architect (From Scratch)

Use this skill to design a model architecture for **pretraining from scratch**.
It should output practical configs that match compute, memory, and quality goals.

## Scale Templates

Provide architecture templates by parameter scale:

- **100M class**
- **300M–700M class**
- **1B–3B class**
- **7B+ class**

For each template, define reasonable defaults for:

- hidden size
- number of layers
- number of attention heads
- intermediate (FFN) size
- context length

## Component Selections

Use modern decoder-only transformer defaults unless user requests otherwise:

- **GQA** (Grouped Query Attention) for memory/throughput efficiency at scale.
- **SwiGLU** feed-forward activation.
- **RoPE** positional encoding.
- **RMSNorm** normalization.

When deviating from these defaults, explain why and expected tradeoffs.

## Config Output

Emit a Hugging Face Transformers-compatible `config.json` suitable for training scripts.
Include all required architecture fields and tokenizer special token ids.

## Parameter + Memory Calculator

Always provide estimates for:

- total parameter count (with per-block breakdown)
- optimizer state memory
- activation memory (approximate by batch/seq settings)
- checkpoint size (fp16/bf16 and optional fp32)

Report both training-time and inference-time memory expectations.

## Tokenizer Integration

Before generating `config.json`, import outputs from the **tokenizer-trainer** skill:

- Read `vocab_size` from trained tokenizer and set in config.
- Import `bos_token_id`, `eos_token_id`, `pad_token_id`, `unk_token_id` from `special_tokens_map.json`.
- Validate that `max_position_embeddings` aligns with tokenizer sequence length analysis.
- Fail explicitly if tokenizer artifacts are missing — never use placeholder token IDs.

## Decision Protocol

- Start with target quality, latency, and budget constraints.
- Produce at least two candidate designs when tradeoffs are non-trivial.
- Recommend one primary architecture with rationale.
- Highlight scaling path (for example 100M -> 1B -> 7B) for future expansion.

## Deliverables

1. `config.json` (HF-compatible)
2. `architecture_report.md` (tradeoffs + estimates)
3. `capacity_plan.md` (GPU/memory implications)

---
> Converted and distributed by [TomeVault](https://tomevault.io/claim/rachasumanth) — claim your Tome and manage your conversions.
<!-- tomevault:4.0:skill_md:2026-04-15 -->