# Add Model 05 Port Encoder

> Use during /add-model Phase 4 or Phase 6 to prototype or parity-debug one FastVideo-native text, image, audio, or compound encoder component.

- Skill: `hao-ai-lab/add-model-05-port-encoder` (Agent Skill)
- Install (CLI): `npx skillmds@latest add hao-ai-lab/add-model-05-port-encoder`
- Raw SKILL.md: https://api.skillmd.com/api/skills/hao-ai-lab/add-model-05-port-encoder/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Web & Frontend
- Author: hao-ai-lab (https://skillmd.com/u/hao-ai-lab)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/hao-ai-lab/add-model-05-port-encoder

---


# Add Model Port Encoder

## Goal

Prototype or parity-debug one encoder or encoder-like conditioner in
FastVideo-native code. Use this for text encoders, image encoders, audio
encoders, and compound conditioners that fit the encoder config/loader bucket.

## Inputs

Follow `../add-model/shared/component_skill_common.md` and require the complete
packet from `../add-model/contracts/component_context.md`.

Encoder-specific packet fields:

- `component`: encoder or encoder-like conditioner name.
- `parity_test`: `tests/local_tests/encoders/test_<family>_<component>_parity.py`.
- `weights`: converted encoder dir, HF subfolder, or external HF id.
- `target_files`: `fastvideo/models/encoders/<arch_or_family>.py` and
  `fastvideo/configs/models/encoders/<arch_or_family>.py`.

## Modes

Use the common prototype and parity-debug modes from
`../add-model/shared/component_skill_common.md`.

Encoder-specific prototype concerns include tokenizer kwargs, hidden-state
extraction, output packing, connector order, and external/passthrough weight
needs.

## Reuse Proof

Apply the shared reuse proof. Encoder-specific comparison must include tokenizer
contracts, hidden-state extraction, masks, positional IDs, output packing,
connector/projection ordering, passthrough paths, and returned dataclass shape.

## Existing FastVideo Patterns

- Base classes: `TextEncoder` and `ImageEncoder` in
  `fastvideo/models/encoders/base.py`.
- Output type: `BaseEncoderOutput`.
- Config bases: `TextEncoderConfig`, `ImageEncoderConfig`,
  `TextEncoderArchConfig`, and `ImageEncoderArchConfig` in
  `fastvideo/configs/models/encoders/base.py`.
- Use the matching encoder config bucket. Wrong bucket inheritance can typecheck
  but fail during pipeline wiring.
- Config export: add the config to
  `fastvideo/configs/models/encoders/__init__.py`.
- Registry discovery: set `EntryClass = <ClassName>` or a list of class names in
  the model file.
- Reference examples: native `t5.py`, `clip.py`, `siglip.py`, `llama.py`,
  `qwen2_5.py`, `gemma.py`, and compound `stable_audio_conditioner.py`.
- Layer guidance: `fastvideo/layers/AGENTS.md`.

## Implementation Rules

- Reuse tokenizers and pure data utilities when needed, but do not add runtime
  third-party model-class imports as a placeholder for a component that owns
  weights or numerical behavior.
- For LLM-style encoders, follow existing tensor-parallel patterns such as
  `QKVParallelLinear`, `MergedColumnParallelLinear`, `RowParallelLinear`,
  `VocabParallelEmbedding`, and `RMSNorm` when matching native examples.
- Match official hidden-state extraction exactly: layer index, pooled output,
  attention mask dtype, padding side, truncation, special tokens, final norm,
  output_hidden_states, and returned tuple/dataclass shape.
- For connector or conditioner modules, preserve sub-conditioner order and the
  exact packing of cross-attention tokens, masks, and global conditioning.
- Put tokenizer kwargs and architecture constants on the arch config when they
  affect numerical behavior.
- If an external HF encoder is explicitly accepted as a lazy wrapper, keep it
  isolated, document why it is not a native port, and still require parity for
  the wrapper's output contract.

Hybrid external-HF encoder checklist:

- Put external model folders in passthrough subfolders such as
  `text_encoder/<external_name>/`, or record a root `model_index.json` path field
  that the loader resolves to a local directory.
- Keep external model parameters out of the FastVideo-owned state-dict surface
  when the external model is loaded lazily from its own HF files.
- Convert and strict-check only the FastVideo-owned connector/projection weights;
  document external model weights as passthrough.
- Add parity for the wrapper's final output contract and, when useful, a narrower
  connector-only parity test that labels its scope as
  `implementation_subcomponent`.
- Verify the production loader resolves the same external path used by the
  pipeline, not just the direct class used in the parity test.

## Prototype Checks

Follow the shared prototype success criteria.

## Parity-Debug Loop

Run the shared parity-debug loop. The component test command is:

```bash
pytest <parity_test> -v -s
```

For numerical drift, check tokenization, masks, hidden-state selection,
positional IDs, dtype/autocast, and output packing before changing layers.

## Escape Hatches

Follow `../add-model/shared/common_rules.md` and the component-specific guidance
in `../add-model/shared/component_skill_common.md`. Encoder-specific ask cases
include accepting private model-code execution, choosing between incompatible
tokenizer/encoder references, or dropping a required conditioning stream.

## Handoff

Return `../add-model/contracts/component_skill_handoff.md` following the common
handoff rules in `../add-model/shared/component_skill_common.md`.