# Versatileffn Adaptive Ffn

> Enable parameter-efficient computation through dual-pathway feed-forward networks. Create virtual experts via hidden subspace slicing (width-versatile) and recursive weight application (depth-versatile). Use difficulty-aware gating to balance pathways, adding computation not memory to fixed parameter budget.

- Skill: `adu2021/versatileffn-adaptive-ffn` (Agent Skill)
- Install (CLI): `npx skillmds@latest add adu2021/versatileffn-adaptive-ffn`
- Raw SKILL.md: https://api.skillmd.com/api/skills/adu2021/versatileffn-adaptive-ffn/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Finance & Business
- License: MIT
- Author: adu2021 (https://skillmd.com/u/adu2021)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/adu2021/versatileffn-adaptive-ffn

---


## Skill Summary

VersatileFFN introduces a parameter-efficient feed-forward architecture that reuses parameters across two complementary computational pathways: (1) width-versatile pathway creating virtual experts through hidden subspace slicing, and (2) depth-versatile pathway recursively applying FFN weights for iterative refinement. A difficulty-aware gating mechanism dynamically balances pathways, routing easy tokens through efficient width processing and allocating deeper computation to hard tokens—inspired by dual-process cognition theory.

## When To Use

- Building language models with flexible computation-parameter trade-offs
- Scenarios where parameter budgets are fixed but computational budgets are flexible
- Projects inspired by dual-process cognition models
- Research exploring efficient alternatives to standard mixture-of-experts

## When NOT To Use

- Latency-sensitive applications where variable-depth computation adds unpredictability
- Scenarios with strict computational budgets but flexible parameter budgets
- Domains already using standard FFNs that work well
- Applications requiring uniform computational cost across all tokens

## Core Technique

Two complementary computation pathways share parameters:

**1. Width-Versatile Pathway**
Create multiple virtual experts by slicing a shared base FFN into non-overlapping hidden subspaces, mimicking mixture-of-experts routing without increasing parameters. Route different tokens to different slices.

**2. Depth-Versatile Pathway**
Recursively apply the same FFN weights multiple times, allowing tokens to undergo iterative refinement with a token-specific iteration count predicted via Gumbel-Softmax. Easy tokens exit early; hard tokens receive multiple passes.

**3. Difficulty-Aware Fusion**
A gating mechanism dynamically balances the two pathways based on expected computational depth. Route "easy" tokens through the efficient width path and allocate deeper processing to "hard" tokens. As authors state: "both pathways reuse the same parameters, so all additional capacity comes from computation rather than memory."

## Implementation Notes

Start with standard FFN architecture. Split hidden state into multiple non-overlapping subspaces for width versatility. Implement recursive FFN application with token-specific iteration prediction. Design gating mechanism to balance pathways based on token difficulty. This approach inspired by dual-process cognition enables flexible computation allocation within fixed memory.

## References

- Original paper: VersatileFFN (Dec 2025)
- Mixture-of-experts architectures
- Dual-process cognition theory