# Sequence To Sequence Architecture Design

> Use when when you have encoder-produced fixed-size embeddings and need to generate variable-length discrete sequences (e.g., SMILES tokens, protein sequences, chemical formulas) as outputs.

- Skill: `holobiomicslab/sequence-to-sequence-architecture-design` (Agent Skill)
- Install (CLI): `npx skillmds@latest add holobiomicslab/sequence-to-sequence-architecture-design`
- Raw SKILL.md: https://api.skillmd.com/api/skills/holobiomicslab/sequence-to-sequence-architecture-design/raw
- Safety review: PASS (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- License: CC-BY-4.0
- Author: HolobiomicsLab (https://skillmd.com/u/holobiomicslab)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/holobiomicslab/sequence-to-sequence-architecture-design

---


# sequence-to-sequence-architecture-design

## Summary

Design and implement a sequence-to-sequence (seq2seq) model with attention that maps fixed-size embeddings to variable-length token sequences, such as reconstructing SMILES strings from molecular embeddings. This skill bridges continuous latent representations to discrete symbolic outputs using an encoder-decoder paradigm with teacher forcing during training.

## When to use

When you have encoder-produced fixed-size embeddings and need to generate variable-length discrete sequences (e.g., SMILES tokens, protein sequences, chemical formulas) as outputs. Specifically applicable when the input is a continuous embedding and the output is a sequence of categorical tokens with no predetermined length.

## When NOT to use

- Input is already a sequence (use sequence-to-sequence without an embedding bottleneck instead).
- Output is fixed-length or continuous-valued (use a fully connected regression head or classifier instead).
- Target sequences are very short (single tokens) or deterministic transformations (use a simple lookup table or rule-based decoder).

## Inputs

- encoder-produced embeddings (fixed-size continuous vectors)
- SMILES token vocabulary or domain-specific tokenizer
- target sequence dataset (e.g., canonical SMILES strings, protein sequences)

## Outputs

- trained decoder model weights
- decoded variable-length sequences (e.g., SMILES strings)
- validation metrics (exact-match accuracy, Tanimoto similarity, cross-entropy loss)

## How to apply

Define a sequence-to-sequence model in PyTorch with an embedding layer that accepts the fixed-size encoder output, followed by an attention-equipped recurrent decoder (e.g., LSTM or GRU) that predicts one token at a time. Prepare a tokenizer for your target domain (e.g., SMILES tokenizer via RDKit) and convert all target sequences to canonical representations. Train the decoder end-to-end using cross-entropy loss on token prediction with teacher forcing (feeding ground-truth tokens during training rather than model predictions). Evaluate using exact-match accuracy on held-out embeddings and domain-specific similarity metrics (e.g., Tanimoto similarity between decoded and reference molecules via RDKit). Log validation metrics and save trained decoder weights separately from the encoder for modularity.

## Related tools

- **PyTorch** (framework for implementing sequence-to-sequence model architecture with attention, cross-entropy loss computation, and backpropagation training)
- **RDKit** (tokenization of SMILES strings, canonicalization of molecular structures, and computation of Tanimoto similarity for evaluation) — https://www.rdkit.org/

## Examples

```
decoder = Seq2SeqDecoder(embedding_dim=256, vocab_size=120, hidden_dim=512, num_layers=2, attention=True); criterion = nn.CrossEntropyLoss(); optimizer = torch.optim.Adam(decoder.parameters(), lr=0.001); for epoch in range(num_epochs): loss = train_epoch(decoder, train_loader, criterion, optimizer, teacher_forcing_ratio=1.0); val_acc, val_tanimoto = evaluate(decoder, val_loader); print(f'Epoch {epoch}: train_loss={loss:.4f}, val_acc={val_acc:.4f}, val_tanimoto={val_tanimoto:.4f}')
```

## Evaluation signals

- Exact-match accuracy on held-out test embeddings (reconstructed SMILES == reference SMILES without RDKit canonicalization errors).
- Tanimoto similarity between decoded and reference molecules (via RDKit fingerprints; expect high similarity if decoding is faithful).
- Cross-entropy loss converging and validation loss decreasing over training epochs.
- All generated sequences are valid tokens according to the target tokenizer vocabulary.
- Decoder output sequences vary in length as expected (not all identical lengths) and respect chemical feasibility constraints when decoded to molecules.

## Limitations

- Teacher forcing during training can cause exposure bias: at inference, the decoder conditions on its own (potentially erroneous) predictions rather than ground truth, degrading sequence quality over long horizons.
- Attention mechanism assumes the embedding is sufficiently informative; if the encoder embedding loses critical structural information, the decoder cannot recover it.
- Exact-match accuracy is brittle for SMILES (same molecule can have multiple valid SMILES representations); Tanimoto similarity or other fuzzy metrics are more robust but slower to compute.
- Training requires paired encoder embeddings and target sequences; unpaired or weakly labeled data cannot be used directly.

## Evidence

- [other] Define the decoder architecture in PyTorch as a sequence-to-sequence model with attention, mapping fixed-size embeddings to variable-length SMILES token sequences.: "Define the decoder architecture in PyTorch as a sequence-to-sequence model with attention, mapping fixed-size embeddings to variable-length SMILES token sequences."
- [other] Train the decoder end-to-end using cross-entropy loss on SMILES token prediction, with teacher forcing during training.: "Train the decoder end-to-end using cross-entropy loss on SMILES token prediction, with teacher forcing during training."
- [other] Evaluate decoder on held-out test embeddings by measuring exact-match accuracy and Tanimoto similarity of reconstructed vs. reference molecules (decoded via RDKit).: "Evaluate decoder on held-out test embeddings by measuring exact-match accuracy and Tanimoto similarity of reconstructed vs. reference molecules (decoded via RDKit)."
- [readme] The decoder reconstructs the molecular structure, in a SMILES format, given the embedding that the encoder generates.: "The decoder reconstructs the molecular structure, in a SMILES format, given the embedding that the encoder generates."
- [readme] The implementation of the Spec2Mol architecture is based on the Pytorch library. Processing of the chemical data is based on the RDKit software.: "The implementation of the Spec2Mol architecture is based on the Pytorch library. Processing of the chemical data is based on the RDKit software."