# Bio Applied Deep Learning For Biology

> Train PyTorch CNN/LSTM/Transformer/VAE on DNA/protein sequences: one-hot encoding, motif filters, saliency. Use when classifying sequences, predicting TF binding sites, denoising scRNA-seq, or choosing DL vs ML.

- Skill: `pavel-kravchenko/bio-applied-deep-learning-for-biology` (Agent Skill)
- Install (CLI): `npx skillmds@latest add pavel-kravchenko/bio-applied-deep-learning-for-biology`
- Raw SKILL.md: https://api.skillmd.com/api/skills/pavel-kravchenko/bio-applied-deep-learning-for-biology/raw
- Safety review: WARNING
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: pavel-kravchenko (https://skillmd.com/u/pavel-kravchenko)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/pavel-kravchenko/bio-applied-deep-learning-for-biology

---


# Deep Learning for Biology

## When to Use

- Classifying raw DNA/protein sequences (promoter vs non-promoter, TF binding site prediction) where hand-crafted k-mer features would lose information a CNN/Transformer can learn directly.
- Deciding between classical ML (random forest, SVM, logistic regression) and a neural network for a given biological dataset and sample size.
- Denoising or compressing high-dimensional gene expression / single-cell data with a variational autoencoder (VAE).
- Building the standard PyTorch train/eval loop for any tabular-, sequence-, or image-shaped biological input.
- Interpreting a trained model — extracting learned CNN filters as motifs, or computing saliency maps to see which positions drove a prediction.

## Version Compatibility

- PyTorch >= 2.2 (CPU or CUDA build), Python >= 3.10
- scikit-learn >= 1.3 (train_test_split, StandardScaler), NumPy >= 1.24, pandas >= 2.0

## Prerequisites

- `pip install torch numpy pandas scikit-learn matplotlib` (CPU wheel: `--index-url https://download.pytorch.org/whl/cpu`)
- Familiarity with classical ML workflow (see `bio-applied-machine-learning-for-biology`) — this skill assumes you already know when feature engineering + RF/SVM is the right call and are past that point.

## Classical ML vs Deep Learning Decision Table

| Criterion | Classical ML | Deep Learning |
|-----------|-------------|---------------|
| Sample size | 100s-1000s | 10,000+ (or transfer learning) |
| Feature engineering | Manual (k-mers, physicochemical) | Learned automatically |
| Input type | Tabular features | Raw sequences / images |
| Interpretability | High (feature importance) | Lower (needs SHAP/saliency/attention) |
| Training time | Minutes | Hours-days |
| Hardware | CPU | GPU recommended |

Use classical ML when: tabular features, small dataset, interpretability required.
Use DL when: raw sequence/image input, large dataset, hierarchical patterns (motifs within motifs), or a relevant pre-trained model exists for transfer learning.

## Core Training Loop and Autograd

**Goal:** build, train, and evaluate a feedforward classifier — the pattern every architecture below reuses.
**Approach:** define the model as an `nn.Module`, then run the standard 5-step loop (zero_grad → forward → loss → backward → step) inside `model.train()`, and switch to `model.eval()` + `torch.no_grad()` for evaluation.

```python
import numpy as np
import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import DataLoader, TensorDataset

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')


class SimpleClassifier(nn.Module):
    """Feedforward network for binary classification from tabular features."""

    def __init__(self, input_dim, hidden_dim=64):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(input_dim, hidden_dim),
            nn.ReLU(),
            nn.Dropout(0.3),
            nn.Linear(hidden_dim, hidden_dim // 2),
            nn.ReLU(),
            nn.Linear(hidden_dim // 2, 1),
        )  # returns logits; pair with BCEWithLogitsLoss

    def forward(self, x):
        return self.net(x)


def train_classifier(X_train, y_train, input_dim, epochs=100, lr=1e-3, batch_size=32):
    """Standard PyTorch training loop. X_train/y_train are numpy arrays."""
    X_t = torch.FloatTensor(X_train)
    y_t = torch.FloatTensor(y_train).unsqueeze(1)
    loader = DataLoader(TensorDataset(X_t, y_t), batch_size=batch_size, shuffle=True)

    model = SimpleClassifier(input_dim).to(device)
    criterion = nn.BCEWithLogitsLoss()  # numerically stable: sigmoid + BCE fused
    optimizer = optim.Adam(model.parameters(), lr=lr)

    losses = []
    for epoch in range(epochs):
        model.train()
        epoch_loss = 0.0
        for X_batch, y_batch in loader:
            X_batch, y_batch = X_batch.to(device), y_batch.to(device)
            optimizer.zero_grad()              # 1. reset gradients
            logits = model(X_batch)            # 2. forward pass
            loss = criterion(logits, y_batch)  # 3. compute loss
            loss.backward()                    # 4. backprop
            optimizer.step()                   # 5. update weights
            epoch_loss += loss.item() * X_batch.size(0)
        losses.append(epoch_loss / len(loader.dataset))
    return model, losses


@torch.no_grad()
def evaluate(model, X_test, y_test):
    """Accuracy on held-out data; note torch.no_grad() + eval() disable dropout/grad tracking."""
    model.eval()
    logits = model(torch.FloatTensor(X_test).to(device))
    preds = (torch.sigmoid(logits) > 0.5).float().cpu().squeeze(1)
    return (preds == torch.FloatTensor(y_test)).float().mean().item()
```

## One-Hot Encoding and 1D CNN for Motif Detection

**Goal:** classify raw DNA sequences (e.g., promoter vs non-promoter) without hand-crafted k-mer features.
**Approach:** one-hot encode to `(N, 4, L)` (Conv1d expects `(batch, channels, length)`), then stack Conv1d → ReLU → MaxPool blocks so filters learn motifs and pooling gives position invariance.

```python
def one_hot_encode(sequences, alphabet='ACGT'):
    """One-hot encode DNA/RNA sequences to shape (N, C, L) for Conv1d."""
    mapping = {c: i for i, c in enumerate(alphabet)}
    n, seq_len = len(sequences), len(sequences[0])
    encoded = np.zeros((n, len(alphabet), seq_len), dtype=np.float32)
    for i, seq in enumerate(sequences):
        for j, char in enumerate(seq[:seq_len]):
            if char in mapping:
                encoded[i, mapping[char], j] = 1.0
    return torch.FloatTensor(encoded)


class SequenceCNN(nn.Module):
    """1D CNN: Conv(detect motifs) -> ReLU -> MaxPool(position invariance) -> Dense."""

    def __init__(self, seq_length=200):
        super().__init__()
        self.conv_layers = nn.Sequential(
            nn.Conv1d(4, 32, kernel_size=8, padding=3),
            nn.ReLU(),
            nn.MaxPool1d(4),
            nn.Conv1d(32, 64, kernel_size=6, padding=2),
            nn.ReLU(),
            nn.AdaptiveAvgPool1d(1),  # global pooling -> fixed-size vector regardless of L
        )
        self.classifier = nn.Sequential(
            nn.Flatten(),
            nn.Linear(64, 64),
            nn.ReLU(),
            nn.Linear(64, 1),
        )  # logits; use BCEWithLogitsLoss

    def forward(self, x):
        return self.classifier(self.conv_layers(x))


def extract_motif_filters(model):
    """Pull first-layer Conv1d weights as PWM-like motifs, shape (n_filters, 4, kernel_size)."""
    return model.conv_layers[0].weight.data.cpu().numpy()


def compute_saliency(model, x):
    """Input-gradient saliency: which sequence positions most influenced the prediction."""
    x = x.unsqueeze(0).clone().requires_grad_(True)
    model.eval()
    output = model(x.to(device))
    output.backward()
    return x.grad.data.abs().squeeze(0).sum(dim=0).cpu().numpy()  # sum abs grad over ACGT channels
```

## VAE for Gene Expression Denoising

**Goal:** learn a low-dimensional latent space that separates cell types despite dropout noise in scRNA-seq-like data.
**Approach:** encoder outputs mean/log-variance of a Gaussian latent, reparameterization trick samples `z`, decoder reconstructs input; loss = reconstruction + beta-weighted KL divergence.

```python
class VAE(nn.Module):
    """Variational autoencoder for high-dimensional expression data."""

    def __init__(self, input_dim, hidden_dim=128, latent_dim=10):
        super().__init__()
        self.encoder = nn.Sequential(
            nn.Linear(input_dim, hidden_dim), nn.ReLU(),
            nn.Linear(hidden_dim, hidden_dim // 2), nn.ReLU(),
        )
        self.fc_mu = nn.Linear(hidden_dim // 2, latent_dim)
        self.fc_logvar = nn.Linear(hidden_dim // 2, latent_dim)
        self.decoder = nn.Sequential(
            nn.Linear(latent_dim, hidden_dim // 2), nn.ReLU(),
            nn.Linear(hidden_dim // 2, hidden_dim), nn.ReLU(),
            nn.Linear(hidden_dim, input_dim),
        )

    def encode(self, x):
        h = self.encoder(x)
        return self.fc_mu(h), self.fc_logvar(h)

    def reparameterize(self, mu, logvar):
        std = torch.exp(0.5 * logvar)
        return mu + std * torch.randn_like(std)  # reparam trick: keeps sampling differentiable

    def forward(self, x):
        mu, logvar = self.encode(x)
        z = self.reparameterize(mu, logvar)
        return self.decoder(z), mu, logvar


def vae_loss(recon, x, mu, logvar, beta=0.5):
    """Reconstruction (MSE) + beta * KL divergence to standard normal prior."""
    recon_loss = nn.functional.mse_loss(recon, x, reduction='sum')
    kl_div = -0.5 * torch.sum(1 + logvar - mu.pow(2) - logvar.exp())
    return recon_loss + beta * kl_div
```

## Pitfalls

- **`model.eval()` + `torch.no_grad()`**: both required for inference — `eval()` disables dropout/batchnorm; `no_grad()` prevents gradient tracking (memory/speed).
- **`optimizer.zero_grad()` before backward**: forgetting this accumulates gradients across batches and produces wrong updates.
- **Conv1d input shape**: expects `(batch, channels, length)`; one-hot DNA is `(N, 4, L)` — transpose from `(N, L, 4)` if your encoder produced the other layout.
- **Loss/output layer mismatch**: prefer `BCEWithLogitsLoss` on raw logits over `Sigmoid()` + `BCELoss` — the fused version is numerically stable near 0/1. Use `CrossEntropyLoss` (not `Sigmoid`+`BCELoss`) for multiclass.
- **Class imbalance**: use `pos_weight` in `BCEWithLogitsLoss` or a weighted sampler — biological datasets are often highly imbalanced (e.g., 1% binding sites vs background).
- **Data leakage**: split train/test BEFORE any scaling; fit `StandardScaler` on train only, then `.transform()` (not `.fit_transform()`) on test.
- **Overfitting on small biological datasets**: dropout, early stopping, weight decay, and data augmentation (e.g., reverse-complement for DNA) all help when samples are in the hundreds.
- **GPU/CPU device mismatch**: move model AND every tensor to the same `device`; a cryptic `RuntimeError` about tensors on different devices means you forgot a `.to(device)`.

## See Also

- `bio-applied-machine-learning-for-biology` — classical ML baseline (RF/SVM, feature engineering) to compare against before reaching for DL.
- `bio-applied-structural-methods` / `structural-bioinformatics` — for transformer-based structure prediction (AlphaFold-style) rather than sequence classification.
- `bio-applied-single-cell-scanpy` — upstream QC/normalization for the expression data fed into the VAE example.