Bio Applied Deep Learning For Biology

Deep Learning for Biology

When to Use

• Classifying raw DNA/protein sequences (promoter vs non-promoter, TF binding site prediction) where hand-crafted k-mer features would lose information a CNN/Transformer can learn directly.
• Deciding between classical ML (random forest, SVM, logistic regression) and a neural network for a given biological dataset and sample size.
• Denoising or compressing high-dimensional gene expression / single-cell data with a variational autoencoder (VAE).
• Building the standard PyTorch train/eval loop for any tabular-, sequence-, or image-shaped biological input.
• Interpreting a trained model — extracting learned CNN filters as motifs, or computing saliency maps to see which positions drove a prediction.

Version Compatibility

• PyTorch >= 2.2 (CPU or CUDA build), Python >= 3.10
• scikit-learn >= 1.3 (train_test_split, StandardScaler), NumPy >= 1.24, pandas >= 2.0

Prerequisites

• pip install torch numpy pandas scikit-learn matplotlib (CPU wheel: --index-url https://download.pytorch.org/whl/cpu)
• Familiarity with classical ML workflow (see bio-applied-machine-learning-for-biology) — this skill assumes you already know when feature engineering + RF/SVM is the right call and are past that point.

Classical ML vs Deep Learning Decision Table

Use classical ML when: tabular features, small dataset, interpretability required. Use DL when: raw sequence/image input, large dataset, hierarchical patterns (motifs within motifs), or a relevant pre-trained model exists for transfer learning.

Core Training Loop and Autograd

Goal: build, train, and evaluate a feedforward classifier — the pattern every architecture below reuses. Approach: define the model as an nn.Module, then run the standard 5-step loop (zero_grad → forward → loss → backward → step) inside model.train(), and switch to model.eval() + torch.no_grad() for evaluation.

import numpy as np
import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import DataLoader, TensorDataset

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')


class SimpleClassifier(nn.Module):
    """Feedforward network for binary classification from tabular features."""

    def __init__(self, input_dim, hidden_dim=64):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(input_dim, hidden_dim),
            nn.ReLU(),
            nn.Dropout(0.3),
            nn.Linear(hidden_dim, hidden_dim // 2),
            nn.ReLU(),
            nn.Linear(hidden_dim // 2, 1),
        )  # returns logits; pair with BCEWithLogitsLoss

    def forward(self, x):
        return self.net(x)


def train_classifier(X_train, y_train, input_dim, epochs=100, lr=1e-3, batch_size=32):
    """Standard PyTorch training loop. X_train/y_train are numpy arrays."""
    X_t = torch.FloatTensor(X_train)
    y_t = torch.FloatTensor(y_train).unsqueeze(1)
    loader = DataLoader(TensorDataset(X_t, y_t), batch_size=batch_size, shuffle=True)

    model = SimpleClassifier(input_dim).to(device)
    criterion = nn.BCEWithLogitsLoss()  # numerically stable: sigmoid + BCE fused
    optimizer = optim.Adam(model.parameters(), lr=lr)

    losses = []
    for epoch in range(epochs):
        model.train()
        epoch_loss = 0.0
        for X_batch, y_batch in loader:
            X_batch, y_batch = X_batch.to(device), y_batch.to(device)
            optimizer.zero_grad()              # 1. reset gradients
            logits = model(X_batch)            # 2. forward pass
            loss = criterion(logits, y_batch)  # 3. compute loss
            loss.backward()                    # 4. backprop
            optimizer.step()                   # 5. update weights
            epoch_loss += loss.item() * X_batch.size(0)
        losses.append(epoch_loss / len(loader.dataset))
    return model, losses


@torch.no_grad()
def evaluate(model, X_test, y_test):
    """Accuracy on held-out data; note torch.no_grad() + eval() disable dropout/grad tracking."""
    model.eval()
    logits = model(torch.FloatTensor(X_test).to(device))
    preds = (torch.sigmoid(logits) > 0.5).float().cpu().squeeze(1)
    return (preds == torch.FloatTensor(y_test)).float().mean().item()

One-Hot Encoding and 1D CNN for Motif Detection

Goal: classify raw DNA sequences (e.g., promoter vs non-promoter) without hand-crafted k-mer features. Approach: one-hot encode to (N, 4, L) (Conv1d expects (batch, channels, length)), then stack Conv1d → ReLU → MaxPool blocks so filters learn motifs and pooling gives position invariance.

def one_hot_encode(sequences, alphabet='ACGT'):
    """One-hot encode DNA/RNA sequences to shape (N, C, L) for Conv1d."""
    mapping = {c: i for i, c in enumerate(alphabet)}
    n, seq_len = len(sequences), len(sequences[0])
    encoded = np.zeros((n, len(alphabet), seq_len), dtype=np.float32)
    for i, seq in enumerate(sequences):
        for j, char in enumerate(seq[:seq_len]):
            if char in mapping:
                encoded[i, mapping[char], j] = 1.0
    return torch.FloatTensor(encoded)


class SequenceCNN(nn.Module):
    """1D CNN: Conv(detect motifs) -> ReLU -> MaxPool(position invariance) -> Dense."""

    def __init__(self, seq_length=200):
        super().__init__()
        self.conv_layers = nn.Sequential(
            nn.Conv1d(4, 32, kernel_size=8, padding=3),
            nn.ReLU(),
            nn.MaxPool1d(4),
            nn.Conv1d(32, 64, kernel_size=6, padding=2),
            nn.ReLU(),
            nn.AdaptiveAvgPool1d(1),  # global pooling -> fixed-size vector regardless of L
        )
        self.classifier = nn.Sequential(
            nn.Flatten(),
            nn.Linear(64, 64),
            nn.ReLU(),
            nn.Linear(64, 1),
        )  # logits; use BCEWithLogitsLoss

    def forward(self, x):
        return self.classifier(self.conv_layers(x))


def extract_motif_filters(model):
    """Pull first-layer Conv1d weights as PWM-like motifs, shape (n_filters, 4, kernel_size)."""
    return model.conv_layers[0].weight.data.cpu().numpy()


def compute_saliency(model, x):
    """Input-gradient saliency: which sequence positions most influenced the prediction."""
    x = x.unsqueeze(0).clone().requires_grad_(True)
    model.eval()
    output = model(x.to(device))
    output.backward()
    return x.grad.data.abs().squeeze(0).sum(dim=0).cpu().numpy()  # sum abs grad over ACGT channels

VAE for Gene Expression Denoising

Goal: learn a low-dimensional latent space that separates cell types despite dropout noise in scRNA-seq-like data. Approach: encoder outputs mean/log-variance of a Gaussian latent, reparameterization trick samples z, decoder reconstructs input; loss = reconstruction + beta-weighted KL divergence.

class VAE(nn.Module):
    """Variational autoencoder for high-dimensional expression data."""

    def __init__(self, input_dim, hidden_dim=128, latent_dim=10):
        super().__init__()
        self.encoder = nn.Sequential(
            nn.Linear(input_dim, hidden_dim), nn.ReLU(),
            nn.Linear(hidden_dim, hidden_dim // 2), nn.ReLU(),
        )
        self.fc_mu = nn.Linear(hidden_dim // 2, latent_dim)
        self.fc_logvar = nn.Linear(hidden_dim // 2, latent_dim)
        self.decoder = nn.Sequential(
            nn.Linear(latent_dim, hidden_dim // 2), nn.ReLU(),
            nn.Linear(hidden_dim // 2, hidden_dim), nn.ReLU(),
            nn.Linear(hidden_dim, input_dim),
        )

    def encode(self, x):
        h = self.encoder(x)
        return self.fc_mu(h), self.fc_logvar(h)

    def reparameterize(self, mu, logvar):
        std = torch.exp(0.5 * logvar)
        return mu + std * torch.randn_like(std)  # reparam trick: keeps sampling differentiable

    def forward(self, x):
        mu, logvar = self.encode(x)
        z = self.reparameterize(mu, logvar)
        return self.decoder(z), mu, logvar


def vae_loss(recon, x, mu, logvar, beta=0.5):
    """Reconstruction (MSE) + beta * KL divergence to standard normal prior."""
    recon_loss = nn.functional.mse_loss(recon, x, reduction='sum')
    kl_div = -0.5 * torch.sum(1 + logvar - mu.pow(2) - logvar.exp())
    return recon_loss + beta * kl_div

Pitfalls

• model.eval() + torch.no_grad(): both required for inference — eval() disables dropout/batchnorm; no_grad() prevents gradient tracking (memory/speed).
• optimizer.zero_grad() before backward: forgetting this accumulates gradients across batches and produces wrong updates.
• Conv1d input shape: expects (batch, channels, length); one-hot DNA is (N, 4, L) — transpose from (N, L, 4) if your encoder produced the other layout.
• Loss/output layer mismatch: prefer BCEWithLogitsLoss on raw logits over Sigmoid() + BCELoss — the fused version is numerically stable near 0/1. Use CrossEntropyLoss (not Sigmoid+BCELoss) for multiclass.
• Class imbalance: use pos_weight in BCEWithLogitsLoss or a weighted sampler — biological datasets are often highly imbalanced (e.g., 1% binding sites vs background).
• Data leakage: split train/test BEFORE any scaling; fit StandardScaler on train only, then .transform() (not .fit_transform()) on test.
• Overfitting on small biological datasets: dropout, early stopping, weight decay, and data augmentation (e.g., reverse-complement for DNA) all help when samples are in the hundreds.
• GPU/CPU device mismatch: move model AND every tensor to the same device; a cryptic RuntimeError about tensors on different devices means you forgot a .to(device).

See Also

• bio-applied-machine-learning-for-biology — classical ML baseline (RF/SVM, feature engineering) to compare against before reaching for DL.
• bio-applied-structural-methods / structural-bioinformatics — for transformer-based structure prediction (AlphaFold-style) rather than sequence classification.
• bio-applied-single-cell-scanpy — upstream QC/normalization for the expression data fed into the VAE example.