Model Merging Dual Anchors

Model Merging via Functional Dual Anchors: Beyond Parameter Space

Parameter-space merging methods suffer from conflicts when task vectors diverge. Functional Dual Anchors (FDAs) capture task shifts in representation space through synthetic inputs, enabling more robust model combination.

FDAs bridge multi-task training and post-hoc merging, offering flexibility to merge models trained independently while maintaining task-specific capabilities.

Core Concept

Rather than averaging or interpolating model weights directly, FDAs operate by:

• Creating synthetic inputs in input-representation space
• Designing these inputs so their gradients match task-specific vectors
• Using gradient alignment to represent how models diverge from base
• Merging in this aligned gradient space rather than parameter space

This approach mitigates parameter inconsistencies that plague traditional merging.

Architecture Overview

• Synthetic input generation that produces task-aligned gradients
• Principled initialization scheme for anchor points
• Gradient-based representation of task shifts
• Complementary to parameter-space merging methods

Implementation Steps

Define synthetic inputs as learnable parameters whose gradients capture task shifts. Rather than working with weight parameters directly, optimize inputs to produce task-relevant gradients:

class FunctionalDualAnchors:
    def __init__(self, base_model, num_tasks, input_dim=768):
        self.base_model = base_model
        self.num_tasks = num_tasks
        # Create synthetic inputs per task
        self.synthetic_inputs = nn.ParameterList([
            nn.Parameter(torch.randn(1, input_dim) * 0.01)
            for _ in range(num_tasks)
        ])

    def compute_task_gradient(self, task_id, target_vector):
        """Compute gradient of synthetic input to match task vector."""
        synthetic = self.synthetic_inputs[task_id]
        synthetic.requires_grad_(True)

        # Forward through model
        output = self.base_model(synthetic)

        # Loss: align output gradients with task vector
        loss = -torch.dot(output.squeeze(), target_vector)
        loss.backward()

        return synthetic.grad

Compute task vectors from fine-tuned models as the difference from base model. This captures what each fine-tuning task adds:

def compute_task_vector(fine_tuned_model, base_model):
    """Extract task vector as parameter difference."""
    task_vector = {}
    for (name_ft, param_ft), (name_base, param_base) in zip(
        fine_tuned_model.named_parameters(),
        base_model.named_parameters()
    ):
        task_vector[name_ft] = param_ft - param_base

    return task_vector

Merge models by aligning synthetic inputs with computed task vectors. The FDA bridges training and inference by providing a unified representation:

def merge_with_functional_anchors(base_model, fine_tuned_models, task_vectors):
    """Merge multiple models using functional dual anchors."""
    anchors = FunctionalDualAnchors(base_model, len(fine_tuned_models))

    # Align synthetic inputs to task vectors
    for task_id, task_vec in enumerate(task_vectors):
        # Iterative alignment
        for _ in range(10):  # Alignment iterations
            grad = anchors.compute_task_gradient(task_id, task_vec)
            # Gradient descent step
            anchors.synthetic_inputs[task_id].data -= 0.01 * grad

    # Merge: Average task shifts across all tasks
    merged_params = copy_params(base_model)
    for name, param in merged_params.items():
        shifts = [
            task_vectors[i].get(name, torch.zeros_like(param))
            for i in range(len(task_vectors))
        ]
        # Average task shifts
        merged_params[name] += torch.stack(shifts).mean(dim=0)

    return merged_params

Practical Guidance

When to use:

• Merging independently fine-tuned models without retraining
• Multi-task scenarios where parameter conflicts arise
• When post-hoc merging is required (models already trained)
• Combining domain-specific specialists with foundation models

When NOT to use:

• Single-task scenarios (no merging needed)
• Training from scratch (joint training is simpler)
• Extremely heterogeneous models with little overlap
• When parameter-space merging works well (simpler and faster)

Common pitfalls:

• Misaligned initialization of synthetic inputs (poor convergence)
• Insufficient alignment iterations (incomplete task capture)
• Using incompatible base models across tasks
• Not validating merged model on task distribution

Reference: Model Merging with Functional Dual Anchors on arXiv