adu2021 Persona Vectors: Monitoring and Controlling LLM Character Traits
Persona Vectors provide a novel approach to understanding and controlling language model behavior by identifying interpretable directions in activation space that correspond to specific personality traits. This technique enables both monitoring and controlled modification of model behavior at scale, addressing critical safety and alignment challenges.
Core Concept
The fundamental insight is that language models develop consistent personality traits (e.g., honesty, helpfulness, harmfulness) that manifest as coherent directions in their hidden activation space. By identifying these "persona vectors," practitioners can:
- Monitor personality drift during deployment by tracking projections onto trait vectors
- Predict behavioral shifts before they occur during training
- Apply steering interventions to prevent undesirable personality changes
- Flag problematic training data that induces unwanted trait shifts
- Control model behavior through post-hoc activation manipulation
Architecture Overview
The persona vector framework consists of:
- Trait Description Collection: Natural language descriptions of personality traits
- Activation Space Mapping: Extraction of hidden layer activations for trait-labeled examples
- Vector Identification: Automated method to isolate trait directions (PCA, contrastive learning)
- Monitoring System: Tracks projection magnitudes over time to detect drift
- Steering Mechanism: Applies activation interventions to shift model along trait vectors
- Data Curation Pipeline: Identifies problematic training examples based on trait shifts
Implementation Steps
Step 1: Collect trait descriptions and labeled examples
The first step creates a dataset of examples exhibiting different trait intensities:
from typing import List, Dict, Tuple
import numpy as np
class TraitDataset:
"""Collects examples of language model outputs along trait dimensions"""
def __init__(self):
self.traits: Dict[str, str] = {}
self.examples: Dict[str, List[Tuple[str, float]]] = {}
def add_trait(self, trait_name: str, description: str):
"""
Register a personality trait with natural language description.
Args:
trait_name: e.g., "harmfulness", "honesty", "sycophancy"
description: What this trait means and how it manifests
"""
self.traits[trait_name] = description
self.examples[trait_name] = []
def add_labeled_example(self, trait_name: str, text: str, intensity: float):
"""
Add an example with trait intensity score (0-1).
Args:
trait_name: Which trait this example demonstrates
text: Model output or example text
intensity: 0=trait absent, 1=extreme trait manifestation
"""
if trait_name not in self.traits:
raise ValueError(f"Unknown trait: {trait_name}")
self.examples[trait_name].append((text, intensity))
def create_contrastive_pairs(self, trait_name: str,
low_threshold: float = 0.3,
high_threshold: float = 0.7) -> List[Tuple[str, str]]:
"""
Create pairs of examples with low vs high trait intensity.
These pairs train the direction identification.
"""
examples = self.examples[trait_name]
low_intensity = [text for text, intensity in examples if intensity < low_threshold]
high_intensity = [text for text, intensity in examples if intensity > high_threshold]
# Pair each high-intensity example with a random low-intensity example
pairs = []
for high_text in high_intensity:
for low_text in low_intensity:
pairs.append((low_text, high_text))
return pairs
This creates the training data for identifying trait vectors from naturally labeled examples.
Step 2: Extract activations and identify trait vectors
Extract hidden layer activations and compute the direction that maximizes trait variation:
class PersonaVectorExtractor:
"""Identifies personality trait directions in activation space"""
def __init__(self, model, layer_name: str = "transformer.h.10"):
self.model = model
self.layer_name = layer_name # Which layer to extract from
self.hook_handle = None
def extract_activations(self, text: str, token_idx: int = -1) -> np.ndarray:
"""
Extract hidden activations from specified layer for given text.
Args:
text: Input text to process
token_idx: Which token position to extract (-1 for last token)
Returns:
Activation vector of shape (hidden_size,)
"""
activations = None
def hook_fn(module, input, output):
nonlocal activations
# output is (batch_size, seq_len, hidden_size)
activations = output[0, token_idx, :].detach().cpu().numpy()
# Register forward hook
for name, module in self.model.named_modules():
if name == self.layer_name:
self.hook_handle = module.register_forward_hook(hook_fn)
break
# Forward pass
with torch.no_grad():
self.model(text)
# Remove hook
if self.hook_handle:
self.hook_handle.remove()
return activations
def compute_trait_vector_pca(self, low_texts: List[str],
high_texts: List[str]) -> np.ndarray:
"""
Compute trait vector as primary direction of variation
between low and high trait intensity examples.
Uses PCA: fits PCA to centered difference vectors.
"""
low_activations = np.array([self.extract_activations(text) for text in low_texts])
high_activations = np.array([self.extract_activations(text) for text in high_texts])
# Center and compute mean difference
low_mean = low_activations.mean(axis=0)
high_mean = high_activations.mean(axis=0)
mean_diff = high_mean - low_mean
# Combine for PCA: all examples centered at origin
combined = np.vstack([
low_activations - low_mean,
high_activations - high_mean
])
# PCA: first component is trait direction
cov = combined.T @ combined
eigvals, eigvecs = np.linalg.eigh(cov)
# Largest eigenvalue corresponds to trait direction
trait_vector = eigvecs[:, -1]
# Ensure it points toward high intensity
if np.dot(trait_vector, mean_diff) < 0:
trait_vector = -trait_vector
return trait_vector
def compute_trait_vector_contrastive(self, low_texts: List[str],
high_texts: List[str]) -> np.ndarray:
"""
Alternative: compute trait vector as regression direction
that separates high from low intensity examples.
"""
low_acts = np.array([self.extract_activations(text) for text in low_texts])
high_acts = np.array([self.extract_activations(text) for text in high_texts])
# Simple approach: maximize difference magnitude using least squares
X = np.vstack([low_acts, high_acts])
y = np.hstack([np.zeros(len(low_acts)), np.ones(len(high_acts))])
# Solve: min ||Xw - y||^2
w = np.linalg.lstsq(X, y, rcond=None)[0]
w = w / np.linalg.norm(w) # Normalize
return w
These methods identify vectors pointing toward trait manifestation in activation space.
Step 3: Implement monitoring and drift detection
Track how trait vectors evolve during training or deployment:
class TraitMonitor:
"""Monitors personality trait drift in deployed models"""
def __init__(self, trait_vectors: Dict[str, np.ndarray],
baseline_projections: Dict[str, float]):
self.trait_vectors = trait_vectors
self.baseline_projections = baseline_projections
self.history: List[Dict[str, float]] = []
def measure_traits(self, text: str, extractor: PersonaVectorExtractor) -> Dict[str, float]:
"""
Measure trait intensities for given output.
Args:
text: Model output to evaluate
extractor: PersonaVectorExtractor instance
Returns:
Dict mapping trait names to projection magnitudes (0-1)
"""
activation = extractor.extract_activations(text)
projections = {}
for trait_name, trait_vector in self.trait_vectors.items():
# Project activation onto trait vector
projection = np.dot(activation, trait_vector)
# Normalize relative to baseline
baseline = self.baseline_projections.get(trait_name, 0.0)
normalized = (projection - baseline) / (np.linalg.norm(trait_vector) + 1e-8)
projections[trait_name] = normalized
return projections
def detect_drift(self, measurements: Dict[str, float],
threshold: float = 0.2) -> List[str]:
"""
Detect if traits have drifted significantly from baseline.
Returns:
List of traits that have drifted beyond threshold
"""
drifted_traits = []
for trait_name, projection in measurements.items():
baseline = self.baseline_projections.get(trait_name, 0.0)
drift_magnitude = abs(projection - baseline)
if drift_magnitude > threshold:
drifted_traits.append(trait_name)
return drifted_traits
def log_measurement(self, measurements: Dict[str, float],
timestamp: str = None):
"""Record trait measurements for trend analysis"""
entry = {"timestamp": timestamp or "now", **measurements}
self.history.append(entry)
This enables real-time monitoring of personality shifts.
Step 4: Implement steering interventions
Apply controlled activation manipulations to shift traits:
class PersonaSteering:
"""Applies activation interventions to control personality traits"""
def __init__(self, model, trait_vectors: Dict[str, np.ndarray],
layer_name: str = "transformer.h.10"):
self.model = model
self.trait_vectors = trait_vectors
self.layer_name = layer_name
self.steering_strength = {} # Map trait -> intervention strength
def set_steering_strength(self, trait_name: str, strength: float):
"""
Configure how strongly to steer toward/away from trait.
Args:
trait_name: Which trait to control
strength: Positive to increase trait, negative to decrease
"""
self.steering_strength[trait_name] = strength
def create_steering_hook(self):
"""
Create a forward hook that applies trait steering during inference.
"""
trait_vectors = self.trait_vectors
steering_strength = self.steering_strength
def hook_fn(module, input, output):
# output: (batch_size, seq_len, hidden_size)
modified_output = output.clone()
for i in range(output.size(0)): # For each example in batch
for j in range(output.size(1)): # For each token
activation = output[i, j, :]
# Apply steering for each trait
for trait_name, trait_vector in trait_vectors.items():
strength = steering_strength.get(trait_name, 0.0)
if abs(strength) > 1e-6:
# Shift activation along trait vector
trait_vec_tensor = torch.from_numpy(trait_vector).float()
shift = strength * trait_vec_tensor.to(activation.device)
modified_output[i, j, :] = activation + shift
return modified_output
return hook_fn
def apply_steering(self, generate_fn, prompt: str,
steering_config: Dict[str, float]) -> str:
"""
Generate text with steering applied.
Args:
generate_fn: Model's generation function
prompt: Input prompt
steering_config: Dict mapping trait names to steering strengths
Returns:
Generated text with trait steering applied
"""
self.steering_strength = steering_config
# Register hook
hook_handle = None
for name, module in self.model.named_modules():
if name == self.layer_name:
hook_handle = module.register_forward_hook(self.create_steering_hook())
break
# Generate with steering
output = generate_fn(prompt)
# Remove hook
if hook_handle:
hook_handle.remove()
return output
This enables post-hoc control of model behavior through activation manipulation.
Step 5: Data curation via trait-based filtering
Identify problematic training examples by detecting unexpected trait shifts:
class TrainingDataCurator:
"""Identifies problematic training examples based on trait shifts"""
def __init__(self, trait_vectors: Dict[str, np.ndarray],
extractor: PersonaVectorExtractor):
self.trait_vectors = trait_vectors
self.extractor = extractor
def detect_anomalous_examples(self, training_examples: List[str],
expected_trait_profile: Dict[str, float],
anomaly_threshold: float = 0.5) -> List[Tuple[str, str]]:
"""
Find training examples that induce unexpected trait shifts.
Args:
training_examples: List of training texts
expected_trait_profile: Expected trait intensities for this dataset
anomaly_threshold: How different from expected to flag
Returns:
List of (example, anomalous_trait) tuples
"""
flagged = []
for example in training_examples:
measurements = {}
for trait_name, trait_vec in self.trait_vectors.items():
activation = self.extractor.extract_activations(example)
projection = np.dot(activation, trait_vec)
measurements[trait_name] = projection
# Compare to expected profile
for trait_name, expected_value in expected_trait_profile.items():
if trait_name in measurements:
diff = abs(measurements[trait_name] - expected_value)
if diff > anomaly_threshold:
flagged.append((example, trait_name))
return flagged
This enables filtering of training data that would introduce undesired personality shifts.
Practical Guidance
When to use Persona Vectors:
- Monitoring production LLM systems for personality drift
- Preventing specific undesirable traits (harmfulness, dishonesty, sycophancy)
- Curating training data to maintain consistent model personality
- Fine-tuning systems where trait control is important
- Safety-critical applications requiring behavioral guarantees
When NOT to use Persona Vectors:
- Real-time, low-latency applications (monitoring adds overhead)
- Domains where trait vectors aren't well-defined (task-specific models)
- When steering strength needs to be extremely precise (use dedicated RLHF instead)
- Systems already using strong constitutional AI supervision
Key hyperparameters:
layer_name: Middle-to-late transformer layers work best (8-12 for 12-layer models)steering_strength: Range 0.01-0.5; higher values cause more personality shiftanomaly_threshold: 0.3-0.5 good for moderate sensitivityactivation_token_idx: -1 (last token) or last non-padding token- Contrastive pair ratio: 1:1 (low vs high intensity) typical
Expected monitoring overhead:
- Activation extraction: ~5-10% additional latency
- Monitoring + steering: ~10-20% overhead per inference
- Storage: One trait vector = one hidden size vector (1-3 MB for 7B models)
Recommended trait vectors to monitor:
- Truthfulness/Honesty (core safety property)
- Helpfulness (utility metric)
- Harmfulness/Danger (safety metric)
- Sycophancy (alignment metric)
- Hallucination propensity (reliability metric)
Reference
Persona Vectors: Monitoring and Controlling Character Traits in Language Models. arXiv:2507.21509