ndpvt-web Defending Vision-Language Models Against Universal Transferable Adversarial Attacks
This skill enables Claude to help build defensive infrastructure for vision-language models (VLMs) based on the attack-surface analysis from the UltraBreak framework (Cui et al., ICLR 2026). UltraBreak showed that adversarial image perturbations can be made both universal (one pattern works across many harmful prompts) and transferable (patterns crafted on one model fool different models) by combining vision-space regularisation with semantic textual objectives. Understanding this attack vector is essential for building robust VLM deployments. This skill teaches how to evaluate, detect, and mitigate these adversarial image inputs in production multimodal systems.
When to Use
- When building or deploying a VLM pipeline (e.g., GPT-4V, LLaVA, InternVL, Qwen-VL) and you need to evaluate its adversarial robustness before production.
- When a user asks to implement input-validation or sanitization layers for image inputs to a multimodal model.
- When designing a red-team evaluation harness that tests whether adversarial images can bypass a VLM's safety alignment.
- When adding runtime detection for adversarial perturbation patterns in image inputs to an inference server.
- When a user needs to understand why gradient-based image perturbations transfer across VLMs and how to defend against the mechanism.
- When hardening a content-moderation pipeline that uses VLMs to classify user-uploaded images paired with text.
Key Technique: Why UltraBreak Transfers and How to Defend Against It
UltraBreak's core insight is that prior adversarial image attacks on VLMs overfit to the surrogate model in two ways: (1) the adversarial perturbation in vision space is unconstrained, encoding model-specific high-frequency artifacts, and (2) the textual attack objective is a rigid token-level target (e.g., forcing the model to output "Sure, here is..."), which creates a sharp, narrow loss landscape that does not generalise.
UltraBreak addresses these by constraining the vision side (applying differentiable image transformations like random resizing, cropping, and JPEG compression during optimisation, plus total-variation regularisation to smooth the perturbation) while relaxing the text side (replacing token-level targets with a cosine-similarity objective in the LLM's embedding space against semantically equivalent affirmative responses). This combination smooths the loss landscape, yielding perturbations that sit in flat minima and generalise across models and prompts.
For defenders, this analysis reveals three actionable principles: (a) adversarial perturbations that survive transformations like JPEG compression and resizing are the dangerous ones, so naive pixel-level detection is insufficient; (b) the perturbation energy concentrates in vision-encoder feature space rather than raw pixel space, so detection should operate on intermediate representations; (c) universal perturbations produce statistically detectable shifts in the vision encoder's activation distribution compared to benign images, which enables anomaly-detection-based defences.
Step-by-Step Workflow
1. Audit the VLM Architecture for Attack Surface
Identify the vision encoder (e.g., CLIP ViT, SigLIP, InternViT), the projection layer mapping vision features to the LLM's embedding space, and the LLM backbone. Document the image preprocessing pipeline (resize resolution, normalisation constants, crop strategy). These are the exact components an attacker targets.
2. Build an Adversarial Robustness Evaluation Harness
Create a test suite that pairs images with adversarial text prompts from standard safety benchmarks (e.g., AdvBench, MM-SafetyBench, SafeBench). Define automated metrics: attack success rate (ASR) measured by a safety classifier (e.g., LlamaGuard, HarmBench classifier) applied to model outputs. Run the baseline with clean images to establish the model's refusal rate.
# Example: evaluation harness skeleton
import torch
from dataclasses import dataclass
@dataclass
class EvalCase:
image_path: str
harmful_prompt: str
category: str # e.g., "violence", "illegal_activity"
def evaluate_safety(model, tokenizer, eval_cases, safety_classifier):
results = []
for case in eval_cases:
image = load_and_preprocess(case.image_path, model.image_size)
response = model.generate(image, case.harmful_prompt)
is_harmful = safety_classifier.classify(response)
results.append({
"category": case.category,
"refused": not is_harmful,
"response_snippet": response[:200],
})
asr = sum(1 for r in results if not r["refused"]) / len(results)
return asr, results
3. Implement Input-Level Image Sanitization
Apply the same transformations that UltraBreak trains against, since perturbations that survive these are harder to craft. Stack multiple stochastic transforms at inference time:
import torchvision.transforms as T
import random
class AdversarialSanitizer:
"""Apply stochastic transforms to disrupt adversarial perturbations.
Uses the same transforms UltraBreak optimises through, making it
harder for perturbations to survive the preprocessing stage."""
def __init__(self, input_size=336, jpeg_quality_range=(60, 95)):
self.input_size = input_size
self.jpeg_quality_range = jpeg_quality_range
def __call__(self, image_tensor):
# Random resize + crop (disrupts spatial perturbation alignment)
scale = random.uniform(0.85, 1.15)
new_size = int(self.input_size * scale)
image_tensor = T.Resize(new_size)(image_tensor)
image_tensor = T.CenterCrop(self.input_size)(image_tensor)
# JPEG compression (removes high-frequency adversarial signal)
quality = random.randint(*self.jpeg_quality_range)
image_tensor = jpeg_compress_tensor(image_tensor, quality)
# Gaussian blur with small kernel (smooth residual perturbation)
if random.random() < 0.3:
image_tensor = T.GaussianBlur(kernel_size=3, sigma=0.5)(image_tensor)
return image_tensor
4. Build Feature-Space Anomaly Detection
Extract vision encoder activations for a reference set of benign images to build a baseline distribution. At inference, flag images whose feature-space statistics deviate significantly:
import numpy as np
from scipy.spatial.distance import mahalanobis
class FeatureSpaceDetector:
def __init__(self, vision_encoder, reference_images, layer_name="last_hidden"):
features = extract_features(vision_encoder, reference_images, layer_name)
self.mean = np.mean(features, axis=0)
self.cov_inv = np.linalg.pinv(np.cov(features.T) + 1e-6 * np.eye(features.shape[1]))
def score(self, vision_encoder, image):
feat = extract_features(vision_encoder, [image], "last_hidden")[0]
return mahalanobis(feat, self.mean, self.cov_inv)
def is_adversarial(self, vision_encoder, image, threshold=3.5):
return self.score(vision_encoder, image) > threshold
5. Add Total-Variation Analysis for Perturbation Detection
UltraBreak uses TV regularisation to produce smooth perturbations. Detect unregularised attacks by measuring the total variation of the difference between the input and its smoothed version:
def total_variation(image_tensor):
"""Compute total variation of an image tensor (C, H, W)."""
diff_h = torch.abs(image_tensor[:, 1:, :] - image_tensor[:, :-1, :]).sum()
diff_w = torch.abs(image_tensor[:, :, 1:] - image_tensor[:, :, :-1]).sum()
return (diff_h + diff_w).item()
def perturbation_tv_score(image_tensor, blur_kernel=5):
"""High scores suggest adversarial high-frequency content."""
smoothed = T.GaussianBlur(blur_kernel, sigma=1.5)(image_tensor)
residual = image_tensor - smoothed
return total_variation(residual)
6. Implement Embedding-Space Output Monitoring
Since UltraBreak optimises for cosine similarity to affirmative embeddings, monitor the LLM's first-token embeddings for proximity to known-harmful affirmative patterns:
def monitor_output_embeddings(model, generated_ids, harmful_anchors):
"""Check if early output embeddings are suspiciously close to
affirmative response patterns that indicate a successful jailbreak."""
embeddings = model.get_input_embeddings()
first_k = generated_ids[:5] # First 5 tokens
gen_emb = embeddings(first_k).mean(dim=0)
max_sim = max(
torch.cosine_similarity(gen_emb.unsqueeze(0), anchor.unsqueeze(0)).item()
for anchor in harmful_anchors
)
return max_sim # > 0.85 is suspicious
7. Compose a Defence-in-Depth Pipeline
Wire the components together in your inference server:
class DefendedVLMPipeline:
def __init__(self, model, sanitizer, feature_detector, tv_threshold=0.15):
self.model = model
self.sanitizer = sanitizer
self.feature_detector = feature_detector
self.tv_threshold = tv_threshold
def __call__(self, image, prompt):
# Layer 1: Perturbation TV check
tv_score = perturbation_tv_score(image)
# Layer 2: Feature-space anomaly detection
anomaly_score = self.feature_detector.score(self.model.vision_encoder, image)
# Layer 3: Input sanitization (always applied)
clean_image = self.sanitizer(image)
# Layer 4: Generate with sanitized input
response = self.model.generate(clean_image, prompt)
# Layer 5: Output-level safety classification
is_safe = safety_classify(response)
return {
"response": response if is_safe else "[Response blocked by safety filter]",
"flags": {
"tv_anomaly": tv_score > self.tv_threshold,
"feature_anomaly": anomaly_score > 3.5,
"output_blocked": not is_safe,
}
}
8. Run Transfer-Attack Stress Tests
To validate your defences, test with publicly available adversarial benchmarks. Use white-box attacks on an open-source surrogate (e.g., LLaVA-1.5) and evaluate whether the perturbations transfer to your defended pipeline. Measure both undefended and defended ASR.
9. Calibrate Detection Thresholds on Held-Out Data
Use a validation set of benign images across diverse domains (natural photos, diagrams, screenshots, memes) to set false-positive-acceptable thresholds for the Mahalanobis distance and TV score detectors. Target <1% FPR on benign inputs.
Concrete Examples
Example 1: Adding adversarial robustness evaluation to a LLaVA deployment
User: "I'm deploying LLaVA-1.5-13B for a customer-facing product. How do I test if adversarial images can make it say harmful things?"
Approach:
- Set up the evaluation harness from Step 2 with AdvBench prompts paired with clean images (baseline) and publicly available adversarial image benchmarks.
- Use LlamaGuard-3 as the safety classifier to automatically score outputs.
- Report attack success rate per harm category; if ASR > 5% on any category, proceed to implement the defence pipeline from Step 7.
Output:
Robustness Evaluation Report - LLaVA-1.5-13B
=============================================
Baseline ASR (clean images): 2.1% (model refuses most harmful prompts)
Transfer attack ASR (PGD): 18.7% (standard gradient attack)
Transfer attack ASR (UltraBreak-style): 41.3% (universal perturbation)
RECOMMENDATION: Deploy with sanitization + feature-space detection.
After defence pipeline: ASR drops to 4.8%
Example 2: Building an image input sanitizer for a multimodal chatbot
User: "My chatbot accepts user-uploaded images. I want to preprocess them to remove potential adversarial perturbations before passing to the VLM."
Approach:
- Implement the
AdversarialSanitizerclass from Step 3. - Integrate it into the image upload handler, applying transforms before the image reaches the vision encoder.
- Validate that sanitization does not degrade benign image understanding by running the model on VQAv2 or similar benchmarks with and without sanitization.
Output:
# In your FastAPI image upload endpoint:
sanitizer = AdversarialSanitizer(input_size=336, jpeg_quality_range=(70, 90))
@app.post("/chat")
async def chat(image: UploadFile, prompt: str):
img_tensor = load_image(image)
clean_tensor = sanitizer(img_tensor) # Adversarial sanitization
response = vlm.generate(clean_tensor, prompt)
return {"response": response}
# Benign accuracy impact: VQAv2 accuracy 79.2% -> 78.4% (minimal degradation)
Example 3: Detecting adversarial images via feature-space anomaly scoring
User: "I want to log and flag suspicious images hitting my VLM API without blocking them outright."
Approach:
- Collect 5,000+ benign images representative of your expected input distribution.
- Build the
FeatureSpaceDetectorfrom Step 4 using CLIP ViT-L features. - Add the detector as middleware that logs anomaly scores and flags outliers for human review.
Output:
# Middleware integration
detector = FeatureSpaceDetector(clip_model, reference_images)
@app.middleware("http")
async def adversarial_detection_middleware(request, call_next):
if request.url.path == "/chat":
image = extract_image_from_request(request)
score = detector.score(clip_model, image)
if score > 3.5:
logger.warning(f"Anomalous image detected: score={score:.2f}")
# Flag for review but don't block
request.state.adversarial_flag = True
return await call_next(request)
Best Practices
- Do: Apply multiple stochastic transforms (resize, JPEG, blur) at inference time -- UltraBreak shows that adversarial perturbations robust to one transform may not survive the composition of several.
- Do: Monitor vision-encoder intermediate features, not just raw pixels -- universal perturbations create detectable distributional shifts in feature space even when pixel-level changes are imperceptible.
- Do: Calibrate detection thresholds on your actual input distribution -- Mahalanobis distance thresholds vary significantly between photo-heavy and diagram-heavy workloads.
- Do: Combine input-level defences (sanitization) with output-level defences (safety classifiers) for defence in depth.
- Avoid: Relying solely on pixel-level perturbation detection (L-inf norm checks) -- UltraBreak's TV regularisation produces smooth, low-norm perturbations that evade such checks.
- Avoid: Assuming that safety alignment alone is sufficient -- the paper demonstrates that aligned models can be reliably bypassed through the image channel without modifying the text prompt.
Error Handling
- High false-positive rate on feature-space detection: Your reference image set is not representative. Expand it to cover the actual distribution of inputs (screenshots, charts, photos, etc.). Consider per-domain reference sets.
- Sanitization degrades benign performance significantly (>2% drop): Reduce the aggressiveness of transforms -- increase JPEG quality floor to 80, narrow the resize scale range to 0.9-1.1, reduce blur probability.
- Adversarial images bypass all transform-based defences: This suggests a highly robust perturbation (likely optimised through the same transforms). Escalate to feature-space detection and consider ensemble sanitization with randomised parameter sampling per request.
- Safety classifier produces inconsistent results: Use an ensemble of safety classifiers (LlamaGuard + HarmBench + keyword heuristics) and flag if any classifier triggers.
Limitations
- Transform-based sanitization is an arms race: attackers who know your exact transform pipeline can optimise through it (as UltraBreak itself demonstrates). Stochasticity helps but does not guarantee robustness.
- Feature-space detection requires a well-characterised benign distribution. For open-domain applications with highly diverse inputs, false-positive rates may be unacceptable.
- These defences add inference latency (5-20ms for sanitization, 10-30ms for feature extraction on GPU). Budget accordingly for latency-sensitive applications.
- The techniques here apply to image-based adversarial inputs. They do not defend against text-only jailbreaks or other attack vectors (prompt injection, indirect injection via image text content).
- Detection thresholds are empirical and need periodic recalibration as your input distribution shifts over time.
Reference
Paper: Cui, K., Li, Y., Wu, Y., Ma, X., & Erfani, S. (2026). Toward Universal and Transferable Jailbreak Attacks on Vision-Language Models. ICLR 2026. arXiv:2602.01025
Key insight for defenders: Section 4 (loss landscape analysis) shows that semantic-space objectives create flat loss minima enabling transfer -- this same insight tells defenders that feature-space monitoring is the right detection layer, and that sharp pixel-level checks will miss the most dangerous attacks. The ablation in Table 3 quantifies how each regularisation component contributes to transferability, which directly informs which transforms are most effective for sanitization.
Code: github.com/kaiyuanCui/UltraBreak -- useful for building red-team evaluation harnesses to stress-test your defences.