Adversarial Ml Evasion

Adversarial ML Evasion

Adversarial examples are inputs modified with small, deliberate perturbations that reliably cause a trained classifier to produce incorrect predictions. The perturbation is typically imperceptible to humans or functionally irrelevant, while the model's decision boundary is crossed. This is distinct from LLM jailbreaks: the target is a trained discriminative model (CNN, gradient-boosted tree, SVM, LSTM) served via an API or embedded in a product.

Authorized use only. All testing must be conducted against systems you own or have explicit written permission to assess. Generating adversarial inputs against production ML APIs without authorization may violate the CFAA and equivalent statutes.

ATT&CK Mapping

1. Reconnaissance — understand the target model

Before crafting perturbations, determine the attack surface:

# 1a. Identify the ML stack from job postings, open-source repos, error messages
# Common deployment stacks: TensorFlow Serving, TorchServe, ONNX Runtime, SageMaker, AzureML

# 1b. Probe input shape and output format
curl -s -X POST "$TARGET/predict" \
  -H "Content-Type: application/json" \
  -d '{"data": [[0.0]*784]}' | jq .
# Note: number of output classes, confidence scores vs hard labels
# Confidence scores = white-box-equivalent gradient signal via finite differences

# 1c. Check if the API returns confidence values (score-based black-box) or label only (hard-label)
# Score-based: enables gradient estimation, ZOO, NES attacks
# Label-only: requires hard-label attacks (HopSkipJump, QEBA)

2. White-box evasion (full model access)

Use when you have the model weights (pentest scope, internal system, open-source model).

2a. FGSM — Fast Gradient Sign Method (single step)

import torch, torchvision.transforms as T
from PIL import Image
import requests, json

def fgsm(model, x, y_true, epsilon=0.03):
    """Single-step gradient attack. Fast, low distortion for strong models."""
    x.requires_grad_(True)
    loss = torch.nn.CrossEntropyLoss()(model(x), y_true)
    loss.backward()
    return (x + epsilon * x.grad.sign()).clamp(0, 1).detach()

2b. PGD — Projected Gradient Descent (iterative, stronger)

def pgd(model, x, y_true, epsilon=0.03, alpha=0.007, steps=40):
    """Madry's PGD — iterative FGSM with projection back into epsilon-ball."""
    x_adv = x.clone().detach().requires_grad_(True)
    for _ in range(steps):
        loss = torch.nn.CrossEntropyLoss()(model(x_adv), y_true)
        loss.backward()
        with torch.no_grad():
            x_adv = x_adv + alpha * x_adv.grad.sign()
            delta = torch.clamp(x_adv - x, -epsilon, epsilon)
            x_adv = torch.clamp(x + delta, 0, 1).detach().requires_grad_(True)
    return x_adv

2c. C&W — Carlini & Wagner (optimization-based, minimal distortion)

pip install adversarial-robustness-toolbox  # IBM ART
python - <<'EOF'
from art.attacks.evasion import CarliniL2Method
from art.estimators.classification import PyTorchClassifier
# wrap your model in PyTorchClassifier, then:
attack = CarliniL2Method(classifier=clf, confidence=0.5, max_iter=100)
x_adv = attack.generate(x=x_test[:10])
EOF

3. Black-box evasion (API access only)

3a. Score-based: NES / ZOO (gradient-free estimation)

When confidence scores are returned, estimate gradient via finite differences:

import numpy as np

def nes_gradient_estimate(query_fn, x, sigma=0.01, n=50):
    """Natural Evolution Strategy gradient estimate from prediction scores."""
    grads = np.zeros_like(x)
    for _ in range(n):
        noise = np.random.randn(*x.shape)
        pos = query_fn(x + sigma * noise)
        neg = query_fn(x - sigma * noise)
        grads += (pos - neg) * noise
    return grads / (2 * n * sigma)

# query_fn: callable that sends x to the API and returns target-class score
# Use grads to step: x_adv = x - alpha * np.sign(grads)

3b. Transfer attack (surrogate model)

The reference approach from 13o-bbr-bbq/machine_learning_security's CNN_test:

# 1. Collect input samples via normal API usage
# 2. Train a local surrogate on (input, label) pairs
# 3. Run white-box attack (FGSM/PGD) against surrogate
# 4. Transfer adversarial examples to the target
# Transferability is high when surrogate and target share architecture family

pip install cleverhans
python - <<'EOF'
import tensorflow as tf
from cleverhans.tf2.attacks.fast_gradient_method import fast_gradient_method
# Build surrogate_model from collected (x, y) pairs
x_adv = fast_gradient_method(surrogate_model, x_test, eps=0.03, norm=np.inf)
# Submit x_adv to the black-box API
EOF

3c. Hard-label: HopSkipJump

When only the predicted label is returned (no scores):

python - <<'EOF'
from art.attacks.evasion import HopSkipJump
from art.estimators.classification import BlackBoxClassifier

def predict_fn(x):
    # Call your API here, return one-hot array
    ...

clf = BlackBoxClassifier(predict_fn, input_shape=(32,32,3), nb_classes=10)
attack = HopSkipJump(classifier=clf, max_iter=50, max_eval=1000)
x_adv = attack.generate(x=x_test[:5])
EOF

4. Physical-world / domain-specific attacks

4a. Adversarial patches (object detection, face recognition)

Craft a printable patch that fools YOLO/RetinaNet regardless of placement:

pip install adversarial-robustness-toolbox
python - <<'EOF'
from art.attacks.evasion import AdversarialPatch
attack = AdversarialPatch(classifier=clf, rotation_max=22.5, scale_min=0.1, scale_max=1.0)
patch, mask = attack.generate(x=x_train[:100])
# Print patch, affix to subject — fools real-time camera-based classifiers
EOF

4b. Malware binary perturbation (AV/EDR evasion)

For ML-based static AV/EDR (MalConv, EMBER model):

# Append bytes to DOS overlay — does not affect execution, changes feature vector
python - <<'EOF'
import lief, numpy as np

binary = lief.parse("sample.exe")
# Append random payload to avoid changing import table / section headers
binary.dos_stub = bytes(np.random.randint(0, 256, 64))
builder = lief.PE.Builder(binary)
builder.build()
builder.write("sample_adv.exe")
# Iterate: score the modified binary against the target ML model
# Stop when target-class score drops below threshold
EOF

4c. Network traffic / IDS evasion

For ML-based IDS (random-forest or neural-net on flow features):

# Identify mutable features: packet timing, padding, flow ordering
# Immutable: payload content that triggers the attack itself
# Strategy: inflate packet-count, adjust inter-arrival times to move
# feature vector away from known-malicious region

scapy - <<'EOF'
from scapy.all import *
# Add benign-looking padding packets to inflate flow duration
# Shift timing to match benign profile learned from surrogate
EOF

5. Tooling reference

pip install adversarial-robustness-toolbox foolbox cleverhans

6. Validation checklist

• Clean input correctly classified before perturbation (baseline confirmed)
• Adversarial input misclassified at the same model endpoint
• Perturbation is within declared epsilon-ball (L-inf or L2 norm verified)
• For transfer: adversarial success rate on black-box > 30% (non-trivial transfer)
• For physical: photograph and re-score — confirm printed patch works under camera

7. Detection signals (defender perspective)

• Input pixel distribution deviates from natural image statistics (high-frequency noise)
• Feature-squeeze defence: score changes drastically when input is median-filtered
• Input preprocessing (JPEG re-encode, resize+crop) breaks gradient-based perturbations
• Ensemble disagreement: 3 models with different architectures disagree on the adversarial input
• Activation distribution of adversarial inputs differs from clean examples (mahalanobis-distance detector)