Here is how adversarial examples fool image classifiers: the attacker takes the gradient of the model’s loss with respect to the input pixels and nudges every pixel a tiny, bounded amount in the direction that raises that loss. Summed over the 150,528 values in a 224×224 RGB image, those nudges push the input across a decision boundary while a human sees the same picture. In the canonical demonstration from Goodfellow, Shlens and Szegedy, a perturbation of 0.007 per pixel, roughly the smallest bit of an 8-bit image encoding, turned GoogLeNet’s 57.7%-confidence “panda” into a 99.3%-confidence “gibbon.”
That is not a bug in one network. Szegedy et al. reported the phenomenon in December 2013, along with the uncomfortable detail that the same perturbation could fool a different network trained on a different subset of the dataset. A decade of defense papers later, the core mechanics are unchanged, and they carry over from MNIST digits to stop signs to the vision encoders inside multimodal LLMs.
How it works: the model hands you the attack direction
Training minimizes loss by moving weights. An evasion attack freezes the weights and moves the input instead. Formally: find a perturbation δ that maximizes the loss L(f(x + δ), y) subject to ‖δ‖ ≤ ε, where the choice of norm and ε defines what “imperceptible” means in your threat model. L∞ caps the change to any single pixel, L2 caps the total energy, L0 caps how many pixels change at all.
Projected gradient descent (PGD) is the workhorse. Stripped down, it is a dozen lines:
import torch
import torch.nn.functional as F
def pgd(model, x, y, eps=8/255, alpha=2/255, steps=10):
# untargeted L-inf PGD; x in [0,1], model.eval()
delta = torch.empty_like(x).uniform_(-eps, eps)
for _ in range(steps):
delta.requires_grad_(True)
loss = F.cross_entropy(model(x + delta), y)
grad, = torch.autograd.grad(loss, delta)
delta = (delta + alpha * grad.sign()).clamp(-eps, eps)
delta = ((x + delta).clamp(0, 1) - x).detach() # stay a valid image
return x + deltaEach iteration reads the gradient, keeps only its sign, takes a small step uphill on the loss, and projects back into the ε-ball and the valid pixel range. Drop the random start and run a single step with alpha equal to eps and you have FGSM, which the FGSM walkthrough covers line by line. For a targeted attack (“classify this as a gibbon”), pass the target label instead of the true one and subtract the step instead of adding it. The objectives of FGSM, PGD and Carlini-Wagner are compared side by side in evasion attacks on image classifiers.
Nothing here exploits a software flaw. The attack runs on the same autograd machinery the defender used to train the model. If you can backpropagate through it, you can attack it.
Why do such tiny perturbations work?
Tiny perturbations work because the model adds them up. Goodfellow et al. argued the primary cause is the linear nature of deep networks: for a weight vector with n dimensions and average magnitude m, a perturbation of ε per element aligned with sign(w) shifts the activation by ε·m·n. The per-pixel change stays fixed while the total effect grows with input dimensionality.
High-resolution images hand the attacker a large n. The same paper shows how badly this plays out even on toy data. At ε = 0.25 on MNIST, FGSM drove a shallow softmax classifier to a 99.9% error rate and a maxout network to 89.4%, the latter with 97.6% average confidence in its wrong answers. On CIFAR-10 at ε = 0.1, a convolutional maxout network hit 87.15% error.
Look at the confidence. The model is not confused; it is certain and wrong. Any defense built on rejecting low-confidence predictions is watching the wrong signal.
The features humans cannot see
Linearity explains the arithmetic. Ilyas et al. explain why the gradient points somewhere useful. Standard training rewards any feature that predicts the label, including what they call non-robust features: patterns that are highly predictive “yet brittle and incomprehensible to humans.” A classifier is behaving rationally when it uses them, because they work on the test set. From the model’s point of view the perturbation is not noise. It is a targeted rewrite of features the model actually relies on, features a human never sees.
This also explains transfer. Two models trained on the same distribution learn overlapping non-robust features, so a perturbation that flips one tends to flip the other.
Do you need the model’s weights to fool it?
No. Gradients make the attack cheap, but transferability makes weights optional. Papernot, McDaniel and Goodfellow trained a local substitute model from the victim’s answers, crafted adversarial examples against the substitute, and replayed them against the victim. Against hosted classifiers in 2016, they reported 96.19% misclassification on Amazon and 88.94% on Google, using 800 queries.
Those numbers describe 2016-era ML-as-a-service endpoints, not current ones, but the technique has not aged. Access level maps to technique roughly like this:
| Attacker access | Technique | Demonstrated in |
|---|---|---|
| Weights and gradients | FGSM, PGD, C&W optimization | Goodfellow et al. 2014 |
| Label or score queries | Train a substitute, then transfer | Papernot et al. 2016 |
| No target access, same task | Craft on a surrogate, then transfer | Szegedy et al. 2013; Papernot et al. 2016 |
| Physical scene only | Stickers or patches optimized across conditions | Eykholt et al. 2017 |
Push transfer further and you get perturbations that are input-agnostic, model-agnostic or both. See universal adversarial perturbations and the CLIP attacks where one perturbation carries across encoders.
Do adversarial examples survive the physical world?
Yes, if the attacker optimizes for it. Kurakin, Goodfellow and Bengio fed adversarial images captured through a cell-phone camera to an ImageNet Inception classifier and found that a large fraction were still misclassified even when perceived through the camera.
Eykholt et al. went further with black and white stickers on a real stop sign. Their attack produced targeted misclassification in 100% of lab images and 84.8% of video frames captured from a moving vehicle. The perturbation was no longer imperceptible; it read as graffiti. That is the realistic trade for physical attacks: drop the stealth constraint, gain robustness across distance, angle and lighting. The full treatment is in adversarial patch attacks in the physical world.
At the other extreme, Su, Vargas and Sakurai showed the L0 case. Changing a single pixel pushed 67.97% of CIFAR-10 test images into at least one other class, against 16.04% on ImageNet. One pixel is a far larger share of a 32×32 CIFAR-10 image than of an ImageNet-sized input.
Why don’t the obvious defenses work?
Most cheap defenses break the gradient, not the vulnerability. Athalye, Carlini and Wagner examined nine white-box defenses from ICLR 2018, found seven relying on obfuscated gradients, and circumvented six of those completely and one partially, inside each paper’s own threat model.
The pattern repeats. JPEG compression, denoisers, random resizing and bit-depth reduction make the gradient useless to a naive attacker. An adaptive attacker approximates the non-differentiable step on the backward pass or averages over the randomness, and accuracy collapses again. Preprocessing is especially weak on small edge architectures; a 2026 preprint found it fails on depthwise-separable CNNs. If a defense only works because the attacker cannot compute a useful gradient, assume someone will compute one.
What to do
- Write the threat model before the defense. Pin down the norm, ε, attacker access (white-box, query, transfer, physical) and goal (targeted or untargeted). NIST AI 100-2 E2025 structures attacks by lifecycle stage and by attacker goals, capabilities and knowledge; use that as the template so your robustness claim has a scope.
- Adversarially train at the budget you care about. PGD-based adversarial training from Madry et al. remains the empirical baseline. Expect lower clean accuracy and robustness that holds only near the training budget. The knobs are covered in adversarial training best practices.
- Evaluate adaptively, then try to break your own numbers. Run PGD with restarts or AutoAttack, plus a transfer attack and a gradient-free attack. If a black-box attack beats your white-box attack, your gradients are masked. The checklist lives in evaluating adversarial robustness correctly, and the adversarial attack libraries comparison covers which tools implement what.
- Starve the oracle. Return top-1 labels instead of full probability vectors, rate-limit inference, and alert on bursts of near-duplicate queries from one client. This raises the cost of query-based search and substitute training. It does nothing against pure transfer from a public surrogate, so do not sell it as a fix.
- Never let one classifier be the security boundary. If a misclassification unlocks money, access or a physical action, corroborate it with an independent signal: a second sensor, a different modality, a human review threshold.
FAQ
what is an adversarial example in image classification
An adversarial example is an image deliberately modified, usually by a small bounded perturbation, so that a classifier outputs a wrong label while a human still sees the original content. The modification is computed, not random: it follows the model’s own loss gradient or a surrogate’s, which is why random noise of the same size rarely works.
are adversarial examples the same as data poisoning
No. Adversarial examples are evasion attacks at inference time: the trained model is untouched and only the input changes. Data poisoning corrupts the training set so the model learns the wrong behavior or a hidden backdoor. The defenses differ accordingly; adversarial examples vs. data poisoning compares the two threat models side by side.
can adversarial training make an image classifier fully robust
No. Adversarial training buys robustness against the norm and budget it was trained on, at a cost in clean accuracy, and gives no guarantee outside that box. Models hardened against L∞ PGD can still fall to patches, spatial transforms or other perturbation types, which is the gap the UAR metric measures.
do adversarial examples work on vision-language models
Yes. Vision-language models read pixels through an image encoder, and that encoder inherits the same gradient-driven weakness as any classifier. Perturbations crafted against open encoders such as CLIP can steer captions and answers and have been used to jailbreak aligned models; adversarial attacks on vision-language models covers the techniques.
Related across the network
- Evasion Attacks in Production ML: Malware, Spam, and Fraud — aiattacks.dev
- What Is Adversarial ML? A Practitioner’s Primer — ai-alert.org
