Adversarial Attack Libraries: ART, Foolbox, torchattacks

torchattacks is a PyTorch-only library of 32 adversarial attacks, from FGSM and PGD to DeepFool, CW and a bundled AutoAttack, installed with pip install torchattacks; its latest PyPI release is 3.5.1 (October 2023). Pick it for fast PyTorch evasion loops, Foolbox for multi-framework minimal-perturbation and hard-label attacks, and IBM’s Adversarial Robustness Toolbox (ART) for anything beyond evasion.

Choosing an adversarial attack library is usually treated as a taste question and it is not. The libraries differ on which threat models they can express at all, and picking the wrong one produces a robustness number that is quietly meaningless, not because the implementation is wrong, but because the attack the library ships is not the attack that would be run against the deployed system.

The confusion is worse now that “AI red teaming tool” mostly means an LLM prompt scanner. Those are a separate lineage, covered in the site’s guide to AI red teaming tools for ML models. This piece is about the other lineage: the Python libraries that implement the actual adversarial-ML attack families against classifiers, detectors and text models, where the output is a perturbed input and a robust-accuracy figure rather than a pass/fail on a prompt probe.

The comparison

Repository metadata below is read from each project’s public GitHub record as of 18 August 2026. Last activity is the date of the most recent commit in the default branch’s history, which is the cheapest honest proxy for whether a project is still tracking the literature.

Two things fall out of that table immediately. The licence column is not uniform: advertorch is LGPL-3.0 and RobustBench carries no machine-identifiable licence, both of which matter before anything gets vendored into a commercial codebase. And the maintenance column separates the list into projects still moving with the field and projects that are now historical references.

For the three libraries people most often look up, this is what pip actually installs, checked against PyPI on 9 October 2026:

Dates come from the PyPI records for torchattacks, Foolbox and ART; ART’s extras are listed in its setup guide. Note that pip install torchattacks still delivers the October 2023 code, even though the default branch has a later commit, from December 2023.

ART is the only one with the full threat surface

IBM’s Adversarial Robustness Toolbox, maintained under the Linux Foundation’s Trusted-AI umbrella, is the only library in the set that implements attacks outside the evasion family. It covers data poisoning and backdoors, model extraction, and membership inference alongside evasion, and it does so across a framework list that includes the tabular estimators (scikit-learn, XGBoost, LightGBM, CatBoost) that the PyTorch-only libraries cannot touch.

That breadth is the reason to reach for it, and it is also the reason it is heavier to use than the alternatives. If the model under assessment is a fraud classifier or a gradient-boosted tabular model, ART is effectively the only option. If the threat model includes anything at training time or anything privacy-related, it is again the only option, because every other library on the list assumes the model is finished and the attacker is at inference.

How ART’s DeepFool attack works

ART’s DeepFool class (art.attacks.evasion.DeepFool) implements the DeepFool attack of Moosavi-Dezfooli, Fawzi and Frossard (CVPR 2016): an untargeted attack that searches for the smallest perturbation that changes the predicted label, instead of the strongest perturbation inside a fixed epsilon. What it reports is distance to the decision boundary, not accuracy at a budget.

The loop in ART’s implementation linearises the classifier around the current input. For each candidate class it takes the logit gap to the current class and the difference of their gradients, picks the class whose linearised boundary is nearest, and steps exactly onto it. It re-predicts and repeats until the label flips or max_iter runs out, then scales the accumulated perturbation by (1 + epsilon) to push the input across. The same routine is the inner step of the original universal adversarial perturbation algorithm.

Constructor defaults, from the ART API reference:

from art.attacks.evasion import DeepFool
from art.estimators.classification import PyTorchClassifier

classifier = PyTorchClassifier(model=model, clip_values=(0.0, 1.0), loss=criterion,
                               optimizer=optimizer, input_shape=(3, 32, 32), nb_classes=10)
attack = DeepFool(classifier, max_iter=100, epsilon=0.02, batch_size=64)
x_adv = attack.generate(x=x_test)

The wrapper follows ART’s PyTorch get-started example. Three details in the source change results. The model must output logits: ART warns when predictions look like probabilities and raises a ValueError for a binary classifier with a single output column. nb_grads only narrows the search below the class count, so the default covers every class on a 10-class model but only the top ten on a 1,000-class one. And the default overshoot is tiny next to the alternatives: Foolbox’s L2DeepFoolAttack and LinfDeepFoolAttack default to steps=50, candidates=10, overshoot=0.02, and torchattacks’ DeepFool to steps=50, overshoot=0.02. An ART example generated at 1e-6 sits almost on the boundary and may flip back after rounding to 8-bit pixels, and DeepFool figures from different libraries are only comparable once these settings match.

Foolbox and torchattacks are the ergonomic evasion tools

Foolbox exists to make the attack loop short. It wraps PyTorch, TensorFlow and JAX behind a common tensor abstraction and leans toward minimal-perturbation search (find the smallest perturbation that succeeds, rather than the strongest attack within a fixed budget), which makes it well suited to producing a robustness curve across epsilon rather than a single number. Its black-box coverage, including the decision-based attacks that need only hard labels, is the strongest of the ergonomic options.

torchattacks trades that portability for a PyTorch-only API that is close to trivial to drop into an existing training or evaluation script, and it bundles AutoAttack so the standardised evaluation is one call. The tradeoff is visible in the maintenance column: the default branch’s last commit predates a significant amount of recent literature, so treat it as a stable implementation of the classical attacks rather than a source of current ones.

CleverHans is where several of these attacks were first implemented and its code remains a useful reference when checking somebody else’s implementation, but its activity has slowed to the point where it should not be the primary tool for new work.

torchattacks: install, API and the 32 attacks

torchattacks installs with pip install torchattacks, and its README cites Hoki Kim’s 2020 arXiv paper as the reference. Each attack is an object that wraps the model and is called like a function; the PGD arguments below are the documented defaults:

import torchattacks
atk = torchattacks.PGD(model, eps=8/255, alpha=2/255, steps=10, random_start=True)
atk.set_normalization_used(mean=[...], std=[...])  # only if the model normalises inputs
adv_images = atk(images, labels)

The README sets three constraints. Inputs must lie in [0, 1], because clipping is always applied after the perturbation, so a model trained on normalised tensors gets set_normalization_used() rather than pre-normalised inputs. The model must return a single (N, C) tensor, so anything returning a tuple or output object needs a thin wrapper. Reproducible examples need torch.backends.cudnn.deterministic = True. Targeted attacks are a mode switch on the same object, such as set_mode_targeted_random() or set_mode_targeted_by_label(), reverted with set_mode_default().

The 32 attacks group roughly like this, with the mechanics of the first group covered in FGSM, PGD and C&W explained:

• Gradient-based, mostly L∞: FGSM, BIM, RFGSM, PGD, TPGD, EOTPGD, MIFGSM, APGD, APGDT, Jitter
• Transfer-oriented: DIFGSM, TIFGSM, NIFGSM, SINIFGSM, VMIFGSM, VNIFGSM, PIFGSM, PIFGSM++, LGV
• L2, L1 and minimum-norm: CW, PGDL2, DeepFool, FAB, EADL1, EADEN
• Sparse (L0): OnePixel, SparseFool, Pixle, JSMA
• Gradient-free: Square, SPSA
• Ensemble: AutoAttack

Watch the bundled AutoAttack’s signature, AutoAttack(model, norm='Linf', eps=8/255, version='standard', n_classes=10, seed=None): a CIFAR-100 or ImageNet model needs n_classes set, and the docs’ parameter notes disagree with the signature on eps and seed, so pass both explicitly. On speed, the README’s author-run benchmark (first 50 CIFAR-10 images, RTX 2080) timed L∞ PGD at 174 ms against 354 ms for Foolbox and 1,384 ms for ART, while Foolbox was fastest on FGSM at 15 ms to torchattacks’ 54 ms.

Foolbox 3: install and the epsilon-list API

Foolbox 3 is the EagerPy rewrite published as Foolbox Native in JOSS in 2020: one code base that runs natively on PyTorch, TensorFlow and JAX. pip install foolbox deliberately leaves the framework out because, per the README, builds differ by architecture and CUDA version. Attacks take a list of epsilons and report success for each, so one call yields the robustness curve:

import foolbox as fb
fmodel = fb.PyTorchModel(model, bounds=(0, 1))
attack = fb.attacks.LinfPGD()
epsilons = [0.0, 0.001, 0.01, 0.03, 0.1, 0.3, 0.5, 1.0]
_, advs, success = attack(fmodel, images, labels, epsilons=epsilons)

The attack catalogue is where it separates from torchattacks: decision-based BoundaryAttack and HopSkipJumpAttack that need neither gradients nor probabilities, Brendel & Bethge and Fast Minimum Norm attacks in L0, L1, L2 and L∞ variants, and DeepFool in both L2 and L∞ form.

AutoAttack is the floor, not an option

Croce and Hein’s AutoAttack (arXiv:2003.01690, ICML 2020) is not a library you choose between; it is the reason most published robustness numbers before it were wrong. The paper reports that applying its parameter-free ensemble to over 50 previously published models produced lower robust accuracy than those papers claimed (in the authors’ words, “often by more than 10%”) and identified several defences as broken outright.

The ensemble combines two variants of an automatically step-size-tuned PGD, a targeted FAB attack and the black-box Square attack, and it has no hyperparameters to tune, which removes the most common way an evaluation flatters itself: an under-tuned attack. It is the harness behind RobustBench, whose leaderboard is the reference point for what robust accuracy on CIFAR-10 and ImageNet actually looks like under a standard threat model.

The practical rule is simple. Whatever library generates the attacks during development, the number that leaves the building should be produced by AutoAttack under a declared threat model, because that is the number a reader can compare against a leaderboard. Everything else is a development signal. The failure modes that make a self-run evaluation optimistic (gradient masking, too few steps, a single restart, an attack that never sees the defence it is supposed to break) are covered in evaluating adversarial robustness without fooling yourself.

Text and everything the libraries do not cover

TextAttack is the outlier in the list and the most actively maintained project in it. It targets NLP models through the Hugging Face ecosystem and implements the word- and character-level search strategies (synonym substitution under semantic constraints, importance ranking, masked-language-model-guided replacement) that make up the adversarial robustness landscape for text. Its constraint system, which bounds how far a perturbed sentence may drift in meaning, is the part with no analogue in the image libraries, because an image attack’s budget is a norm ball and a text attack’s budget is a semantics argument.

The coverage gap worth naming explicitly is multimodal. None of these libraries implements attacks against contrastive vision-language encoders, which is now one of the most consequential surfaces in deployed systems; those attacks live in the paper authors’ own repositories, as with the pre-generated universal perturbations discussed in CLIP adversarial attacks. Nor does any of them cover agent-level attacks, where the exploit targets tool invocation rather than a decision boundary.

Choosing by threat model, not by preference

• Tabular, tree-based or scikit-learn model: ART. Nothing else supports the estimator.
• Training-time, privacy or extraction threat model: ART. The other libraries stop at inference.
• PyTorch image classifier, quick internal evaluation: torchattacks for the loop, AutoAttack for the reported number.
• Robustness curve across perturbation budgets, or hard-label black-box: Foolbox.
• Distance to the boundary rather than accuracy at a budget: DeepFool or Fast Minimum Norm in Foolbox, or ART’s DeepFool with the overshoot set explicitly.
• Text classifier or transformer NLP model: TextAttack.
• A number somebody outside the team will compare: AutoAttack, with the RobustBench leaderboard as context.
• Contrastive encoders and multimodal stacks: none of the above; go to the paper repositories.

Maintenance status belongs in this decision rather than beside it. An attack library that stopped tracking the literature will still run, still return a robust-accuracy figure, and still miss the attack class published after its last commit, which produces a robustness claim that is technically reproducible and substantively stale. To reason about which attacks are feasible against a particular access level, modality and adversary goal before choosing a tool, the site’s attack selector filters the catalogue on those constraints and names a reference implementation for each entry.

See also

• AI attack techniques
• AI security blog

FAQ

what is torchattacks used for

torchattacks is used to generate adversarial examples against PyTorch classifiers and to measure how robust those models are. It wraps a model in an attack object (PGD, CW, DeepFool, AutoAttack and 28 others) and returns perturbed images in one call, which suits adversarial training loops and quick internal robustness checks.

is torchattacks still maintained

Only lightly. The newest PyPI release is 3.5.1 from 21 October 2023, and the most recent commit on the GitHub repository’s default branch dates from December 2023. The classical attacks it ships remain usable, but attacks published since then are missing, so report final numbers with the reference AutoAttack implementation.

what is the deepfool attack in adversarial robustness toolbox

ART’s DeepFool (art.attacks.evasion.DeepFool) is an untargeted minimal-perturbation attack from Moosavi-Dezfooli et al. (CVPR 2016). It repeatedly linearises the classifier, steps to the nearest class boundary and stops once the label flips. Defaults are max_iter=100, epsilon=1e-6 (an overshoot, not a budget), nb_grads=10 and batch_size=1, and the classifier must expose class gradients.

how do i install adversarial robustness toolbox

Run pip install adversarial-robustness-toolbox; the latest PyPI release, checked in October 2026, is 1.20.1 from 7 July 2025. Add an extra to install a framework alongside it, such as [pytorch], [tensorflow], [xgboost] or [all], otherwise install the framework separately. The package imports as art, and models must be wrapped in an ART estimator first.

foolbox or torchattacks which should i use

Use torchattacks when the model is PyTorch and you want PGD-style attacks inside a training loop with minimal code. Use Foolbox when you need TensorFlow or JAX support, hard-label decision-based attacks such as HopSkipJump and Boundary, or a robustness curve across many epsilons from a single call. Both are MIT-licensed.