Adversarial Robustness in NLP: Why Text Attacks Differ

Text attacks differ from image attacks because text has no norm ball. An NLP adversarial example is a discrete edit that must pass a stack of heuristic meaning filters, and the filters decide the result: TextFooler’s defaults collapse BERT’s accuracy, while filters calibrated against human judgment cut its success rate by over 70 percentage points.

The discreteness problem

In vision, a perturbation is small in L-infinity or L2 norm, invisible, and two papers quoting epsilon = 8/255 quote the same threat. Text is discrete; there is no adding epsilon to the word “dog.” An adversarial example in text needs another definition of admissible change:

• Character-level: typos, homoglyphs, invisible or control characters a reader parses as unchanged
• Word-level: synonym substitution, dropping or inserting uninformative words
• Sentence-level: paraphrase that keeps the meaning and changes the surface form

Whether the result still means the same thing is a human judgment, which attacks approximate with automated filters. Encoding tricks that render to identical glyphs are valid by construction; everything else needs its constraints written down.

What TextFooler actually reported

TextFooler (Jin et al., AAAI 2020) ranks words by how much deleting each one changes the prediction, takes up to 50 nearest neighbors of the top word in counter-fitted GloVe space, filters them by part of speech and Universal Sentence Encoder (USE) similarity, keeps the candidate that most lowers the true-class confidence, and repeats until the label flips.

It attacked WordCNN, WordLSTM and BERT on five classification datasets (AG, Fake, MR, IMDB, Yelp), and InferSent, ESIM and BERT on SNLI and MultiNLI; RoBERTa and XLNet were not targets. The BERT results:

Two columns say more than the accuracy collapse. Short inputs lose a far larger share of their words: 6.1% of the average IMDB review is about 13 words, 16.7% of an MR review about three. The lowest similarity, 0.45, is on SNLI, the shortest inputs. Jin et al.’s human evaluation found 92% label agreement between original and adversarial text on MR and 85% on SNLI.

BERT-Attack (Li et al., EMNLP 2020) draws candidates from a masked language model instead; its substitutes are fluent, which is not the same as meaning-preserving.

One attack name, three constraint settings

“TextFooler” covers at least three configurations:

TextAttack’s source comment derives 0.840845057 as 1 − 0.5/π, correcting the released code’s missing division by π; that assumes 0.5 on the original scale, not the code’s 0.7 default. Checking each swap against the previous step also lets drift accumulate.

Li et al.’s unified defense benchmark (EMNLP 2021) pins the dial explicitly: USE similarity of at least 0.84 “following the setting in” Jin et al., 50 candidates, at most 30% (AG News) or 10% (IMDB) of words changed, and 50 queries per input word.

What the human studies found

Reevaluating Adversarial Examples in Natural Language (Morris et al., Findings of EMNLP 2020) measured the gap. For each attack and dataset, 100 successful examples got 10 Mechanical Turk ratings each on whether meaning was preserved, on a 1-to-5 scale with 4 as the proposed bar. TextFooler averaged 3.28, the genetic attack of Alzantot et al. 2.70. LanguageTool found more grammatical errors in the adversarial text than the original for 50% of TextFooler’s perturbations, and judges spotted computer-altered MR text with 69.2% accuracy.

Their tightened TFAdjusted required word-embedding cosine of at least 0.9, cosine of at least 0.98 on a BERT sentence encoder fine-tuned for semantic similarity (where judges on average agreed meaning survived), and a LanguageTool grammar check. Success against BERT:

AdvGLUE’s builders found the same from the other side: around 90% of the examples their 14 attack methods produced changed the meaning or misled human annotators too.

The constraint stack, side by side

In vision the constraint is one number; in text it is a stack of heuristics with free parameters, and the stack is what a success rate measures.

Three consequences follow.

The threshold is a dial on the result. Tightening the stack cut TextFooler’s success rate against BERT by 71 to 88 points; a rate reported without its stack compares with nothing.

The filters are not calibrated to human judgment. None of the four rows asks whether a reader would keep the same label, and only a human study says whether a number reflects label agreement or preserved meaning.

The stack is part of the threat model. Fix it before measuring, write it into the evaluation spec, and report it with every number, the way a robust-accuracy figure carries its epsilon.

Worked example: turn the dial on one review

Jin et al.’s example table includes a negative MR review: “The characters, cast in impossibly contrived situations, are totally estranged from reality.” Against a WordLSTM, not BERT, TextFooler swapped in “engineered circumstances” and “fully” and flipped the label to positive. At 12 words, the review sits under the 15-word window.

Install with pip install "textattack[tensorflow]" (the USE constraint needs tensorflow_hub); TextAttack 0.3.11 (August 14, 2026) needs Python 3.9 or later. textattack attack --recipe textfooler --model bert-base-uncased-mr --num-examples 100 gives the stock baseline at dataset scale. dial.py rebuilds that recipe with its thresholds as arguments, reusing its stopword list and omitting only the NLI premise constraint.

# dial.py (TextAttack 0.3.11). Usage: python dial.py USE_MIN skip|noskip EMB_MIN
import math, sys
import numpy as np
import transformers
from textattack import Attack
from textattack.attack_recipes import TextFoolerJin2019
from textattack.constraints.grammaticality import PartOfSpeech
from textattack.constraints.pre_transformation import RepeatModification, StopwordModification
from textattack.constraints.semantics import WordEmbeddingDistance
from textattack.constraints.semantics.sentence_encoders import UniversalSentenceEncoder
from textattack.goal_functions import UntargetedClassification
from textattack.models.wrappers import HuggingFaceModelWrapper
from textattack.search_methods import GreedyWordSwapWIR
from textattack.shared import WordEmbedding
from textattack.transformations import WordSwapEmbedding

use_min, skip, emb_min = float(sys.argv[1]), sys.argv[2] == "skip", float(sys.argv[3])
TEXT = "The characters, cast in impossibly contrived situations, are totally estranged from reality."
NAME = "textattack/bert-base-uncased-rotten-tomatoes"  # CLI alias: bert-base-uncased-mr
model = HuggingFaceModelWrapper(
    transformers.AutoModelForSequenceClassification.from_pretrained(NAME),
    transformers.AutoTokenizer.from_pretrained(NAME))

stock = TextFoolerJin2019.build(model)
stopwords = next(c for c in stock.pre_transformation_constraints
                 if isinstance(c, StopwordModification))
attack = Attack(
    UntargetedClassification(model),
    [RepeatModification(), stopwords,
     WordEmbeddingDistance(min_cos_sim=emb_min),
     PartOfSpeech(allow_verb_noun_swap=True),
     UniversalSentenceEncoder(threshold=use_min, metric="angular",
                              compare_against_original=False, window_size=15,
                              skip_text_shorter_than_window=skip)],
    WordSwapEmbedding(max_candidates=50),
    GreedyWordSwapWIR(wir_method="delete"))
print(attack)  # the constraint values actually in force

result = attack.attack(TEXT, 0)  # ground truth: negative
print(type(result).__name__, result.goal_function_result_str())
orig, adv = result.original_result.attacked_text, result.perturbed_result.attacked_text
emb = WordEmbedding.counterfitted_GLOVE_embedding()
for i in sorted(orig.all_words_diff(adv)):
    a, b = orig.words[i].lower(), adv.words[i].lower()
    print(f"{a} -> {b}: counter-fitted cosine {emb.get_cos_sim(a, b):.3f}")
u = UniversalSentenceEncoder().encode([orig.text, adv.text])
cos = float(np.dot(u[0], u[1]) / (np.linalg.norm(u[0]) * np.linalg.norm(u[1])))
print(f"{len(orig.words)} words; USE cosine {cos:.3f}; "
      f"angular {1 - math.acos(min(cos, 1.0)) / math.pi:.3f}")

Run it four times and record the result type, each substitution’s counter-fitted cosine, the whole-sentence USE cosine and angular similarity, and the output label with confidence. SkippedAttackResult means the model already misclassifies the review.

Reading the runs:

• B should reproduce A exactly. Below the window size every candidate scores 1, so the raised threshold is never consulted; if B differs, the skip logic changed in your version.
• C is the first run in which USE binds. A failed attack or different swaps mean the stock result depended on the filter being off.
• D bounds each swap. Every surviving substitution must show a counter-fitted cosine of at least 0.9.

Morris et al. applied 0.98 to a BERT-based encoder, not USE, so C and D are analogues of TFAdjusted, not replications. Report the gap between A and D with both stacks; at dataset scale, run both stacks on the same inputs and have people rate a sample of successes.

Benchmarks: AdvGLUE and DecodingTrust

AdvGLUE (Wang et al., NeurIPS 2021) is the closest thing NLP has to RobustBench: 14 attack methods at word, sentence and human-crafted levels across five GLUE tasks (SST-2, QQP, MNLI, QNLI, RTE), keeping an example only when at least four of five annotators agree it keeps the original label. Every model it tested scored far below its benign accuracy:

Robust training bought little: SMART on BERT scored 30.29, below plain BERT. The leaderboard’s newest entry is that July 2022 submission; the public dev split, 738 examples in six configurations, is on Hugging Face as AI-Secure/adv_glue.

DecodingTrust (Wang et al., NeurIPS 2023) measured the GPT rows on the AdvGLUE test set and cites 65.77% for the leaderboard’s best model (the page shows 65.45). The largest drop from benign accuracy across prompt templates was 14.43% for GPT-3.5 and 9.90% for GPT-4, against 26.89% for that model; AdvGLUE predates both, so these are transfer results. AdvGLUE++ generates fresh attacks against Alpaca-7B, Vicuna-13B and StableVicuna-13B with TextBugger, TextFooler, BERT-ATTACK, SememePSO and SemAttack: SemAttack text built on Alpaca reached 89.2% attack success against GPT-4 on QQP, and BERT-ATTACK text built on Vicuna 100% against GPT-3.5 on MNLI-mm. The files are under data/adv-glue-plus-plus/ in the DecodingTrust repository.

A benchmark score is still a claim about an attack; evaluating adversarial robustness without fooling yourself applies, with the constraint stack added to the threat model.

Defenses compared

Text defenses train against attacks, certify against a defined set, or detect perturbed inputs. As with randomized smoothing on images, a certificate is exact about its set and silent outside it.

SAFER smooths by replacing each word at random from its perturbation set; RanMASK masks a fraction of words, so it needs no synonym list. Figures marked Li et al. are accuracies at one shared attack budget; that benchmark’s strongest defense was FreeLB++, a modified adversarial training method, at 93.2% clean and 45.3% under TextFooler on IMDB versus 92.1% and 10.3% for plain BERT.

Character-level attacks and a Unicode hardening recipe

White-box character attacks such as HotFlip (Ebrahimi et al., ACL 2018) choose flips from the gradients of one-hot inputs; the ones that matter in deployment need no gradients. Bad Characters (Boucher et al., IEEE S&P 2022) uses four encoding tricks that leave rendered text unchanged: invisible characters (ZWSP, ZWNJ, ZWJ), homoglyphs from Unicode’s confusables data, reorderings through bidi controls (RLO, LRO, PDF, LRI, PDI), and deletions through backspace, DEL and carriage return. One injection significantly degraded vulnerable models and three broke most, including commercial systems from Microsoft and Google.

The sanitizer pins its data files to the Unicode version Python ships (3.14’s unicodedata is UCD 16.0.0), since UTS #39 warns that skeletons are not stable across versions. Run it identically in training and inference.

import unicodedata, urllib.request

UCD = "16.0.0"  # = unicodedata.unidata_version on Python 3.14

def rows(url):
    for line in urllib.request.urlopen(url).read().decode("utf-8-sig").splitlines():
        line = line.split("#", 1)[0].strip()
        if line:
            yield [field.strip() for field in line.split(";")]

IGNORABLE = set()
for cps, prop, *_ in rows(f"https://www.unicode.org/Public/{UCD}/ucd/DerivedCoreProperties.txt"):
    if prop == "Default_Ignorable_Code_Point":
        lo, _, hi = cps.partition("..")
        IGNORABLE.update(map(chr, range(int(lo, 16), int(hi or lo, 16) + 1)))

PROTOTYPE = {chr(int(src, 16)): "".join(chr(int(cp, 16)) for cp in dst.split())
             for src, dst, *_ in rows(f"https://www.unicode.org/Public/security/{UCD}/confusables.txt")}

# UAX #9 embeddings, overrides, isolates and implicit marks
BIDI = set(map(chr, [0x202A, 0x202B, 0x202C, 0x202D, 0x202E, 0x2066,
                     0x2067, 0x2068, 0x2069, 0x200E, 0x200F, 0x061C]))

def sanitize(text):
    """Model input plus flags. A flag means block or review, not cleaned."""
    flags = set()
    if "\x08" in text or "\x7f" in text or "\r" in text.replace("\r\n", ""):
        flags.add("deletion-control")
    if BIDI & set(text):
        flags.add("bidi-control")
    text = "".join(ch for ch in text if ch not in IGNORABLE)
    return unicodedata.normalize("NFKC", text), flags

def skeleton(text):
    """UTS #39 internalSkeleton: a comparison key, never model input."""
    text = unicodedata.normalize("NFD", text)
    text = "".join(PROTOTYPE.get(ch, ch) for ch in text if ch not in IGNORABLE)
    return unicodedata.normalize("NFD", text)

UTS #39, now at version 18.0.0, also defines a fuller skeleton that first reorders text for display with the bidirectional algorithm, which Python’s standard library does not implement. The heavyweight alternative is Boucher et al.’s: render the text and OCR it back, which they report fully prevents invisible-character, reordering and deletion attacks and strongly mitigates most homoglyphs, at 6.2% lower baseline BLEU.

Test it the way it will be attacked. TextAttack’s bad-characters recipe (BadCharacters2021) covers all four classes through perturbation_type (homoglyphs, invisible, deletions, reorderings) with differential evolution. Wrap the classifier so every query passes through sanitize(), attack raw and wrapped models at perturbs=1 and 3, and record per class the success rate and the share of successful inputs that raised a flag, which counts as a block.

Prompt injection and GCG, briefly

Prompt injection and optimized suffixes are adversarial text judged on behavior rather than a label, with no pretense of imperceptibility. GCG (Zou et al., 2023) combines greedy and gradient-based token search to find a suffix that maximizes the probability of an affirmative response; suffixes trained on Vicuna-7B and 13B transferred to the public interfaces of ChatGPT, Bard and Claude. Mechanics and cost are in how the GCG optimizer finds adversarial suffixes; retrieval and memory as the delivery path, in stored prompt injection through agent memory.

Decision procedure: what to test for your deployment

Match the deployment to a perturbation class, fix the constraints, then pick the tool; adversarial attack libraries covers where TextAttack fits.

Whatever the row, write the stack into the evaluation spec before the first run, run stock and tightened stacks on the same inputs, have people rate a sample of successes on the 1-to-5 meaning scale, and report robust accuracy with the stack attached. Imperceptible-encoding attacks skip the rating: render and compare.

FAQ

is there a robustbench for nlp?

The closest equivalent is AdvGLUE, which applies 14 attack methods to five GLUE tasks and keeps only examples that human annotators validated. Its leaderboard’s newest entry dates from July 2022, so for LLMs add DecodingTrust’s AdvGLUE and AdvGLUE++ results and your own evaluation under a written constraint stack.

does unicode normalization stop homoglyph attacks?

Only partly. NFKC folds compatibility variants such as ℌ to H or ① to 1, but cross-script lookalikes such as Cyrillic а survive because they are distinct letters. Compare UTS #39 confusable skeletons to catch those, and treat invisible characters, bidi reorderings and deletion controls as separate steps, since NFKC touches none of them.

can nlp models be certified robust?

Yes, within a stated perturbation set. IBP certifies every combination of swaps from a fixed neighbor list, SAFER certifies substitutions from a defender-known synonym set by randomized smoothing, and RanMASK certifies edits to a few words with no synonym list, over half of texts at five words on AG News. None certifies paraphrase.