Gradient Inversion Attacks in Federated Learning

Gradient inversion reconstructs a federated-learning client’s training data from the model update it sends. It works at realistic settings: published attacks rebuild ImageNet images from gradients averaged over 48 examples, recover about half of a client’s images from ten-epoch FedAvg updates, and recover batches of 128 sequences exactly from LLM gradients.

Those results assume an honest-but-curious server that follows the protocol and only looks. A server that alters the model it ships, or picks who shares a round with its target, extracts more, and several such attacks get around secure aggregation without breaking its cryptography.

Gradient inversion is the federated branch of model inversion, which OWASP lists as ML03:2023. Its siblings invert a prediction API’s confidence scores or a stored text embedding, and are compared at the end. Gradient inversion needs only one client update, which is exactly what a federated server receives every round.

How a gradient gives up its inputs

Every model-inversion attack runs a trained model backwards by search: find an input that reproduces an output the model produced. Fredrikson et al. named the attack in a 2014 warfarin-dosing study that predicted patients’ genetic markers from the model and demographic data. Against confidence scores it is the adversarial example loop reversed, pushing noise toward what the model learned for one class.

Deep Leakage from Gradients (Zhu et al., 2019) aimed that search at federated updates. A server holding the weights W it shipped and one client’s update ∇W proceeds like this:

1. Draw a dummy input x′ and dummy label y′ from Gaussian noise.
2. Run x′ through the model at W, take the loss against y′, and differentiate with respect to W.
3. Measure the squared distance between the resulting gradient and ∇W.
4. Differentiate that distance with respect to x′ and y′, not W, and step both. This needs second derivatives, so the model must be twice differentiable.
5. Repeat until the gradients match; x′ then approximates the private data. DLG used L-BFGS, with 1,200 iterations for images and 100 for text.

x*, y* = argmin_{x', y'} || ∂L(F(x'), y') / ∂W - ∇W ||²

Three later results made it practical.

Match direction, not magnitude. Geiping et al. (2020) replaced the Euclidean distance with cosine similarity and added a total-variation prior. Trained networks produce much smaller gradients than untrained ones, and the magnitude-oblivious objective still recovers visual information from their direction.

Linear layers leak analytically. The same paper proves that the input to a fully connected layer can be computed exactly from that layer’s parameter gradients, whatever the rest of the architecture, provided the loss derivative with respect to the layer’s output has one non-zero entry. Most of the malicious-server attacks below build on this.

Labels come almost free. iDLG showed that under cross-entropy with one-hot labels, the signs of the last layer’s gradient reveal a single example’s label. GradInversion extended label restoration to batches and used the target batch’s BatchNorm statistics, which distributed setups can share or leak, as an image prior.

What the published attacks recovered

Reported numbers, with the conditions that produced them (HbC: honest-but-curious server):

Dimitrov et al. count a reconstruction as successful above 20 dB PSNR on FEMNIST and 19 dB on CIFAR-100.

Rows are not interchangeable. PSNR on an untrained 32×32 ConvNet and identifiability on a 224×224 ResNet-50 measure different things. A leakage claim needs resolution, batch size, update rule and training state quoted together; missing any of them, it is a fragment.

Local training does not rescue FedAvg. Geiping et al. concluded that “even averaging gradients over several iterations or several images does not protect the user’s privacy”, and reported reconstruction quality unimpeded after 100 local steps. Dimitrov et al.’s attack simulates the client’s local epochs instead of treating the update as one gradient, which lifts recovery from at most 10.6% to about half. More epochs over the same 50 images helped the attacker (CIFAR-100 at batch 5: 5.0% after one epoch, 46.7% after five). Among ten-epoch schedules, 500 single-image steps hurt it most: 21.4% on FEMNIST, 6.7% on CIFAR-100.

Labels outlast images. GradInversion restored 97.88% of labels at batch 96, twice the largest batch at which it reported image recovery. For a health or finance classifier, the labels in a client’s batch can be the sensitive part on their own.

What moves the numbers

Batch size raises the attacker’s cost rather than closing the attack, and teams set it for convergence and hardware, so a privacy argument resting on it can change for unrelated reasons.

Architecture helps less than intuition suggests. Geiping et al. found reconstruction quality increased with network width, which “does not provide greater security”, while depth degraded the attack “very little”. Fully connected layers are the weak point, per the analytic result above, and training the model lowers PSNR without stopping the leak.

Compression and precision are not privacy controls. DLG’s authors found half-precision gradients failed to protect the data, and that pruning beyond roughly 20% of gradient entries made their reconstructions unrecognizable. That measured one attack, and no ε attaches to a pruning ratio. As with robustness evaluation, a defense checked only against the attack it was tuned on has not been tested.

When the server stops being honest

Everything above assumes the server runs the protocol as written. Five papers drop that assumption, and they change what secure aggregation (SA) and differential privacy (DP) can promise.

Robbing the Fed needs no optimization. The imprint layer sorts inputs into bins by a statistic such as brightness, and a bin holding one image returns it from the gradient. “With only 156 bins, an attacker can expect to recover over 50% of a batch of 64 images perfectly”; on ImageNet at batch 64 the average PSNR was 75.75. Split into 8 local updates, the batch still scored 70.31% image identifiability.

Fishing edits only the classifier layer, so one example dominates the update. The authors retrieved an individual ImageNet image from an aggregate of 256 after 8 queries on average.

Model inconsistency exploits an assumption inside SA: that honest clients’ inputs are beyond the server’s reach. In FL the server writes the model those inputs come from. Non-target clients get parameters that drive a ReLU layer dead, their updates return as zeros, and the “aggregate” is the target’s gradient. The authors’ fixes: clients compare signed copies of the parameters they received, or SA’s masks are derived from the received model so that mismatches produce random-looking masks.

Trap weights and Sybils. Boenisch et al. perfectly extract more than 50% of data points from ImageNet mini-batches of 100 by rescaling weights. Their follow-up attacks SA plus distributed DP: in a 50-client round with 20 data points per client, Sybils surrounding the target let the server extract most of its data points individually, a share that fell as honest clients were added. Their advice: “users only participate in FL protocols that are orchestrated by a trusted server.”

Text and LLM gradients in federated fine-tuning

Text looked safer because tokens are discrete and gradient matching works in continuous space, the same discreteness problem that separates text adversarial examples from image ones. Three attacks closed the gap for an honest-but-curious server:

DAGER does not optimize at all. Self-attention projection gradients are low-rank linear combinations of the input embeddings, so the attacker tests whether a candidate token’s embedding lies in the gradient’s span and assembles exact sequences. Its stated condition is a batch with fewer non-padding tokens than the model’s embedding width: d = 768 for GPT-2, 4,096 for LLaMa-2 7B. DAGER also recovered a 512-token ECHR sequence perfectly from GPT-2, and reached about 95% ROUGE on LLaMa-3.1 8B fine-tuned with LoRA at rank 256.

For federated LLM fine-tuning, compare tokens per client batch with the model’s hidden width; if the first is smaller, your batches sit inside DAGER’s operating condition, LoRA or not. Malicious servers have a text attack too: Decepticons (Fowl et al., ICLR 2023) plants malicious transformer parameters and recovers text across mini-batches, multiple users and long sequences.

Worked example: measure your own leakage with breaching

Geiping’s breaching library (MIT licence) implements many of the attacks above against fixed checkpoints, with user-level DP and aggregation as its only defenses. The procedure below measures a CIFAR-10 ResNet-18 at batch 1, 8 and 32 under FedSGD and FedAvg.

1. Install and check. Clone the repository, install it per its setup notes, and confirm with python simulate_breach.py dryrun=True.

2. FedSGD sweep. The 1_single_image_small case is CIFAR-10 with an honest-but-curious server. Override model and batch, and withhold labels so the attack must recover them:

for B in 1 8 32; do
  python benchmark_breaches.py name=r18_fedsgd_b$B \
    case=1_single_image_small case.model=resnet18 \
    case.user.num_data_points=$B case.user.provide_labels=False \
    attack=invertinggradients attack.regularization.total_variation.scale=1e-3 \
    num_trials=10
done

3. FedAvg sweep. Start from 4_fedavg_small_scale with its dataset swapped to CIFAR-10, as the repository’s FedAvg notebook does. Each client trains E local epochs at full local batch, so num_local_updates equals E (for mini-batches of m, set num_data_per_local_update_step to m and num_local_updates to E × B / m):

for B in 1 8 32; do for E in 1 5; do
  python benchmark_breaches.py name=r18_fedavg_b${B}_e$E \
    case=4_fedavg_small_scale case/data=CIFAR10 case.model=resnet18 \
    case.user.num_data_points=$B case.user.num_data_per_local_update_step=$B \
    case.user.num_local_updates=$E case.user.provide_labels=False \
    attack.regularization.total_variation.scale=1e-3 num_trials=10
done; done

4. Defense runs. Append these user options to any setting:

    case.user.local_diff_privacy.per_example_clipping=1.0 \
    case.user.local_diff_privacy.distribution=gaussian \
    case.user.local_diff_privacy.gradient_noise=0.125

The FedSGD user clips each example, averages, and adds noise once, with gradient_noise as the standard deviation. DP-SGD (Abadi et al.) adds N(0, σ²C²) to the sum of clipped gradients before dividing by the batch size, so gradient_noise = σ·C/B; 0.125 is σ = 1, C = 1, B = 8. The FedAvg user clips each local step’s batch gradient, not each example, so treat those runs as clipped-and-noised FedAvg and compute ε with your own accountant.

5. Record and read. Keep the METRICS line each trial logs (PSNR, SSIM, LPIPS, image identifiability, label accuracy) with the full config. Count a reconstruction as a leak above roughly 19 to 20 dB PSNR, Dimitrov et al.’s thresholds, and read label accuracy that stays high after PSNR falls as label leakage. One trap: breaching’s pretrained flag only loads weights on its ImageNet path, so this ResNet-18 is untrained, the easier target per Geiping’s Table 1. Swap in your trained model (simulate_breach.py marks the line) before drawing production conclusions.

What each defense buys, and against whom

Secure aggregation (Bonawitz et al., CCS 2017) lets the server learn only the sum of many clients’ updates, which removes the individual gradient every honest-but-curious attack needs. It costs pairwise masking every round and a recovery path for clients that drop out. Against a malicious server it fails without being broken, because the server shapes the inputs, fills the cohort, or attacks the sum.

Differential privacy is the only family that attaches a number to the guarantee, and where the noise is added decides whom it protects against. Central DP, added by the server after aggregation, protects the released model from its users, not clients from the server. Local DP, added on the device, protects against the server at a steep utility cost. Distributed DP splits the noise across clients inside SA and holds only while the server cannot stuff the cohort. Ad hoc noise is no substitute: in DLG’s tests, added noise stopped the attack only above a variance of 10⁻², where accuracy began to suffer. Clipping without calibrated noise gives no formal guarantee.

Homomorphic encryption and TEEs. HE-based aggregation computes over encrypted updates. Boenisch et al. suggest trusted execution environments against active manipulation, with a side-channel caveat.

So “secure aggregation plus DP” holds against a server that follows the protocol. Against one that can change the model or the cohort, you need client-side noise with a cohort the server cannot fill, a way for clients to verify what they receive, or a server you are prepared to trust.

A decision procedure for federated deployments

1. Do raw per-client updates reach any party? If no gradient or model delta is ever shared, gradient inversion does not apply; look at the confidence-score, embedding and membership attacks in the next section instead. If updates are shared, list every party that sees an individual one (aggregator, cross-silo coordinator, update logs, debug dumps) and treat each as at least honest-but-curious.

2. Does your framework turn secure aggregation on? Check the running configuration. In Flower, SecAgg+ needs SecAggPlusWorkflow on the server and secaggplus_mod on every client; without them the server receives each update individually. TensorFlow Federated’s build_weighted_fed_avg falls back to tff.aggregators.MeanFactory unless you pass model_aggregator=tff.learning.secure_aggregator(), which sums with tff.federated_secure_sum_bitwidth. NVIDIA FLARE offers homomorphic encryption for secure aggregation, and its privacy filters are set per job in config_fed_client.json and config_fed_server.json or for all jobs through a site privacy policy.

3. What is the minimum cohort, and how is it enforced server-side? Flower’s FedAvg defaults to min_train_nodes=2, and a two-client sum hides nothing from a server that controls one of the two. In SecAgg+, a “higher reconstruction_threshold means better privacy guarantees but less tolerance to dropouts”. Enforce the minimum and log each round’s cohort, but after the Sybil result a minimum is only as strong as client admission, which needs authentication that someone other than the server operator can audit.

4. Can clients verify the model they receive? If clients cannot confirm that every participant got identical parameters, assume model inconsistency is possible. Pasquini et al.’s fixes stop inconsistency but not a consistently malicious model, and Boenisch et al. note there is no fool-proof way to tell manipulated weights from legitimately trained ones.

5. What ε do you target, and who adds the noise? The most detailed production reference is Google’s Gboard deployment (Xu et al., 2023): more than twenty language models trained with DP-FTRL at ρ-zCDP between 0.2 and 2, which the paper converts to ε from 4.49 to 13.69 at δ = 10⁻¹⁰. These are central, device-level guarantees with the provider trusted to implement the mechanism, and only two of the models also used SA. A Gboard-style ε does not cover a server in your threat model; that needs noise added before updates leave the device. TFF’s dp_aggregator docs say a noise multiplier of 1.0 or higher “may be needed for meaningful privacy”, and Flower labels its DP support a preview.

Gradient inversion vs. other inversion and privacy attacks

When no update leaves the device, two other inversion routes need only a deployed model’s outputs.

Confidence-score inversion is the Fredrikson et al. (2015) attack: query a model that returns class probabilities and climb toward the input that maximizes the target class’s score, which yields class-level reconstructions. It feeds on confidence values and spends queries at every step, so two cheap controls apply to classification APIs: return the top label without its probability, and flag clients sending long runs of synthetic inputs aimed at one class. Neither gives a formal bound. DP training does, but the 2014 warfarin study found that budgets strong enough to stop its attack raised simulated patients’ risk of stroke, bleeding and mortality.

Embedding inversion reads text back out of a vector. Vec2Text (Morris et al., EMNLP 2023) generates a guess, re-embeds it, and corrects it toward the target over repeated rounds. The authors built inverters for GTR-base and OpenAI’s ada-002 embeddings; the method recovered 92% of 32-token inputs exactly (BLEU 97.3) and 89% of full names from a dataset of clinical notes. The attacker needs one target vector plus query access to the embedding model, which every customer of a hosted embedding API has. Treat stored and returned embeddings like the source text: same access rules, and no vectors returned for data a client did not submit. Morris et al. found Gaussian noise at λ = 0.01 cut retrieval performance by about 2% while reconstruction fell to 13% of its original BLEU, a result against one attack, like DLG’s pruning test.

Membership inference (Shokri et al., 2017) asks whether a specific record was in the training set; inversion asks what the training data looked like, a reconstruction problem with different defenses and regulatory consequences. Carlini et al.’s training data extraction work on GPT-2 sits between them: it extracts verbatim text, like inversion, but uses a membership signal to pick out successful extractions. Zhou et al.’s 2024 survey covers inversion attacks and defenses across images, text and graphs.

FAQ

how do model inversion attacks work?

They search for an input that makes a trained model reproduce something it revealed. The attacker starts from noise or a rough guess and adjusts it until the model’s confidence scores, gradient or embedding match the observed ones. In federated learning that signal is a client’s gradient, and matching it rebuilds training images or text.

does secure aggregation prevent gradient inversion?

Only against a server that follows the protocol. Secure aggregation hides individual updates from an honest-but-curious server, but published attacks defeat it without breaking the cryptography: sending different models to different clients, filling a round with server-controlled Sybil devices, or magnifying one example inside the aggregate. Pair it with model verification and cohort controls.

does differential privacy stop gradient inversion attacks?

Yes, if the noise is added where the attacker cannot see around it. Noise added on the client bounds what any single update reveals, at an accuracy cost. Central DP, where the server adds noise after aggregation, protects the released model but not clients from the server, which sees updates before the noise.

can gradient inversion recover text from llm fine-tuning?

Yes. DAGER, published at NeurIPS 2024, recovered batches of up to 128 sequences exactly from LLaMa-2 7B gradients, with ROUGE scores above 0.99. It also worked on FedAvg updates and LoRA fine-tuning. Its stated limit is that a batch must hold fewer tokens than the model’s embedding width.

is fedavg safer than fedsgd against gradient inversion?

Not against an attack built for it. Dimitrov et al. simulated local training and reconstructed, on average, about half of a client’s images from ten-epoch FedAvg updates on FEMNIST and CIFAR-100, while attacks assuming a single gradient recovered at most 10.6%. Many tiny local steps reduced recovery, but never to zero.