NeurIPS 2026 · Poster

Backdoor Purification for LoRA-Tuned LLMs via Null-Space Projection

Jianwei Li  ·  Jung-Eun Kim

A weight-space defense that projects suspect LoRA updates away from estimated backdoor directions without retraining the suspect adapter.

Read the paper OpenReview Code

01 / Overview

Can we remove a backdoor from a LoRA update?

A LoRA adapter can carry both useful skills and a hidden association between a trigger and an attacker’s intended behavior. The defense uses the adapter’s weight update as the point of intervention.

The method uses independently constructed backdoored variants to estimate directions associated with a specified behavior. It then projects the suspect update away from those directions, aiming to reduce attack success while preserving useful behavior.

02 / Method

From reference variants to a purified update

  1. STEP 01

    Construct reference variants

    Train LoRA variants for the behavior of interest to provide a reference for estimating backdoor directions.

  2. STEP 02

    Estimate backdoor directions

    Aggregate directions across variants and construct orthogonal complements in the input and output channels.

  3. STEP 03

    Project the suspect update

    Apply the projectors to the LoRA update. The suspect adapter itself does not require gradient-based retraining.

03 / Selected result

Attack success fell while measured utility held

LLaMA2-7B-Chat on sentiment steering, averaged over the five attacks in Table 2.

Attack success rate100.0% → 2.8%Lower is better
Downstream utility (GSM8K)27.81 → 27.93Higher is better

Before → after purification. The composite-trigger attack (CTBA) uses an attack-specific rank-4 aggregation; the other rows use consensus aggregation. See Table 2 and Appendix F.7 in the paper.

04 / Scope

What the defense assumes

The defender has access to the suspect LoRA adapter and its corresponding base model, and specifies the behavior class to defend against. The method does not require knowing the exact trigger or target, or obtaining a clean counterpart to the suspect adapter.

The experiments support transfer across attacks within a behavior class. The paper also reports a failed transfer across behavior classes, so projectors should not be assumed to generalize to arbitrary behaviors.

05 / Citation

Cite this work

BibTeX
@inproceedings{li2026backdoor,
  title = {Backdoor Purification for LoRA-Tuned LLMs via Null-Space Projection},
  author = {Li, Jianwei and Kim, Jung-Eun},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2026},
  url = {https://openreview.net/forum?id=NAtwr6xj7t}
}