Semantic-Guided Latent Space Backdoor Attack: a Novel Threat to Stable Diffusion

EasyChair Preprint no. 13979

Abstract

Stable Diffusion (SD) models have achieved remarkable success in text-to-image synthesis, but their security vulnerabilities remain largely unexplored. In this paper, we introduce a novel semantic-guided latent space backdoor attack (SG-LSBA) that leverages the semantic information in the text input to inject stealthy and seman- tically coherent backdoors into SD models. Our approach outperforms existing methods by crafting context-aware semantic triggers, identifying target visual features in the latent space, and employing an adversarial optimization framework. Extensive evaluations demonstrate the high success rates, strong semantic relevance, and exceptional stealthiness of SG-LSBA. Our findings highlight the urgent need for considering the complex interplay between semantics and latent representations in developing robust defenses against backdoor attacks in SD models. We make our code and datasets publicly available to facilitate further research and development of secure and reliable text-to-image synthesis models. The code is available at https://github.com/paoche11/SG-LSBA.

Keyphrases: diffusion models, Latent space backdoor attack, semantic guidance, Semantic triggers, Stable Diffusion models

@Booklet{EasyChair:13979,
  author = {Yu Pan and Yi Du and Lin Wang and Bingrong Dai},
  title = {Semantic-Guided Latent Space Backdoor Attack: a Novel Threat to Stable Diffusion},
  howpublished = {EasyChair Preprint no. 13979},

  year = {EasyChair, 2024}}