Bio

I will join UMass Amherst CICS as an Assistant Professor in Fall'24. This year I am spending at Google Research with Peter Kairouz. Recently, I completed my PhD at Cornell Tech advised by Vitaly Shmatikov and Deborah Estrin. My research was recognized by Apple Scholars in AI/ML and Digital Life Initiative fellowships.

My work focuses on security and privacy in emerging AI-based systems under real-life conditions and attacks. During my PhD I worked on backdoor attacks in federated learning and fairness trade-offs with differential privacy. We also proposed new frameworks Backdoors101 and Mithridates to advance research on backdoors and a new attack on generative language models covered by VentureBeat and The Economist. Recently, I have studied vulnerabilities in multi-modal systems: instruction injections and adversarial illusions.

A big focus of my work is data privacy. I worked on Ancile – a framework for language-level control over data usage. At Google, I designed a new algorithm for building private heatmaps. At Apple, I developed a novel way to obtain good tokenizers for private federated learning. Before starting my PhD, I received an engineering degree from Baumanka and worked at Cisco as a software engineer.

I grew up in Tashkent. In my free time I play water polo and spend time with family.

For prospective students: please consider applying to UMass CS PhD. I am looking for students for Fall 2024, so reach out, but provide some concrete ideas on security and privacy that you are interested in.

Papers

• auto_fix_highCeci n'est pas une pomme: Adversarial Illusions in Multi-Modal Embeddings Preprint'23
  Eugene Bagdasaryan, Vitaly Shmatikov

  We propose adversarial alignment in cross-modal settings.

  Work done at Google. [PDF], [Code].
• forum(Ab)using Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs Preprint'23
  Eugene Bagdasaryan, Tsung-Yin Hsieh, Ben Nassi, Vitaly Shmatikov

  A new prompt injection attack that makes LLMs talk like pirates when they see an image or an audio.

  [PDF] [Code]
• scienceoffMithridates: Boosting Natural Resistance to Backdoor Learning Preprint'23
  Eugene Bagdasaryan, Vitaly Shmatikov

  A novel audit method for poisoning attacks.

  [PDF] [Code]
• mapTowards Sparse Federated Analytics: Location Heatmaps under Distributed Differential Privacy with Secure Aggregation PETS'22
  Eugene Bagdasaryan, Peter Kairouz, Stefan Mellem, Adrià Gascón, Kallista Bonawitz, Deborah Estrin, and Marco Gruteser

  A new algorithm for building heatmaps with local-like differential privacy.

  Work done at Google. [PDF] [Code]
• campaignSpinning Language Models: Risks of Propaganda-as-a-Service and Countermeasures S&P'22
  Eugene Bagdasaryan and Vitaly Shmatikov

  We discover new capabilities of large language models to express attacker-chosen opinions on certain topics while performing tasks like summarization, translation, and language generation.

  [PDF] [Code]
• spellcheckTraining a Tokenizer for Free with Private Federated Learning FL4NLP@ACL'22
  Eugene Bagdasaryan, Congzheng Song, Rogier van Dalen, Matt Seigel, and Áine Cahill

  Tokenization is an important part of training a good language model, however in private federated learning where user data are not available generic tokenization methods reduce performance. We show how to obtain a good tokenizer without spending additional privacy budget.

  Work done at Apple. Best paper runner-up award. [PDF]
• visibility_off Blind Backdoors in Deep Learning Models USENIX'21
  Eugene Bagdasaryan and Vitaly Shmatikov

  We propose a novel attack that injects complex and semantic backdoors without access to the training data or the model and evades all known defenses.

  [PDF] [Code]
• smartphoneHow To Backdoor Federated Learning AISTATS'20
  Eugene Bagdasaryan, Andreas Veit, Yiqing Hua, Deborah Estrin, and Vitaly Shmatikov

  We introduce a constrain-and-scale attack, a form of data poisoning, that can stealthily inject a backdoor into one of the participating models during a single round of Federated Learning training. This attack can avoid proposed defenses and propagate the backdoor to a global server that will distribute the compromised model to other participants.

  [PDF] [Code]
• local_hospitalSalvaging Federated Learning by Local Adaptation Preprint
  Tao Yu, Eugene Bagdasaryan, and Vitaly Shmatikov

  Recovering participants' performance on their data when using federated learning with robustness and privacy techniques.

  [Paper] [Code]
• doneAncile: Enhancing Privacy for Ubiquitous Computing with Use-Based Privacy WPES'19
  Eugene Bagdasaryan, Griffin Berlstein, Jason Waterman, Eleanor Birrell, Nate Foster, Fred B. Schneider, and Deborah Estrin

  A novel platform that enables control over application's data usage with language level policies and implementing use-based privacy.

  [PDF] [Code] [Slides]
• faceDifferential Privacy Has Disparate Impact on Model Accuracy NeurIPS'19
  Eugene Bagdasaryan and Vitaly Shmatikov

  This project discusses a new trade off between privacy and fairness. We observe that training a Machine Learning model with Differential Privacy reduces accuracy on underrepresented groups.

  [NeurIPS, 2019], [Code].
• memoryX-containers: Breaking down barriers to improve performance and isolation of cloud-native containers ASPLOS'19
  Zhiming Shen, Zhen Sun, Gur-Eyal Sela, Eugene Bagdasaryan, Christina Delimitrou, Robbert Van Renesse, and Hakim Weatherspoon

  A fast and compact cloud-native implementation of containers.

  [PDF].
• extensionOpenrec: A modular framework for extensible and adaptable recommendation algorithms WSDM'18
  Longqi Yang, Eugene Bagdasaryan, Joshua Gruenstein, Cheng-Kang Hsieh, and Deborah Estrin

  An open and modular Python framework that supports extensible and adaptable research in recommender systems.

  [PDF], [Code].