- This event has passed.
Efficient and Robust Defense against Query-based Black-box Attacks

Title: Efficient and Robust Defense against Query-based Black-box Attacks
Speaker: Shaofei Li
Date and Time: Monday, May 2119, 11:00am
Location: EH 2430
Abstract:
Query-based black-box attacks have emerged as a significant threat to machine learning systems, where adversaries can manipulate the input queries to generate adversarial examples that can cause misclassification of the system. To counter these attacks, researchers have proposed Stateful Defense Models (SDMs) such as BlackLight and PIHA, which can reject queries that are “similar” to historical queries. However, recent studies show that existing approaches are vulnerable to a stronger adaptive attack, Oracle-guided Adaptive Rejection Sampling (OARS). OARS can be easily integrated with existing attack algorithms to evade the SDMs by generating queries with fine-tuned direction and step size of perturbations utilizing the leaked decision boundary from the SDMs. In this talk, I will present our novel approach, Query Provenance Analysis (QPA), for defending against query-based black-box attacks robustly (against both non-adaptive and adaptive attacks) and efficiently (in real-time). The key insight is that, instead of focusing on individual queries, utilizing features from the query sequence (termed query provenance) can distinguish malicious queries from benign queries more effectively. So it constructs a query provenance graph to capture the relationship between a new query and prior historical queries, and then design efficient algorithms to detect malicious queries based on the query provenance graphs. As a results, QPA outperforms the baselines regarding defense robustness and efficiency on both non-adaptive and adaptive attacks. It can also be deployed for various attack scenarios, such as membership inference attack and jailbreaking.
Biography:
Shaofei Li is currently a fourth-year Ph.D. student in the Department of Computer Science at Peking University, advised by Prof. Ding Li, Prof. Yao Guo and Prof. Xiangqun Chen. He received his B.S. degree from the School of Electronics Engineering and Computer Science (EECS) at Peking University in 2021. His research interests lie in cybersecurity, system security and AI security. His research focuses on developing detection and defense mechanisms for various security threats in the real world systems. He has published several papers in top-tier security conferences and transactions such as S&P, NDSS, CCS and TOSEM. More information can be found at his website: https://shaofei-li.org/
Hosted By: Prof. Zhou Li