Avi Schwarzschild
Avi Schwarzschild
[avis4k@gmail.com] [Google Scholar] [Twitter] [GitHub] [CV]
I am a post-doc at Carnegie Mellon University advised by Zico Kolter. My work focuses on safe and secure ML as well as reasoning in AI systems.
In 2023, I finished my Ph.D. in the Applied Math and Scientific Computation program at the University of Maryland. I was advised by Tom Goldstein on my work in deep learning. My research during my PhD spanned from security to generalization and broadly focused on expanding our understanding of when and why neural networks work. My specific interest in data security and model vulnerability has led to work on adversarial attacks and data poisoning. I also studied neural networks' ability to extrapolate from easy training tasks to more difficult problems at test time.
From June 2022 through March 2023, I was a researcher at Arthur AI in New York City. And before starting at UMD, I received a master's degree in applied math at the University of Washington and a bachelor's degree in applied math at Columbia Engineering.
Transformers Can Do Arithmetic with the Right Embeddings
Sean McLeish, Arpit Bansal, Alex Stein, Neel Jain, John Kirchenbauer, Brian R. Bartoldson, Bhavya Kailkhura, Abhinav Bhatele, Jonas Geiping, Avi Schwarzschild, Tom Goldstein.
Neural Information Processing Systems (NeurIPS), 2024. [Arxiv]
Rethinking LLM Memorization through the Lens of Adversarial Compression
Avi Schwarzschild*, Zhili Feng*, Pratyush Maini, Zachary C Lipton, J Zico Kolter.
Neural Information Processing Systems (NeurIPS), 2024. [Arxiv]
Forcing Diffuse Distributions out of Language Models
Yiming Zhang, Avi Schwarzschild, Nicholas Carlini, J Zico Kolter, and Daphne Ippolito.
Conference on Language Modeling (COLM), 2024. [Arxiv]
TOFU: A Task of Fictitious Unlearning for LLMs
Pratyush Maini*, Zhili Feng*, Avi Schwarzschild*, Zachary C Lipton, J Zico Kolter.
Conference on Language Modeling (COLM), 2024. [Arxiv]
Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text
Abhimanyu Hans*, Avi Schwarzschild*, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, Tom Goldstein.
International Conference on Machine Learning (ICML), 2024. [Arxiv]
NEFTune: Noisy Embeddings Improve Instruction Finetuning
Neel Jain, Ping-yeh Chiang, Yuxin Wen, John Kirchenbauer, Hong-Min Chu, Gowthami Somepalli, Brian R Bartoldson, Bhavya Kailkhura, Avi Schwarzschild, Aniruddha Saha, Micah Goldblum, Jonas Geiping, Tom Goldstein.
International Conference on Learning Representations (ICLR), 2024. [Arxiv]
Baseline Defenses for Adversarial Attacks Against Aligned Language Models
Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping-yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, Tom Goldstein.
Preprint. [Arxiv]
Universal Guidance for Diffusion Models
Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupta, Micah Goldblum, Jonas Geiping, Tom Goldstein.
International Conference on Machine Learning (ICML), 2024. [ArXiv]
Transfer Learning with Deep Tabular Models
Roman Levin, Valeriia Cherepanova, Avi Schwarzschild, Arpit Bansal, C Bayan Bruss, Tom Goldstein, Andrew Gordon Wilson, Micah Goldblum.
International Conference on Learning Representations (ICLR), 2023. [Published Version]
Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses
Micah Goldblum, Dimitris Tsipras, Chulin Xie, Xinyun Chen, Avi Schwarzschild, Dawn Song, Aleksander Madry, Bo Li, Tom Goldstein.
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2022. [ArXiv] [Published Version]
End-to-end Algorithm Synthesis with Recurrent Networks: Logical Extrapolation Without Overthinking
Arpit Bansal*, Avi Schwarzschild*, Eitan Borgnia, Zeyad Emam, Furong Huang, Micah Goldblum, Tom Goldstein.
Neural Information Processing Systems (NeurIPS), 2022. [ArXiv]
SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training
Gowthami Somepalli, Micah Goldblum, Avi Schwarzschild, C Bayan Bruss, Tom Goldstein.
Preprint. [ArXiv]
The Uncanny Similarity of Recurrence and Depth
Avi Schwarzschild*, Arjun Gupta*, Amin Ghiasi, Micah Goldblum, Tom Goldstein.
International Conference on Learning Representations (ICLR), 2022. [Published Version]
Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networks
Avi Schwarzschild, Eitan Borgnia, Arjun Gupta, Furong Huang, Uzi Vishkin, Micah Goldblum, Tom Goldstein.
Neural Information Processing Systems (NeurIPS), 2021. [Published Version]
Adversarial Attacks on Machine Learning Systems for High-Frequency Trading
Micah Goldblum*, Avi Schwarzschild*, Ankit Patel, Tom Goldstein.
International Conference on AI in Finance (ICAIF), 2021. [Published Version]
Just How Toxic is Data Poisoning? A Benchmark for Backdoor and Data Poisoning Attacks
Avi Schwarzschild*, Micah Goldblum*, Arjun Gupta, John Dickerson, Tom Goldstein.
International Conference on Machine Learning (ICML), 2021. [Published Version]
Truth or Backpropaganda? An Empirical Investigation of Deep Learning Theory
Micah Goldblum, Jonas Geiping, Avi Schwarzschild, Michael Moeller, Tom Goldstein.
International Conference on Learning Representations (ICLR), 2020. [Published Version]