Avi Schwarzschild
Avi Schwarzschild
avis4k@gmail.com, [Google Scholar] [Twitter] [GitHub] [CV]
I am a post-doc at Carnegie Mellon University advised by Zico Kolter. My work focuses on safe and secure ML as well as reasoning in AI systems.
In 2023, I finished my Ph.D. in the Applied Math and Scientific Computation program at the University of Maryland. I was advised by Tom Goldstein on my work in deep learning. My research during my PhD spanned from security to generalization and broadly focused on expanding our understanding of when and why neural networks work. My specific interest in data security and model vulnerability has led to work on adversarial attacks and data poisoning. I also studied neural networks' ability to extrapolate from easy training tasks to more difficult problems at test time.
From June 2022 through March 2023, I was a researcher at Arthur AI in New York City. And before starting at UMD, I received a master's degree in applied math at the University of Washington and a bachelor's degree in applied math at Columbia Engineering.
Rethinking LLM Memorization through the Lens of Adversarial Compression. Avi Schwarzschild*, Zhili Feng*, Pratyush Maini, Zachary C Lipton, J Zico Kolter. Preprint. [Arxiv]
Forcing Diffuse Distributions out of Language Models. Yiming Zhang, Avi Schwarzschild, Nicholas Carlini, J Zico Kolter, and Daphne Ippolito. Preprint. [Arxiv]
Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text. Abhimanyu Hans*, Avi Schwarzschild*, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, Tom Goldstein. Preprint. [Arxiv]
TOFU: A Task of Fictitious Unlearning for LLMs. Pratyush Maini*, Zhili Feng*, Avi Schwarzschild*, Zachary C Lipton, J Zico Kolter. Preprint. [Arxiv]
NEFTune: Noisy Embeddings Improve Instruction Finetuning. Neel Jain, Ping-yeh Chiang, Yuxin Wen, John Kirchenbauer, Hong-Min Chu, Gowthami Somepalli, Brian R Bartoldson, Bhavya Kailkhura, Avi Schwarzschild, Aniruddha Saha, Micah Goldblum, Jonas Geiping, Tom Goldstein. Preprint. [Arxiv]
Baseline Defenses for Adversarial Attacks Against Aligned Language Models. Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping-yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, Tom Goldstein. Preprint. [Arxiv]
A Cookbook of Self-Supervised Learning. Randall Balestriero, Mark Ibrahim, Vlad Sobal, Ari Morcos, Shashank Shekhar, Tom Goldstein, Florian Bordes, Adrien Bardes, Gregoire Mialon, Yuandong Tian, Avi Schwarzschild, Andrew Gordon Wilson, Jonas Geiping, Quentin Garrido, Pierre Fernandez, Amir Bar, Hamed Pirsiavash, Yann LeCun, Micah Goldblum. Preprint. [ArXiv]
Neural Auctions Compromise Bidder Information. Alex Stein*, Avi Schwarzschild*, Michael Curry, Tom Goldstein, John Dickerson. Preprint. [ArXiv]
Universal Guidance for Diffusion Models. Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupta, Micah Goldblum, Jonas Geiping, Tom Goldstein. Preprint. [ArXiv]
Transfer Learning with Deep Tabular Models. Roman Levin, Valeriia Cherepanova, Avi Schwarzschild, Arpit Bansal, C Bayan Bruss, Tom Goldstein, Andrew Gordon Wilson, Micah Goldblum. International Conference on Learning Representations (ICLR), 2023. [Published Version]
Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses. Micah Goldblum, Dimitris Tsipras, Chulin Xie, Xinyun Chen, Avi Schwarzschild, Dawn Song, Aleksander Madry, Bo Li, Tom Goldstein. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2022. [ArXiv] [Published Version]
End-to-end Algorithm Synthesis with Recurrent Networks: Logical Extrapolation Without Overthinking. Arpit Bansal*, Avi Schwarzschild*, Eitan Borgnia, Zeyad Emam, Furong Huang, Micah Goldblum, Tom Goldstein. Neural Information Processing Systems (NeurIPS), 2022. [ArXiv]
SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training. Gowthami Somepalli, Micah Goldblum, Avi Schwarzschild, C Bayan Bruss, Tom Goldstein. Under Review. [ArXiv]
The Uncanny Similarity of Recurrence and Depth. Avi Schwarzschild*, Arjun Gupta*, Amin Ghiasi, Micah Goldblum, Tom Goldstein. International Conference on Learning Representations (ICLR), 2022. [Published Version]
Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networks. Avi Schwarzschild, Eitan Borgnia, Arjun Gupta, Furong Huang, Uzi Vishkin, Micah Goldblum, Tom Goldstein. Neural Information Processing Systems (NeurIPS), 2021. [Published Version]
Adversarial Attacks on Machine Learning Systems for High-Frequency Trading. Micah Goldblum*, Avi Schwarzschild*, Ankit Patel, Tom Goldstein. International Conference on AI in Finance (ICAIF), 2021. [Published Version]
Just How Toxic is Data Poisoning? A Benchmark for Backdoor and Data Poisoning Attacks. Avi Schwarzschild*, Micah Goldblum*, Arjun Gupta, John Dickerson, Tom Goldstein. International Conference on Machine Learning (ICML), 2021. [Published Version]
Truth or Backpropaganda? An Empirical Investigation of Deep Learning Theory. Micah Goldblum, Jonas Geiping, Avi Schwarzschild, Michael Moeller, Tom Goldstein. International Conference on Learning Representations (ICLR), 2020. [Published Version]