Publications and Manuscripts

Preprints

CD-GraB: Coordinating Distributed Example Orders for Provably Accelerated Training
A. Feder Cooper*, Wentao Guo*, Khiem Pham*, Tiancheng Yuan, Charlie Ruan, Yucheng Lu, Christopher De Sa
[Arxiv]

Publications

(* denotes equal contribution)

CocktailSGD: Fine-tuning Foundation Models over 500Mbps Networks
Jue Wang*, Yucheng Lu*, Binhang Yuan, Beidi Chen, Percy Liang, Christopher De Sa, Christopher Re, Ce Zhang
In the Fortieth International Conference on Machine Learning (ICML) 2023.
To Appear

STEP: Learning N:M Structured Sparsity Masks from Scratch with Precondition
Yucheng Lu, Shivani Agrawal, Suvinay Subramanian, Oleg Rybakov, Christopher De Sa, Amir Yazdanbakhsh
In the Fortieth International Conference on Machine Learning (ICML) 2023.
To Appear

Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam
Yucheng Lu, Conglong Li, Minjia Zhang, Christopher De Sa, Yuxiong He
In the Eleventh International Conference on Learning Representations (ICLR) 2023.
[Arxiv][Tutorial][Code]

GraB: Finding Provably Better Data Permutations than Random Reshuffling
Yucheng Lu, Wentao Guo, Christopher De Sa
In the Thirty-sixth Conference on Neural Information Processing Systems (NeurIPS) 2022.
[Proceedings][Arxiv][Code]

A General Analysis of Example-Selection for Stochastic Gradient Descent
Yucheng Lu*, Si Yi Meng*, Christopher De Sa
In the Tenth International Conference on Learning Representations (ICLR) 2022.
[Proceedings][Code] Spotlight (5%)

Hyperparameter Optimization is Deceiving Us, and How to Stop It
A. Feder Cooper, Yucheng Lu, Jessica Zosa Forde, Christopher De Sa
In the Thirty-fifth Conference on Neural Information Processing Systems (NeurIPS) 2021.
[Proceedings][Arxiv][Code]

Variance Reduced Training with Stratified Sampling for Forecasting Models
Yucheng Lu, Youngsuk Park, Lifan Chen, Yuyang Wang, Christopher De Sa, Dean Foster
In the Thirty-eighth International Conference on Machine Learning (ICML) 2021.
[Proceedings][Arxiv][Code]

Optimal Complexity in Decentralized Training
Yucheng Lu, Christopher De Sa
In the Thirty-eighth International Conference on Machine Learning (ICML) 2021.
Longer version available in the Journal of Machine Learning Research (JMLR) .
[Proceedings][Arxiv][JMLR][Errata][Media Coverage (Chinese)] Outstanding Paper Award Honorable Mention

MixML: A Unified Analysis of Weakly Consistent Parallel Learning
Yucheng Lu, Jack Nash, Christopher De Sa
Unpublished Manuscript
[Arxiv]

Adaptive Diffusion of Sensitive Information In Online Social Networks
Xudong Wu, Luoyi Fu, Huan Long, Dali Yang, Yucheng Lu, Xinbing Wang, Guihai Chen
In IEEE Transactions on Knowledge and Data Engineering (TKDE) 2020.
[Paper]

Moniqua: Modulo Quantized Communication in Decentralized SGD
Yucheng Lu, Christopher De Sa
In the Thirty-seventh International Conference on Machine Learning (ICML) 2020.
[Proceedings][Arxiv]