Research Intern June 2020 - Aug. 2020
It's my fortune to intern at Apple Turi AI/ML Privacy team with Ulfar Erlingsson, to look into and evaluate privacy guarantess in federated learning practically.
I am a PhD student at Cornell University in the Department of Computer Science, where I am co-advised by Prof. Vitaly Shmatikov and Chris De Sa, previously I received my bachelor degree in Mathematics from Shanghai Jiao Tong University in July 2019, where I am fourtunated to work with Prof. John E. Hopcroft and Huan Long.
My primary research interest lies in private and secure machine learning, such as defending against adversarial examples, developing privacy-preserving algorithms and mitigating the tradeoff between accurcy and robustness & privacy. Beyond this scope, I am also interested in adopting hyperbolic geometry in ML for stronger and accurate models, recently I am also pretty interested in federated learning.
We host the Machine Learning Security, Privacy & Fairness group weekly in Ithaca campus [Past Schedule].
[Curriculum Vitae] [Google Scholar] [Github] [LinkedIn]
It's my fortune to intern at Apple Turi AI/ML Privacy team with Ulfar Erlingsson, to look into and evaluate privacy guarantess in federated learning practically.
I am glad to join Computer Science Department at Cornell University in fall 2019 as a Ph.D. student, it's my great fortune to work closely with Prof. Chris De Sa and Vitaly Shmatikov on some cool stuff.
Happy to get the research intern opportunity in Cornell CS from Prof. Kilian Q. Weinberger, to work on defenses against adversarial examples and simplifying GCN for NLP tasks. I also work closely with Prof. Chris De Sa on developing numerically robust and accurate models for hyperbolic embeddings of graphs.
It's my great honor to major in Mathematics and Applied Mathematics (ZhiYuan honours programme) at Shanghai Jiao Tong University, where I am so lucky to work with Prof. John E. Hopcroft and Huan Long, we analyzed both theoretically and experimentally of the intrinsic dimension of the manifolds embedded in neural networks.
Hyperbolic Space is particularly interesting and promising in machine learning due to its non-Euclidean property. For example, volume of a ball in the hyperbolic space increases exponentially w.r.t. the radius, as opposed to Euclidean case where the volume increases only polynomially. However, a major numerical issue called the NaN problem greatly affect its power and practical use. We proposed tiling-based models to solve the NaN problem both theoretically and empirically, currently we are working on more general case and designing robust hyperbolic networks for different tasks.
Despite of the great success of Machine learning, there are also some concerns calling for attention, namely, the security and privacy concern. On the one hand, Machine Learning models are vulnerable to imperceptible adversarial perturbations, which alter the model's decision entirely, it's necessary and worthwhile to design robust and secure models for various applications. On the other hand, Machine Learning models also suffer from information leakage, attacks such as membership inference and model inversion are able to infer information of the dataset. Hence, it's important to measure the information leakage and design privacy-preserving models and algorithms. However, both aspects may degrade the model's performace. What's more, it's particularly interesting to ask whether there is a tradeoff between robustness and privacy, we are currently looking at these tradeoffs in detail.
Federated learning is proposed for collaborative Machine Learning without centralized training data. Users will be able to collaboratively learn a shared model while keeping all the data on device. Latest FL approaches use differential privacy or robust aggregation to ensure privacy and integrity of the federated model, however, we show that these approaches will destroy the accuracy of the federated model for many participants. Thus, we propose local adaptation of federated models, our evaluatation of different techniques demonstrate that all participants benefit from local adaptation.
Tao Yu, Eugene Bagdasaryan, Vitaly Shmatikov. "Salvaging Federated Learning by Local Adaptation", [Code].
Tao Yu, Chris De Sa. "Numerically Accurate Hyperbolic Embeddings Using Tiling-Based Models", Spotlight, [Compression Code, Learning Code, Slides, Poster]. In 33rd Conference on Neural Information Processing Systems (NeurIPS 2019).
Tao Yu*, Shengyuan Hu*, Chuan Guo, Wei-Lun Chao, Kilian Q. Weinberger. "A New Defense Against Adversarial Images: Turning a Weakness into a Strength", [Code, Poster]. In 33rd Conference on Neural Information Processing Systems (NeurIPS 2019)
Felix Wu*, Tianyi Zhang*, Amauri Holanda de Souza Jr.*, Christopher Fifty, Tao Yu, Kilian Q. Weinberger. "Simplifying Graph Convolutional Networks", [Code]. In 36th International Conference on Machine Learning (ICML 2019).
Tao Yu, Huan long, John Hopcroft. "Curvature-based Comparison of Two Neural Networks". In 24th International Conference on Pattern Recognition (ICPR 2018).
Tao Yu, Yu Qiao, Huan Long. "Knowledge-based Fully Convolutional Network and Its Application in Segmentation of Lung CT Images". (Technical Report)
Talks
NeurIPS 2019, "Numerically Accurate Hyperbolic Embeddings Using Tiling-Based Models".