Happy to get the research intern opportunity in Cornell CS from Prof. Kilian Q. Weinberger, to work on defenses against adversarial examples and simplifying GCN for NLP tasks. I also work closely with Prof. Chris De Sa on developing numerically robust and accurate models for hyperbolic embeddings of graphs.
It's my great honor to major in Mathematics and Applied Mathematics (ZhiYuan honours programme) at Shanghai Jiao Tong University, where I am so lucky to work with Prof. John E. Hopcroft and Huan Long, we analyzed both theoretically and experimentally of the intrinsic dimension of the manifolds embedded in neural networks.
Hyperbolic Space is particularly interesting and promising in machine learning due to its non-Euclidean property. For example, volume of a ball in the hyperbolic space increases exponentially w.r.t. the radius, as opposed to Euclidean case where the volume increases only polynomially. However, a major numerical issue called the NaN problem greatly affect its power and practical use. We proposed tiling-based models to solve the NaN problem both theoretically and empirically, currently we are working on more general case and designing robust hyperbolic networks for different tasks.
Despite of the great success of Machine learning, there are also some concerns calling for attention, namely, the security and privacy concern. On the one hand, Machine Learning models are vulnerable to imperceptible adversarial perturbations, which alter the model's decision entirely, it's necessary and worthwhile to design robust and secure models for various applications. On the other hand, Machine Learning models also suffer from information leakage, attacks such as membership inference and model inversion are able to infer information of the dataset. Hence, it's important to measure the information leakage and design privacy-preserving models and algorithms. However, both aspects may degrade the model's performace. What's more, it's particularly interesting to ask whether there is a tradeoff between robustness and privacy, we are currently looking at these tradeoffs in detail.
Federated learning is proposed for collaborative Machine Learning without centralized training data. Users will be able to collaboratively learn a shared model while keeping all the data on device. Latest FL approaches use differential privacy or robust aggregation to ensure privacy and integrity of the federated model, however, we show that these approaches will destroy the accuracy of the federated model for many participants. Thus, we propose local adaptation of federated models, our evaluatation of different techniques demonstrate that all participants benefit from local adaptation.
Tao Yu, Chris De Sa. "Numerically Accurate Hyperbolic Embeddings Using Tiling-Based Models", Spotlight, [Compression Code, Learning Code, Slides, Poster]. In 33rd Conference on Neural Information Processing Systems (NeurIPS 2019).
Tao Yu*, Shengyuan Hu*, Chuan Guo, Wei-Lun Chao, Kilian Q. Weinberger. "A New Defense Against Adversarial Images: Turning a Weakness into a Strength", [Code, Poster]. In 33rd Conference on Neural Information Processing Systems (NeurIPS 2019)
Felix Wu*, Tianyi Zhang*, Amauri Holanda de Souza Jr.*, Christopher Fifty, Tao Yu, Kilian Q. Weinberger. "Simplifying Graph Convolutional Networks", [Code]. In 36th International Conference on Machine Learning (ICML 2019).
Tao Yu, Huan long, John Hopcroft. "Curvature-based Comparison of Two Neural Networks". In 24th International Conference on Pattern Recognition (ICPR 2018).
Tao Yu, Yu Qiao, Huan Long. "Knowledge-based Fully Convolutional Network and Its Application in Segmentation of Lung CT Images". (Technical Report)
TalksNeurIPS 2019, "Numerically Accurate Hyperbolic Embeddings Using Tiling-Based Models".