
Course Description
Machine learning is concerned with the question of how to make computers learn from experience. The ability to learn is not only central to most aspects of intelligent behavior, but machine learning techniques have become key components of many software systems. For examples, machine learning techniques are used to build search engines, to recommend movies, to understand natural language and images, and to build autonomous robots.
This course will introduce the fundamental set of techniques and algorithms that constitute machine learning as of today. The course will not only discuss individual algorithms and methods, but also tie principles and approaches together from a theoretical perspective. In particular, the course will cover the following topics:
 Supervised Batch Learning: model, decision theoretic foundation, model selection, model assessment, empirical risk minimization
 Instancebased Learning: KNearest Neighbors, collaborative filtering
 Decision Trees: TDIDT, attribute selection, pruning and overfitting, boosting, bagging
 Linear Rules: Perceptron, logistic regression, linear regression, duality
 Support Vector Machines: Optimal hyperplane, margin, kernels, stability
 Deep Learning: multilayer perceptrons, deep networks, stochastic gradient, transformers, large language models
 Generative Models: generative vs. discriminative, naive Bayes, linear discriminant analysis
 Structured Output Prediction: predicting sequences, hidden markov model, rankings
 Statistical Learning Theory: generalization error bounds, VC dimension
 Unsupervised Learning: kmeans clustering, hierarchical agglomerative clustering, principal component analysis
The prerequisites for the class are: Probability theory (e.g., BTRY 3080, ECON 3130, MATH 4710, ENGRD 2700, CS 2800), linear algebra (e.g., MATH 2210, MATH 2940, MATH 2310), singlevariable calculus (e.g. MATH 1910, MATH 1110), and programming proficiency (e.g., CS 2110).
Forbidden overlaps: ECE 3200 (previously ECE 4200), ORIE 3741 (previously ORIE 4741), STSCI 3740 (previously STSCI 4740).


Reference Material
The main textbook for the class is:
 Shai ShalevShwartz, Shai BenDavid, "Understanding Machine Learning  From Theory to Algorithms", Cambridge University Press, 2014. (online)
For additional reading, here is a list of other sources:
 Faisal, Ong, Deisenroth, "Mathematics for Machine Learning", Cambridge University Press, 2020. (online)
 Tom Mitchell, "Machine Learning", McGraw Hill, 1997.
 Kevin Murphy, "Machine Learning  a Probabilistic Perspective", MIT Press, 2012. (online via Cornell Library)
 Cristianini, ShaweTaylor, "Introduction to Support Vector Machines", Cambridge University Press, 2000. (online via Cornell Library)
 Schoelkopf, Smola, "Learning with Kernels", MIT Press, 2001. (online)
 Bishop, "Pattern Recognition and Machine Learning", Springer, 2006.
 Ethem Alpaydin, "Introduction to Machine Learning", MIT Press, 2004.
 Duda, Hart, Stork, "Pattern Classification", Wiley, 2000.
 Hastie, Tibshirani, Friedman, "The Elements of Statistical Learning", Springer, 2001.
 Imbens, Rubin, Causal Inference for Statistical Social Science, Cambridge, 2015. (online via Cornell Library)
 Leeds Tutorial on HMMs (online)
 Manning, Schuetze, "Foundations of Statistical Natural Language Processing", MIT Press, 1999. (online via Cornell Library)
 Manning, Raghavan, Schuetze, "Introduction to Information Retrieval", Cambridge, 2008. (online)
 Vapnik, "Statistical Learning Theory", Wiley, 1998.
