CS 4780/5780 - Fall 2012
Cornell University
Department of Computer Science

• Tuesday, 1:25pm - 2:40pm in Olin 255
• Thursday, 1:25pm - 2:40pm in Olin 255

First Prelim Exam: October 16
Second Prelim Exam: November 20
Project Due Date: December 13

• [Piazza Forum]  We greatly prefer that you use this Piazza forum for questions and discussions.  The forum is monitored all the TA's and the prof -- you will get the best response time. And all the TA's will know the question you asked and the answers you receive.
• [CMS] Assignments, the prereq exam, and all grades are posted on CMS.
• [Videonote] Recordings of last year's lectures with annotations. Lectures this year will be somewhat different, but this can still be a useful reference.

This course will introduce the fundamental set of techniques and algorithms that constitute machine learning as of today, ranging from classification methods like decision trees and support vector machines, over structured models like hidden Markov models, to clustering and matrix factorization methods for recommendation. The course will not only discuss individual algorithms and methods, but also tie principles and approaches together from a theoretical perspective. In particular, the course will cover the following topics: 

• Concept Learning : Hypothesis space, version space
• Instance-based Learning : K-Nearest Neighbors, collaborative filtering
• Decision Trees : TDIDT, attribute selection, pruning and overfitting
• ML Experimentation : Hypothesis tests, resampling estimates
• Linear Rules : Perceptron, duality, mistake bound
• Support Vector Machines : Optimal hyperplane, kernels, stability
• Generative Models : Naïve Bayes, linear discriminant analysis
• Hidden Markov Models: probabilistic model, estimation, Viterbi
• Structured Output Prediction : predicting sequences, rankings, etc.
• Learning Theory : PAC learning, mistake bounds, VC dimension
• Clustering : HAC, k-means, mixture of Gaussians
• Recommendation Systems: Similarity based methods, matrix factorization

• Tom Mitchell, "Machine Learning", McGraw Hill, 1997.
• CS4780 course packet available at the Cornell Bookstore.

An additional textbook that can serve as a brief secondary reference on many topics in this class is

Ethem Alpaydin, "Introduction to Machine Learning", MIT Press, 2004.

The reading in the course packet are taken from the following books. In addition, these are some books for further reading beyond the scope of the course:

• Cristianini, Shawe-Taylor, "Introduction to Support Vector Machines", Cambridge University Press, 2000. (online via Cornell Library) Schoelkopf, Smola, "Learning with Kernels", MIT Press, 2001. (online)
• Bishop, "Pattern Recognition and Machine Learning", Springer, 2006.
• Devroye, Gyoerfi, Lugosi, "A Probabilistic Theory of Pattern Recognition", Springer, 1997.
• Duda, Hart, Stork, "Pattern Classification", Wiley, 2000.
• Hastie, Tibshirani, Friedman, "The Elements of Statistical Learning", Springer, 2001.
• Joachims, "Learning to Classify Text using Support Vector Machines", Kluwer, 2002.
• Leeds Tutorial on HMMs (online)
• Manning, Schuetze, "Foundations of Statistical Natural Language Processing", MIT Press, 1999. (online via Cornell Library)
• Manning, Raghavan, Schuetze, "Introduction to Information Retrieval", Cambridge, 2008. (online)
• Vapnik, "Statistical Learning Theory", Wiley, 1998.

• 40%: 2 Prelim Exams
• 15%: Final Project
• 35%: Homework (~5 assignments)
• 5%: Quizzes (in class)
• 2%: Prereq Exam
• 3%: Class Participation

To eliminate outlier grades for homeworks and quizzes, the lowest grade is replaced by the second lowest grade when grades are cumulated at the end of the semester.

All assignments are due at the beginning of class on the due date. Assignments turned in late will be charged a 1 percentage point reduction of the cumulated final homework grade for each period of 24 hours for which the assignment is late. However, every student has a budget of 5 late days (i.e. 24 hour periods after the time the assignment was due) throughout the semester for which there is no late penalty. So, if you have perfect scores of 100 on all 5 homeworks and a total of 7 late days, you final homework score will be 98 (which then accounts for 35% of your course grade). No assignment will be accepted after the solution was made public, which is typically 4 days after the time it was due. You can submit late assignments in class, in office hours, or to the office of a TA.

Graded homework assignments and prelims can be picked up in Upson 305 (note the change of room, opening hours Monday - Thursday 12:00pm - 4:00pm, Friday: 12:30pm - 4:00pm). Regrade requests can be submitted within 7 days after the grades have been made available on CMS. Regrade requests have to be submitted in writing and in hardcopy using this form (or similar). They can be submitted in class, in office hours, or to the office of a TA.

We always appreciate interesting homework solutions that go beyond the minimum. To reward homework solutions that are particularly nice, we will give you "Bonus Points". Bonus points are collected in a special category on CMS. Bonus points are not real points and are not summed up for the final grade, but they can nudge somebody to a higher grade who is right on the boundary.

All assignment, exam, and final grades are roughly on the following scale: A=92-100; B=82-88; C=72-78; D=60-68; F= below 60

We run automatic cheating detection to detect violations of the collaboration rules.