Computer Science Department
Cornell University
Spring 2002

Time and Place

• Tuesday, Thursday: 11:40-12:55, Thurston 203

Personnel

General Information

Course Information (.doc) (last modified 1/22)

Academic integrity policy

Checklist (what have we covered so far):

• Introduction, What is Machine Learning ?
• Non-metric methods:
  1. Concept Learning (candidate-elimination, inductive bias)
  2. Decision trees (ID3, C4.5, pruning methods)
• Bayesian Learning:
  1. Bayesian decision theory
  2. Sequential inference
  3. ML and Bayesian parameter estimation
  4. Hypotheses evaluation using Bayes Theorem
  5. Bayes optimal classifier
  6. Gibbs algorithm
• Graphical models
  1. Bayesian belief networks
  2. The EM algorithm
  3. Hidden Markov Models - the evaluation and decoding problems
  4. Hidden Markov Models - the learning problem
• Nonparametric Techniques:
  1. Density Estimation
  2. The nearest neighbor algorithm
• Linear discriminant functions:
  1. LD functions and decision surfaces
  2. The perceptron criterion function
  3. The sum-of-squared-error criterion function
  4. Gradient descent procedures
     1. Relaxation (error-correcting) procedures
     2. Least-mean-squared (LMS) procedures (also known as minimum-squared-error MSE procedures)
     1. Stochastic (single-sample) procedures
     2. Batch procedures
• Artificial Neural Networks
  1. Feedforward operation
  2. Backpropagation algorithm
  3. Learning curves
  4. Feature mapping
  5. Improving performance (practical tips)
• Stochastic methods
  1. Genetic algorithms
  2. Genetic programming
• Unsupervised learning
  1. Mixture densities
  2. The maximum likelihood estimates
  3. The iterative EM clustering algorithm
  4. The k-means clustering algorithm
  5. Hierarchical/pairwise clustering
  6. Principal component analysis
  7. Multi-dimensional scaling
• Hypothesis evaluation
  1. Sample error vs. true error
  2. Confidence intervals
  3. Comparing hypotheses
  4. Comparing learning algorithms (for a specific target function)
  5. The minimum description length principle
• Algorithm-independent Machine Learning (general principles of ML)
  1. The no free lunch theorem
  2. Bias vs. Variance
  3. Sampling and validation techniques (jackknife, bootstraping)
  4. Bagging and Boosting

• Project ideas: Some project ideas are listed here. Check also last year's projects (.pdf, .ps). Original ideas for projects are most welcome. Graduate students are welcome to suggest a project which is related to their research topic. All projects are practical ("experimental") and involve designing and implementing a learning system.
• Project proposal: one or two paragraphs specifying the problem you are focusing on, the learning system(s) that you are going to apply, any modifications/improvements that you are considering to implement, and the means by which you are going to evaluate your learner (using a benchmark or a validation technique, etc). The goal of the proposal is to make sure that you chose a feasible project, and that you address the important issues. Project proposals are due April 6.
• Final project: Due May 14.
  Information on what should be in the final report is available here

ML Links

• The UC-Irvine ML Dataset Archive | The UC-Irvine KDD archive | more datasets
• ML Projects
  • The WEKA Machine Learning Project (code for many ML algo's, as well as some datasets)
  • Really Cool AI Demos
• Journals
  • Journal of ML Research | Data Mining and Knowledge Discovery | Journal of AI Research | IEEE Neural Networks Council (several journals are connected to this page)
• Knowledge Discovery in Databases
• Other University ML classes
  • Wisc Madison More External AI References
• Pointers to ML Courses
• Neural Network Resources
• Some ILP Stuff
• Some SVM Stuff
• Machine Learning Benchmarking
• AI Bibliography Server | Neural Networks Bibliography Server (Austrian AI Institute)
• AI Resources (Canadian NRC Server)
• Aha's ML Links
• Stuart Russell's:AI on the Web (loads of links)
• Reinforcement Learning Repository

Demos and Supplementary Material