Cornell University

Spring 2001

**Announcements**

- Tuesday, Thursday: 11:15-12:05, Thurston 205
- Project due: Monday, May 11, 2001
- Final Exam: Thursday, May 17, 2001 9:00-11:30

Office Hours |
Office |
||

Instructor | Golan Yona golan@cs.cornell.edu | Tuesday 12:10-12:40 pm Thursday 12:10-12:40 pm Friday 13:30-14:00 pm | Upson 5156 |

Teaching Assistants | Kevin O'Neill oneill@cs.cornell.edu | Monday 14:00-15:00 pm Wednesday 14:00-15:00 pm | Upson 5132 |

Ashish Motivala anm24@cornell.edu |
Wednesday 10:15-11:15 am | Upson 328D |

Course Information (.ps,
.pdf)* (last modified 1/24)*

Checklist (what have we covered so far):

- Introduction
- Concept learning
- Non-metric methods:
- Decision trees
- Grammatical methods - skipped (requires knowledge in formal languages)
- Strings based methods and Rule-based methods - skipped (priority)

- Bayesian Learning:
- Bayesian decision theory
- Sequential inference added
- ML and Bayesian parameter estimation
- Sufficient statistics skipped (priority)
- Hypotheses evaluation using Bayes Theorem
- Bayes optimal classifier
- Gibbs algorithm
- Bayesian belief networks
- The EM algorithm
- Hidden Markov Models

- Nonparametric Techniques:
- Density Estimation
- Parzen Window
- The nearest neighbor algorithm

- Linear discriminant functions:
- LD functions and decision surfaces
- The perceptron function
- Relaxation and MSE procedures
- Support vector machines skipped (we are running out of time..)

- Artificial Neural Networks
- Feedforward operation
- Backpropagation algorithm
- Feature mapping
- Improving performance

- Stochastic methods
- Genetic algorithms
- Genetic programming

- Hypothesis evaluation
- Sample error vs. true error
- Confidence intervals
- Comparing hypotheses
- Comparing learning algorithms
- K-fold cross validation
- The no free lunch theorem
- The minimum description length principle

- Unsupervised learning
- Mixture densities
- The maximum likelihood estimates
- The iterative EM clustering algorithm
- The k-means clustering algorithm
- Hierarchical clustering
- Principal component analysis
- Multi-dimensional scaling
- Kohonen self-organizing maps

The last two appear in lecture notes but were not discussed in class

**Project ideas:**Some projects ideas are listed here. But you are encouraged to suggest other projects. Graduate students who would like to do a project which is related to their research are also encouraged to do so. All projects are practical ("experimental") and involves designing and implementing a learning system. Those who are interested in doing a theoretical project, please contact me. There are many interesting papers that you can based your project on.**Project proposal:**this is not mandatory, but recommended if you already know what you want to work on. The last day to submit proposals is**April 12**. The idea behind the proposal is just to make sure that you chose a feasible project, and that you address the important issues. Project proposals that are well planned and thought over will get extra credit up to 5%. But you don't get bad points for proposals that do not meet these criteria.

If you are interested in projects that are based on material that will be learned in class after the proposal submission deadline, you can still meet with me after the deadline to discuss the outline of the project.

In any case, even if you don't submit a proposal, you must register a project (one of the projects on the list, or your own suggested project). The last day to register a project is**April 12**, by email to me.**Project proposal format:**should be one page at the most. Specify the problem you are focusing on, the learning system(s) that you are going to apply, any modifications/improvements that you are considering to implement, and the means by which you are going to evaluate your learner (using a benchmark or a validation technique, etc)**Final report.**Due**May 14**.

Information on what should be in the final report will be posted here soon. The credit for the project is 40% of the final grade. Exceptional projects will get extra credit.

- The UC-Irvine ML Dataset Archive | The UC-Irvine KDD archive | more datasets
- ML Projects
- The WEKA Machine Learning Project (code for many ML algo's, as well as some datasets)
- Really Cool AI Demos

- Journals
- Machine Learning (the
*on-line*page) | Journal of ML Research | Data Mining and Knowledge Discovery | Journal of AI Research | AI Magazine | IEEE Neural Networks Council (several journals are connected to this page)

- Machine Learning (the
- Knowledge Discovery in Databases
- Other University ML classes
- Wisc Madison More External AI References

- Pointers to ML Courses
- Neural Network Resources
- Some ILP Stuff
- Some SVM Stuff
- Machine Learning Benchmarking
- International Society for Adaptive Behavior
- AI Bibliography Server | Neural Networks Bibliography Server (Austrian AI Institute)
- AI Resources (Canadian NRC Server)
- Aha's ML Links
- Stuart Russell's:AI on the Web (loads of links)
- Reinforcement Learning Repository