NIPS 2002 Workshop:

Beyond Classification and Regression: Learning Rankings, Preferences, Equality Predicates, and Other Structures

Organizers: Rich Caruana & Thorsten Joachims


Not all supervised learning problems fit the standard classification and regression function-learning model. Some problems require that we predict things other than values or classes. For example, sometimes the magnitude of the values predicted for cases are not important, but the ordering/ranking induced by these values is important. Sometimes we don't know a priori what classes exist, but we need to learn which items belong to the same class. Sometimes it is important to retrieve the top 10 objects, and no one cares what ordering is predicted for the remaining 100,000 objects. Sometimes the quality of learning will be judged by measures such as Precision and Recall or ROC that are not well optimized by standard value prediction models.

Mismatch from the value prediction learning model can arise not only with the predictions, but also can arise in the form of the training examples. For example, when a user indicates that one document is more relevant than another, or that two documents should be in the same class, but does not assign values to the documents themselves, the notion of what constitutes a training example is turned on it's head. In these situations a training example is a relation on pairs of what traditionally would have been considered independent training cases. This change has deep ramifications. 

This workshop aims to explore supervised learning problems that go beyond the usual value prediction model. In particular, it addresses problems where either (a) the goal of learning or (b) the input to the learner, are more complex structures than in classification and regression. Examples of such problems are:

The goal of the workshop is twofold. First, to create a forum for discussing recent methods and results for these problems. Second, to inspire research on new learning algorithms and problems.


This will be a one-day workshop. To prevent the workshop from degrading into a mini-conference, we will allot at least 50% of the time to discussion. The workshop will consist of 2-4 short invited talks and a number of additional presentations and discussion sessions. There will be short poster sessions to force people out of their chairs and get them talking to each other.  We probably won't do a panel discussion because it is not clear that anyone has enough expertise yet.


Submissions are invited.  They may be extended abstracts or full papers.  Submissions should be postscript, PDF, or plain text.  Please send submissions via email to, before Nov 10, 2002.   We'll put together a tentative program by Nov 15.

If there is enough interest, we may consider organizing a special journal issue on learning rankings.  We'll discuss this further at the workshop.

Participants and Presentations


This is a one-day workshop on Saturday.


Rich Caruana
4157 Upson Hall
Computer Science
Cornell University
Ithaca, NY 14853
607-255-4428 (fax)
Thorsten Joachims
4153 Upson Hall
Computer Science
Cornell University
Ithaca, NY 14853
607-255-4428 (fax)