My home page
My group
Professional activities

Daphne Koller Publications

Restricted Bayes optimal classifiers (2000)

by S. Tong and D. Koller

Abstract: We introduce the notion of restricted Bayes optimal classifiers. These classifiers attempt to combine the flexibility of the generative approach to classification with the high accuracy associated with discriminative learning. They first create a model of the joint distribution over class labels and features. Instead of choosing the decision boundary induced directly from the model, they restrict the allowable types of decision boundaries and learn the one that minimizes the probability of misclassification relative to the estimated joint distribution. In this paper, we investigate two particular instantiations of this approach. The first uses a non-parametric density estimator - Parzen Windows with Gaussian kernels - and hyperplane decision boundaries. We show that the resulting classifier is asymptotically equivalent to a maximal margin hyperplane classifier, a highly successful discriminative classifier. We therefore provide an alternative justification for maximal margin hyperplane classifiers. The second instantiation uses a mixture of Gaussians as the estimated density; in experiments on real-world data, we show that this approach allows data with missing values to be handled in a principled manner, leading to improved performance over regular discriminative approaches.

Download Information

S. Tong and D. Koller (2000). "Restricted Bayes optimal classifiers." Proceedings of the 17th National Conference on Artificial Intelligence (AAAI) (pp. 658-664). pdf ps.gz full.pdf

Bibtex citation

  title = {Restricted {Bayes} optimal classifiers},
  author = {S. Tong and D. Koller},
  booktitle = {Proceedings of the 17th National Conference on Artificial Intelligence (AAAI)}, 
  month = {August},
  year = 2000, 
  address = {Austin, Texas},
  pages = {658--664},

full list
Click to go to robotics Click to go to theory Click to go to CS Stanford Click to go to Stanford's Webpage
home | biography | research | papers | my group
courses | professional activities | FAQ | personal