1999 ExploitingGenerativeModelsinDis

Jump to: navigation, search

Subject Headings: Kernel Regression.


Cited By



Generative probability models such as hidden Markov models provide a principled way of treating missing information and dealing with variable length sequences. On the other hand, discriminative methods such as support vector machines enable us to construct flexible decision boundaries and often result in classification performance superior to that of the model based approaches. An ideal classifier should combine these two complementary approaches. In this paper, we develop a natural xvay of achieving this combination by deriving kernel functions for use in discriminative methods such as support vector machines from generative probability models. We provide a theoretical justification for this combination as well as demonstrate a substantial improvement in the classification performance in the context of DNA and protein sequence analysis.

1 Introduction

Speech, vision, text and biosequence data can be difficult to deal with in the context of simple statistical classification problems. Because the examples to be classified are often sequences or arrays of variable size that may have been distorted in particular ways, it is common to estimate a generative model for such data, and then use Bayes rule to obtain a classifier from this model. However. many discriminative methods, which directly estimate a posterior probability for a class label (as in Gaussian process classifiers [5]) or a discriminant function for the class label (as in support vector machines [6]) have in other areas proven to be superior to generative models for classification problems. The problem is that there has been no systematic way to extract features or metric relations between examples for use with discriminative methods in the context of difficult data types such as those listed above. Here we propose a general method for extracting these discriminatory features using a generative model. V?"ile the features xve propose are generally applicable, they are most naturally suited to kernel methods.




 AuthorvolumeDate ValuetitletypejournaltitleUrldoinoteyear
1999 ExploitingGenerativeModelsinDisTommi S. Jaakkola
David Haussler
Exploiting Generative Models in Discriminative Classifiers1999
AuthorTommi S. Jaakkola + and David Haussler +
titleExploiting Generative Models in Discriminative Classifiers +
year1999 +