Flexible Discriminant Analysis

AKA: Flexible Discriminant, FDA.
Context:
- It can be produced by Flexible Discriminant Analysis System that implements a Flexible Discriminant Analysis Algorithm to solve a Flexible Discriminant Analysis Task.
Example(s):
Counter-Example(s):
- Linear Discriminant Analysis,
- Penalized Discriminant Analysis.
See: Fisher's Linear Discriminant Analysis, Support Vector Machine, Multiresponse Linear Regression, Optimal Scoring, Multigroup Classification, Nonlinear Classification.

References

(Hastie et al., 2009) ⇒ Trevor Hastie, Robert Tibshirani, and Jerome H. Friedman. (2009). “Chapter 12: Support Vector Machines and Flexible Discriminants". In: "The Elements of Statistical Learning: Data Mining, Inference, and Prediction; 2nd edition. (PDF)" Springer-Verlag. ISBN:0387848576
- QUOTE: In this chapter we describe generalizations of linear decision boundaries for classification. Optimal separating hyperplanes are introduced in Chapter 4 for the case when two classes are linearly separable. Here we cover extensions to the nonseparable case, where the classes overlap. These techniques are then generalized to what is known as the support vector machine, which produces nonlinear boundaries by constructing a linear boundary in a large, transformed version of the feature space. The second set of methods generalize Fisher’s linear discriminant analysis (LDA). The generalizations include flexible discriminant analysis which facilitates construction of nonlinear boundaries in a manner very similar to the support vector machines, penalized discriminant analysis for problems such as signal and image classification where the large number of features are highly correlated, and mixture discriminant analysis for irregularly shaped classes (...)

(Hastie et al., 1994) ⇒ Trevor Hastie, Robert Tibshirani, and Andreas Buja. (1994). “Flexible Discriminant Analysis by Optimal Scoring."In: Journal of the American Statistical Association, 89(428).
- ABSTRACT: Fisher's linear discriminant analysis is a valuable tool for multigroup classification. With a large number of predictors, one can find a reduced number of discriminant coordinate functions that are "optimal" for separating the groups. With two such functions, one can produce a classification map that partitions the reduced space into regions that are identified with group membership, and the decision boundaries are linear. This article is about richer nonlinear classification schemes. Linear discriminant analysis is equivalent to multiresponse linear regression using optimal scorings to represent the groups. In this paper, we obtain nonparametric versions of discriminant analysis by replacing linear regression by any nonparametric regression method. In this way, any multiresponse regression technique (such as MARS or neural networks) can be postprocessed to improve its classification performance.