A pairwise subspace projection method for multi-class linear dimension reduction

Abstract

Linear feature extraction is commonly applied in an all-at-once way, meaning that a single trasformation is used for all the data regardless of the classes. Very good results can be achieved with this approach when the classification problem involves just a few classes. Nevertheless, when the number of classes grows is often difficult to find a low dimensional subspace while preserving the error rates, due to overlapping between the different populations. In this paper we propose an alternative method based on a collection of transformations, each involving two of the classes in the problem. Each transformation in the collection is estimated using an approximation to the information discriminant analysis, which is found to be equivalent to sufficient dimension reduction for heteroscedastic Gaussian data. A regularized version of the objective function is also introduced, allowing for simultaneous variable selection. In this way, each reduction implies only a subset of the original variables. A probabilistic model is build by means of a simple latent variable, so that classification is carried out using standard Bayes decision rule. Several real data sets are used to compare the performance of the proposed method against similar approaches based on ensembles of binary classifiers.