New strategies for OVO feature selection on multiclass problems

Abstract

Feature selection is a useful machine learning technique aimed at reducing the dimensionality of the input space, discarding useless or redundant variables, in order to increase the performance and interpretability of models. The well-known Recursive Feature Elimination (RFE) algorithm provides good performance with moderate computational efforts, in particular for wide datasets. When using Support Vector Machines (SVM) for multiclass classification problems, the most typical strategy is to apply a simple One–Vs–One (OVO) strategy to produce a multiclass classifier starting from binary ones. In this work we introduce improved methods to produce the final ranking of features on multiclass problems with OVO–SVM, based on different combinations of the set of rankings produced by the diverse binary problems. We evaluated our new strategies using wide datasets from mass–spectrometry analysis and standard datasets from the UCI repository. In particular, we compared the new methods with the traditional average strategy. Our results suggest that one of our new methods outperforms the traditional scheme in most situations.