Feature selection with simple ANN ensembles

Abstract

Feature selection is a well-known pre-processing technique, commonly used with high-dimensional datasets. Its main goal is to discard useless or redundant variables, reducing the dimensionality of the input space, in order to increase the performance and interpretability of models. In this work we introduce the ANN-RFE, a new technique for feature selection that combines the accurate and time-e cient RFE method with the strong discrimination capabilities of ANN ensembles. In particular, we discuss two feature importance metrics that can be used with ANN-RFE: the shu ing and dE metrics. We evaluate the new method using an arti cial example and ve real-world wide datasets, including gene-expression data. Our results suggest that both metrics have equivalent capabilities for the selection of informative variables. ANNRFE seems to produce overall results that are equivalent to previous e cient methods, but can be more accurate on particular datasets.