Machine Learning Classifiers Selection in Network Intrusion Detection

Abstract

The objective of this work is to select machine learning classifiers for Network Intrusion Detection NIDS problems. The selection criterion is based upon the hyper-parameter variation, to evaluate and compare consistently the different models configuration. The models were trained and tested by crossvalidation sharing the same dataset partitions. The hyper-parameter search was performed in two ways, exhaustive and randomized upon the structure of the classifier to get feasible results. The performance result was tested for significance according to the frequentist and Bayesian significance test. The Bayesian posterior distribution was further analyzed to extract information in support of the classifiers comparison. The selection of a machine learning classifier is not trivial and it heavily depends on the dataset and the problem of interest. In this experiment seven classes of machine learning classifiers were initially analyzed, from which only three classes were selected to perform cross-validation to get the final selection, Decision Tree, Random Forest, and Multilayer Perceptron Classifiers. This article explores a systematic and rigorous approach to assess and select NIDS classifiers further than selecting the performance scores.