An Information-Theoretic Approach for Setting the Optimal Number of Decision Trees in Random Forests

Data Classification is a process within the Data Mining and Machine Learning field which aims at annotating all instances of a dataset by so-called class labels. This involves in creating a model from a training set of data instances which are already labeled, possibly being this model also used to define the class of data instances which are not classified already. A successful way of performing the classification process is provided by the algorithm Random Forests (RF), which is itself a type of Ensemble-based Classifier. An ensemble-based classifier increases the accuracy of the class label assigned to a data instance by using a set of classifiers that are modeled on different, but possibly overlapping, instance sets, and then combining the so-obtained intermediate classification results. To this end, RF particularly makes use of a number of decision trees to classify an instance, then taking the majority of votes from these trees as the final classifier. The latter one is a critical task of algorithm RF, which heavily impacts on the accuracy of the final classifier. In this paper, we propose a variation of algorithm RF, namely adjusting one of the two parameters that RF takes, the number of decision trees, dependant on a meaningful relation between the dataset predictive power rating and the number of trees itself, with the goal of improving accuracy and performance of the algorithm. This is finally demonstrated by our comprehensive experimental evaluation on several clean datasets.