Time Dependent Analysis of Machine Learning Algorithms

This study introduces a standardisation technique for existent performance metrics which make them time dependent and independent of the number of attributes and testing/training tuples in the dataset, thus enabling a comparison of various supervised methods across different datasets. In this study, this technique has been applied to the achieved Accuracy, F1 score, ROC and Cross Entropy. Ten distinct, supervised learning based, both balanced and unbalanced datasets have been chosen, and 10 different classification algorithms have been trained and tested on this dataset. The training/testing time, the standardised performance measures and the raw accuracy is then used to analyse each algorithm and its strength and weakness based on its accuracy v/s its train/test timing. The suitability of algorithms to real-time systems has been evaluated and optimal algorithms in different time dependent scenarios are outlined.