Discriminative Clustering based Feature Selection and Nonparametric Bayes Error Minimization and Support Vector Machines (SVMs)

In recent years feature selection is an eminent task in knowledge discovery database (KDD) that selects appropriate features from massive amount of high-dimensional data. In an attempt to establish theoretical justification for feature selection algorithms, this work presents a theoretical optimal criterion, specifically, the discriminative optimal criterion (DoC) for feature selection. Computationally DoC is tractable for practical tasks that propose an algorithmic outline, which selects a subset of features by minimizing the Bayes error rate approximate by a non-parametric estimator. A set of existing algorithms as well as new ones can be derived naturally from this framework. In the proposed Discriminative Clustering based feature Selection algorithm (DCBFS) minimum spanning tree is constructed to group the similar feature from the dataset. Also, efficient algorithms for multiple kernel learning and best feature selection algorithm are introduced. Kernel function called Gaussian Radial basis Polynomial Function (GRPF) is introduced in order to improve the classification accuracy of Support Vector Machines (SVMs) for both linear and non-linear data sets. The aim is Support Vector Machines (SVMs) with different kernels compared with back-propagation learning algorithm in classification task. Finally the proposed algorithm is improved in terms of accuracy and time compared to the existing algorithm.