Discriminative Feature Selection by Nonparametric Way with Cluster Validation

Feature Selection is the preprocessing process of identifying the subset of data from large dimension data. To identifying the required data, using some Feature Selection algorithms. Like Relief, Parzen-Relief algorithms, it attempts to directly maximize the classification accuracy and naturally reflects the Bayes error in the objective. Proposed algorithmic framework selects a subset of features by minimizing the Bayes error rate estimated by a nonparametric estimator. As an example, we show that the Relief algorithm greedily attempts to minimize the Bayes error estimated by the k-Nearest-Neighbor (kNN) method. In particular, by exploiting the proposed framework, we establish the Parzen-Relief (P-Relief) algorithm based on Parzen window estimator. The Relief algorithm is a popular approach for feature weight estimation. Many extensions of the Relief algorithm are developed. Because of the randomicity and the uncertainty of the instances used for calculating the feature weight vector in the Relief algorithm, the results will uctuate with the instances, which lead to poor evaluation accuracy. To solve this problem, a feature selection algorithm parzen+relief based algorithm is proposed. It takes both the mean and the variance of the discrimination among instances and weights into account as the criterion of feature weight estimation, which makes the result more stable and accurate.