CMIFS (Conditional Mutual Information-based Feature Selection)

Information

• CMIFS (Conditional Mutual Information-based Feature Selection) is a type of feature selection method that aims to identify the most informative subset of features that are relevant to a target variable.
• CMIFS ranks the features based on their conditional dependence with the target variable, given the other features, and selects the top-ranked features that have the highest conditional mutual information (CMI) with the target variable.
• CMI measures the amount of information that a feature contains about the target variable, given the other features, and reflects the degree of dependence between the feature and the target variable, while controlling for the influence of the other features.
• CMIFS uses the Kraskov-Stögbauer-Grassberger (KSG) estimator to estimate the CMI between each feature and the target variable, given the other features. The KSG estimator is a non-parametric method that is based on the k-nearest neighbor algorithm, and has a low bias and a fast convergence rate, but may have a high variance and a large memory requirement.
• CMIFS can capture both linear and nonlinear dependencies between the features and the target variable, and can identify the interactions or synergies among the features that are relevant to the target variable.
• CMIFS can be used for various machine learning tasks, such as classification, regression, clustering, or dimensionality reduction, and can be combined with other feature selection or dimensionality reduction methods, such as principal component analysis (PCA), linear discriminant analysis (LDA), or non-negative matrix factorization (NMF).
• CMIFS has been applied to various domains, such as bioinformatics, image processing, speech recognition, and financial prediction, and has been shown to achieve competitive or even better performance than other feature selection methods, such as correlation-based feature selection (CFS), minimum redundancy maximum relevance (mRMR), and recursive feature elimination (RFE).