Robust Unsupervised Feature Selection with Double Fuzzy Learning

GAO Yun-long; SHI Shu-guang; ZHAO Zhi-xiang; CAO Chao; PAN Jin-yan

doi:10.12263/DZXB.20240682

您当前的位置：

首页 >

文章列表页 >

Robust Unsupervised Feature Selection with Double Fuzzy Learning

PAPERS | 更新时间：2025-12-08

- Robust Unsupervised Feature Selection with Double Fuzzy Learning
  增强出版
- ACTA ELECTRONICA SINICA Vol. 53, Issue 2, Pages: 604-622(2025)
- 作者机构：
  
  1.厦门大学萨本栋微米纳米科学技术研究院，福建厦门 361102
  2.厦门大学健康医疗大数据国家研究院，福建厦门 361102
  3.自然资源部第三海洋研究所，福建厦门 361005
  4.集美大学海洋信息工程学院，福建厦门 361021
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China(42076058);Special Foundation of Fujian Province to Promote High-quality Development of Marine and Fishery Industries(FJHYF-ZH-2023-05);The Natural Science Foundation of Fujian Province of China(2020J01713;2022J01061);The Guangdong Basic and Applied Basic Research Foundation(2024A1515011682)
- DOI：10.12263/DZXB.20240682
  CLC： TP391;
- Received：22 July 2024，
  
  Revised：2024-10-12，
  
  Published：25 February 2025
- 稿件说明：
移动端阅览
高云龙, 史曙光, 赵志翔, 等. 基于双模糊学习的鲁棒无监督特征选择算法[J]. 电子学报, 2025, 53(02): 604-622.

GAO Yun-long, SHI Shu-guang, ZHAO Zhi-xiang, et al. Robust Unsupervised Feature Selection with Double Fuzzy Learning[J]. Acta Electronica Sinica, 2025, 53(02): 604-622.
高云龙, 史曙光, 赵志翔, 等. 基于双模糊学习的鲁棒无监督特征选择算法[J]. 电子学报, 2025, 53(02): 604-622. DOI：10.12263/DZXB.20240682

GAO Yun-long, SHI Shu-guang, ZHAO Zhi-xiang, et al. Robust Unsupervised Feature Selection with Double Fuzzy Learning[J]. Acta Electronica Sinica, 2025, 53(02): 604-622. DOI：10.12263/DZXB.20240682

摘要

由于维数灾难问题的存在，如何在高维数据中摒弃冗余特征、保留关键信息成为一个关键问题.无监督特征选择无需任何先验类别信息就能进行维数约减的特点引起更多研究者的关注.尽管如此，现有的无监督特征选择算法仍然存在两个问题需要解决：模糊性是数据的普遍特性，但现有大多数基于正则化回归的无监督特征选择算法忽略了数据中的模糊性，导致特征子集不理想；多数算法无法有效区分正常样本和噪声样本且易受噪声的影响.针对这些问题，本文提出了基于双模糊学习的鲁棒无监督特征选择（Robust unsupervised Feature Selection with Double Fuzzy，DFRFS）算法.DFRFS算法在基于正则化回归的无监督特征选择中引入模糊隶属度，允许数据在多个簇之间共享，能够更好地反映数据的复杂结构和不确定性.此外，DFRFS算法通过鲁棒权重学习框架为样本赋予不同的权重，从而在降低噪声样本影响的同时，保留可靠样本的作用.对比相关算法，在合成数据集与真实数据集上的实验结果验证了DFRFS算法的有效性.

Abstract

Due to the curse of dimensionality

effectively discarding redundant features while retaining critical information in high-dimensional data has become a key issue. Unsupervised feature selection

which performs dimensionality reduction without any prior class information

has attracted increasing attention. However

two common issues are ignored by existing unsupervised feature selection methods: Fuzziness is a common characteristic of data

but most existing unsupervised feature selection methods based on regularized regression ignore this aspect

resulting in suboptimal feature subsets; Most methods fail to effectively distinguish between normal and noisy samples and are susceptible to the noise. To tackle the mentioned issues

robust unsupervised feature selection with double fuzzy (DFRFS) learning is proposed. Specifically

DFRFS learning introduces fuzzy membership into unsupervised feature selection based on regularized regression

allowing data to be shared among multiple clusters

thereby better reflecting the complex structure and uncertainty of the data. Additionally

DFRFS learning assigns different weights to samples through the robust weight learning framework

thus suppressing the impact of noise while retaining the effect of normal samples. Experiments on toy and real-world datasets have demonstrated the effectiveness of the proposed method DFRFS learning.

关键词

Keywords

references

SASSI HIDRI M , ALI ZOGHLAMI M , BEN AYED R . Speeding up the large-scale consensus fuzzy clustering for handling Big Data [J ] . Fuzzy Sets and Systems , 2018 , 348 : 50 - 74 .

刘艺 , 曹建军 , 刁兴春 , 等 . 特征选择稳定性研究综述 [J ] . 软件学报 , 2018 , 29 ( 9 ): 2559 - 2579 .

LIU Y , CAO J J , DIAO X C , et al . Survey on stability of feature selection [J ] . Journal of Software , 2018 , 29 ( 9 ): 2559 - 2579 . (in Chinese)

张俐 , 陈小波 . 动态加权条件互信息的特征选择算法 [J ] . 电子与信息学报 , 2021 , 43 ( 10 ): 3028 - 3034 .

ZHANG L , CHEN X B . Feature selection algorithm for dynamically weighted conditional mutual information [J ] . Journal of Electronics & Information Technology , 2021 , 43 ( 10 ): 3028 - 3034 . (in Chinese)

江兵兵 , 何文达 , 吴兴宇 , 等 . 基于自适应图学习的半监督特征选择 [J ] . 电子学报 , 2022 , 50 ( 7 ): 1643 - 1652 .

JIANG B B , HE W D , WU X Y , et al . Semi-supervised feature selection with adaptive graph learning [J ] . Acta Electronica Sinica , 2022 , 50 ( 7 ): 1643 - 1652 . (in Chinese)

任永功 , 王玉玲 , 刘洋 , 等 . 基于用户相关性的动态网络媒体数据无监督特征选择算法 [J ] . 计算机学报 , 2018 , 41 ( 7 ): 1517 - 1535 .

REN Y G , WANG Y L , LIU Y , et al . Unsupervised feature selection algorithm for dynamic network media data based on user correlation [J ] . Chinese Journal of Computers , 2018 , 41 ( 7 ): 1517 - 1535 . (in Chinese)

SOLORIO-FERNÁNDEZ S , CARRASCO-OCHOA J A , MARTÍNEZ-TRINIDAD J F . A review of unsupervised feature selection methods [J ] . Artificial Intelligence Review , 2020 , 53 ( 2 ): 907 - 948 .

LUO M N , NIE F P , CHANG X J , et al . Adaptive unsupervised feature selection with structure regularization [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2018 , 29 ( 4 ): 944 - 956 .

WANG A G , AN N , CHEN G L , et al . Accelerating wrapper-based feature selection with K-nearest-neighbor [J ] . Knowledge-Based Systems , 2015 , 83 : 81 - 91 .

AMBUSAIDI M A , HE X J , NANDA P , et al . Building an intrusion detection system using a filter-based feature selection algorithm [J ] . IEEE Transactions on Computers , 2016 , 65 ( 10 ): 2986 - 2998 .

PEDREGOSA F , VAROQUAUX G , GRAMFORT A , et al . Scikit-learn: machine learning in python [J ] . Journal of Machine Learning Research , 2011 , 12 : 2825 - 2830 .

高慧敏 , 王云鹤 , 卞闯 , 等 . 基于混合进化算法的特征选择方法研究 [J ] . 电子学报 , 2023 , 51 ( 6 ): 1619 - 1636 .

GAO H M , WANG Y H , BIAN C , et al . Research on feature selection based on hybrid evolutionary algorithm [J ] . Acta Electronica Sinica , 2023 , 51 ( 6 ): 1619 - 1636 . (in Chinese)

CAI D , ZHANG C Y , HEX F , et al . Unsupervised feature selection for multi-cluster data [C ] // Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . New York : ACM , 2010 : 333 - 342 .

YANG Y , SHEN H T , MA Z G , et al . ℓ 2 , 1 -norm regularized discriminative feature selection for unsupervised learning [C ] // Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence-Volume Volume Two . New York : ACM , 2011: 1589 - 1594 .

LI Z C , YANG Y , LIU J , et al . Unsupervised feature selection using nonnegative spectral analysis [C ] // Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence . New York : ACM , 2012 : 1026 - 1032 .

QIAN M J , ZHAI C X , QIAN M J , et al . Robust unsupervised feature selection [C ] // Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence . New York : ACM , 2013 : 1621 - 1627 .

WANG S H , TANG J L , LIU H , et al . Embedded unsupervised feature selection [C ] // Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence . New York : ACM , 2015 : 470 - 476 .

HAN D , KIM J . Unsupervised simultaneous orthogonal basis clustering feature selection [C ] // 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2015 : 5016 - 5023 .

NIE F P , ZHU W , LI X L , et al . Unsupervised feature selection with structured graph optimization [C ] // Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence . New York : ACM , 2016 : 1302 - 1308 .

LI X L , ZHANG H , ZHANG R , et al . Generalized uncorrelated regression with adaptive graph for unsupervised feature selection [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2019 , 30 ( 5 ): 1587 - 1595 .

WANG C C , WANG J , GU Z C , et al . Unsupervised feature selection by learning exponential weights [J ] . Pattern Recognition , 2024 , 148 : 110183 .

LIN X C , GUAN J W , CHEN B L , et al . Unsupervised feature selection via orthogonal basis clustering and local structure preserving [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2022 , 33 ( 11 ): 6881 - 6892 .

LUO C , ZHENG J , LI T R , et al . Orthogonally constrained matrix factorization for robust unsupervised feature selection with local preserving [J ] . Information Sciences , 2022 , 586 : 662 - 675 .

ZHANG R , ZHANG Y X , LI X L . Unsupervised feature selection via adaptive graph learning and constraint [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2022 , 33 ( 3 ): 1355 - 1362 .

ZHU P C , HOU X , TANG K K , et al . Unsupervised feature selection through combining graph learning and ℓ 2, 0 -norm constraint [J ] . Information Sciences , 2023 , 622 : 68 - 82 .

ZHOU Q , WANG Q Q , GAO Q X , et al . Unsupervised discriminative feature selection via contrastive graph learning [J ] . IEEE Transactions on Image Processing , 2024 , 33 : 972 - 986 .

LI Z X , NIE F P , BIAN J T , et al . Sparse pca via ℓ 2, p -norm regularization for unsupervised feature selection [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2023 , 45 ( 4 ): 5322 - 5328 .

NIE F P , MA Z Y , WANG J Y , et al . Fast sparse discriminative K-means for unsupervised feature selection [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2024 , 35 ( 7 ): 9943 - 9957 .

KARAMI S , SABERI-MOVAHED F , TIWARI P , et al . Unsupervised feature selection based on variance-covariance subspace distance [J ] . Neural Networks , 2023 , 166 : 188 - 203 .

CAO Z W , XIE X J . Structure learning with consensus label information for multi-view unsupervised feature selection [J ] . Expert Systems with Applications , 2024 , 238 : 121893 .

CAO Z W , XIE X J , LI Y Q . Multi-view unsupervised feature selection with consensus partition and diverse graph [J ] . Information Sciences , 2024 , 661 : 120178 .

ZHOU S X , SONG P . Consistency-exclusivity guided unsupervised multi-view feature selection [J ] . Neurocomputing , 2024 , 569 : 127119 .

ZADEH L A . Fuzzy sets [J ] . Information and Control , 1965 , 8 ( 3 ): 338 - 353 .

PAL S K , DE R K , BASAK J . Unsupervised feature evaluation: A neuro-fuzzy approach [J ] . IEEE Transactions on Neural Networks , 2000 , 11 ( 2 ): 366 - 376 .

LI Y , LU B L , WU Z F . A hybrid method of unsupervised feature selection based on ranking [C ] // The 18th International Conference on Pattern Recognition (ICPR'06) . Piscataway : IEEE , 2006 : 687 - 690 .

MAC PARTHALÁIN N , JENSEN R . Unsupervised fuzzy-rough set-based dimensionality reduction [J ] . Information Sciences , 2013 , 229 : 106 - 121 .

WANG R , BIAN J T , NIE F P , et al . Unsupervised discriminative projection for feature selection [J ] . IEEE Transactions on Knowledge and Data Engineering , 2022 , 34 ( 2 ): 942 - 953 .

JIANG K , ZHU L , SUN Q D . Unsupervised Feature Selection via Fuzzy K-means and Sparse Projection [M ] // Lecture Notes in Computer Science . Singapore : Springer Nature Singapore , 2024 : 224 - 239 .

PAL N R , BEZDEK J C . On cluster validity for the fuzzy c-means model [J ] . IEEE Transactions on Fuzzy Systems , 1995 , 3 ( 3 ): 370 - 379 .

潘金艳 , 高朋 , 高云龙 , 等 . 基于可靠性的鲁棒模糊聚类 [J ] . 控制理论与应用 , 2021 , 38 ( 4 ): 516 - 528 .

PAN J Y , GAO P , GAO Y L , et al . Reliability-based of robust fuzzy flustering [J ] . Control Theory & Applications , 2021 , 38 ( 4 ): 516 - 528 . (in Chinese)

ZHANG R , TONG H H . Robust principal component analysis with adaptive neighbors [J ] . Advances in Neural Information Processing Systems 32 , 2019 , 32 : 6961 - 6969 .

NIE F P , HUANG H , CAI X , et al . Efficient and robust feature selection via joint ℓ 2,1 -norms minimization [C ] // Proceedings of the 24th International Conference on Neural Information Processing Systems-Volume 2 . New York : ACM , 2010 : 1813 - 1821 .

AFONSO M V , BIOUCAS-DIAS J M , FIGUEIREDO M A T . An augmented Lagrangian approach to the constrained optimization formulation of imaging inverse problems [J ] . IEEE Transactions on Image Processing , 2011 , 20 ( 3 ): 681 - 695 .

BEZDEK J C , EHRLICH R , FULL W . FCM: The fuzzy c-means clustering algorithm [J ] . Computers & Geosciences , 1984 , 10 ( 2/3 ): 191 - 203 .

NIE F P , WU D Y , WANG R , et al . Truncated robust principle component analysis with a general optimization framework [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022 , 44 ( 2 ): 1081 - 1097 .

VIKLANDS T . Algorithms for the Weighted Orthogonal Procrustes Problem and Other Least Squares Problems [D ] . Umea : Department of Computing Science, Umea University , 2006 .

KUHN H W . The Hungarian method for the assignment problem [J ] . Naval Research Logistics Quarterly , 1955 , 2 ( 1/2 ): 83 - 97 .

DEMŠAR J . Statistical comparisons of classifiers over multiple data sets [J ] . Journal of Machine Learning Research , 2006 , 7 : 1 - 30 .

ZHANG R X , HANG S J , SUN Z S , et al . Anchor-based fast spectral ensemble clustering [J ] . Information Fusion , 2025 , 113 : 102587 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Unknown Encrypted Malicious Traffic Detection via Burst Feature Token Self-Learning

Comprehensive Causality Multi-Objective Counterfactual Explanation with Feature Selection

Feature Anomaly Detection and Pseudo-Label Regression for Adversarial Domain Adaptation

Estimation of Signal-to-Noise Ratio Based on Feature Selection and Weight Optimization of High-Order Moments

Related Author

SHI Shu-guang

ZHAO Zhi-xiang

SHEN Meng

JIA Ji-zhe

ZHAO Bu-fan

CHANG Li-yuan

YANG Ming

REN Chen-chen

Related Institution

School of Cyberspace Science and Technology, Beiijing Institute of Technology

Department of Basic Ability, China Telecom Cybersecurity Technology Co., Ltd.

National Supercomputing Center in Jinan

College of Information Science and Engineering, Hunan Normal University

School of Information and Control Engineering, China University of Mining and Technology

⁰