Transposed Projection Envelope Linear Discriminant Analysis

LI Yongming; ZHAO Wenqiang; LI Fan; ZHANG Xiaoheng; WANG Pin

doi:10.12263/DZXB.20250938

您当前的位置：

首页 >

文章列表页 >

Transposed Projection Envelope Linear Discriminant Analysis

PAPERS | 更新时间：2026-06-04

- Transposed Projection Envelope Linear Discriminant Analysis
- ACTA ELECTRONICA SINICA Vol. 54, Issue 1, Pages: 32-49(2026)
- 作者机构：
  
  1.重庆大学微电子与通信工程学院，重庆 400044
  2.重庆交通大学信息科学与工程学院，重庆 400074
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China(62201106)
- DOI：10.12263/DZXB.20250938
  CLC： TP391.4;
- Received：16 December 2025，
  
  Accepted：31 December 2025，
  
  Published：25 January 2026
- 稿件说明：
移动端阅览
李勇明, 赵文强, 李帆, 等. 转置投影包络线性判别分析[J]. 电子学报, 2026, 54(01): 32-49.

LI Yongming, ZHAO Wenqiang, LI Fan, et al. Transposed Projection Envelope Linear Discriminant Analysis[J]. Acta Electronica Sinica, 2026, 54(01): 32-49.
李勇明, 赵文强, 李帆, 等. 转置投影包络线性判别分析[J]. 电子学报, 2026, 54(01): 32-49. DOI：10.12263/DZXB.20250938

LI Yongming, ZHAO Wenqiang, LI Fan, et al. Transposed Projection Envelope Linear Discriminant Analysis[J]. Acta Electronica Sinica, 2026, 54(01): 32-49. DOI：10.12263/DZXB.20250938

摘要

线性判别分析（Linear Discriminant Analysis，LDA）是一种运用广泛的特征提取方法，其以Fisher判别准则为指导，增强子空间中异类样本区分性和同类样本紧凑性，提高了降维结果质量，具有成熟、易解释、简单高效等优点，迄今为止仍是学术界和产业界的研究热点之一。诸多学者对LDA进行了改进以进一步提高其性能，然而这些LDA变体方法直接建模在原样本粒度上，只利用了样本自身存在的信息。由DIK（Data-Information-Knowledge）模型表明，人类获取知识有三个层次，即数据层、信息层，以及知识层，数据首先应当被转换为信息，然后再从信息中学习知识。由人类认知机制表明，信息层不仅包含了原始输入自身信息，还包含了与其相似输入间的关联信息。将其类比于LDA降维过程，提取到的特征即为信息，该信息也应当包含相似样本间的关联信息，以提高下游任务性能。且已有相关研究表明，相似样本间存在的关联信息对机器学习模型构建、知识获取至关重要。即现有LDA存在缺陷，其对样本信息的利用不够完备。针对上述问题，本文提出了转置投影包络线性判别分析（Transposed Projection Envelope Linear Discriminant Analysis，TPELDA）。首先，通过转置投影将原始样本转换为包含相似样本间关联信息的包络样本，转置投影的核心思想为在样本维度上对一批最近邻样本进行降维，使得降维所得包络样本尽量包含该批样本所含信息；随后基于包络样本利用Fisher判别准则学习降维子空间；同时引入分布差异惩罚项确保降维子空间对原始样本的适配性；最后通过联合优化，该方法在考虑相似样本间关联信息的基础上使得投影到子空间中的样本具有更好的判别特征，即该特征同时代表了样本自身信息以及相似样本间存在的关联信息。实验结果表明，TPELDA在给定的多个数据集上相比相关对比方法性能更优，提升范围在2.25%至13.19%之间。此外，结合其他实验结果，表明了本文方法的有效性。

Abstract

Linear discriminant analysis (LDA) is a widely used feature extraction method guided by Fisher’s discriminant criterion. It enhances the separability of dissimilar samples and the compactness of similar samples within the subspace

thereby improving the quality of dimensionality reduction results. With its mature

interpretable

simple

and efficient advantages

it remains one of the research hotspots in both academia and industry to date. Numerous scholars have refined LDA to further enhance its performance. However

these LDA variants model directly at the original sample granularity

utilizing only the information inherent within the samples themselves. The data-information-knowledge (DIK) model indicates that human knowledge acquisition occurs across three levels: data

information

and knowledge. Data must first be transformed into information

from which knowledge is then learned. Human cognitive mechanisms reveal that the information layer encompasses not only the inherent properties of raw inputs but also correlation information among similar inputs. Analogously

in LDA’s dimensionality reduction process

extracted features represent information that should also incorporate correlation information among similar samples to enhance downstream task performance. Furthermore

existing research demonstrates that the correlation information between similar samples is crucial for machine learning model construction and knowledge acquisition. This indicates that existing LDA has limitations

as it does not fully utilize sample information. To address these issues

this paper proposes transposed projection envelope linear discriminant analysis (TPELDA). First

transposed projection transforms original samples into envelope samples that encapsulate correlation information among similar samples. The core idea of transposed projection is to reduce the dimensionality of a batch of nearest neighbor samples along the sample dimension

ensuring the resulting envelope sample retain as much information as possible from the original batch. Subsequently

Fisher’s discriminant criterion is employed to learn a reduced-dimension subspace based on these envelope samples. A distribution-difference penalty term is introduced to ensure the reduced subspace’s adaptability to the original samples. Finally

through joint optimization

this method enhances the discriminative features of samples projected into the subspace by incorporating the correlation information among similar samples. Thus

the resulting features simultaneously represent both the intrinsic information of individual samples and the correlation information among similar samples. Experimental results demonstrate that TPELDA outperforms relevant comparison methods across multiple datasets

achieving performance improvements ranging from 2.25% to 13.19%. Furthermore

combined with other experimental findings

the effectiveness of the proposed method is confirmed.

关键词

Keywords

references

Lu Bin , Wang Fuwang , Chen Junxiang , et al . Improving two-dimensional linear discriminant analysis with L1 norm for optimizing EEG signal [J ] . Information Sciences , 2025 , 690 : 121585 . DOI: 10.1016/j.ins.2024.121585 http://dx.doi.org/10.1016/j.ins.2024.121585

刘中华 , 周静波 , 陈燚 , 等 . 距离保持投影非线性降维技术的可视化与分类 [J ] . 电子学报 , 2009 , 37 ( 8 ): 1820 - 1825 .

Liu Zhonghua , Zhou Jingbo , Chen Yi , et al . Non-linear dimensionality reduction techniques of distance-preserving projection for visualization and classification [J ] . Acta Electronica Sinica , 2009 , 37 ( 8 ): 1820 - 1825 . (in Chinese)

Yang Xuanhao , Che Hangjun , Leung M F , et al . Self-paced regularized adaptive multi-view unsupervised feature selection [J ] . Neural Networks , 2024 , 175 : 106295 . DOI: 10.1016/j.neunet.2024.106295 http://dx.doi.org/10.1016/j.neunet.2024.106295

Wang Xiang , Zhu Junxing , Xu Zichen , et al . Local nonlinear dimensionality reduction via preserving the geometric structure of data [J ] . Pattern Recognition , 2023 , 143 : 109663 . DOI: 10.1016/j.patcog.2023.109663 http://dx.doi.org/10.1016/j.patcog.2023.109663

尤殿龙 , 郭松 , 赵春慧 , 等 . 面向分类的流特征在线特征选择算法 [J ] . 电子学报 , 2020 , 48 ( 2 ): 321 - 332 .

You Dianlong , Guo Song , Zhao Chunhui , et al . Online feature selection with streaming features for classification [J ] . Acta Electronica Sinica , 2020 , 48 ( 2 ): 321 - 332 . (in Chinese)

Greenacre M , Groenen P J F , Hastie T , et al . Principal component analysis [J ] . Nature Reviews Methods Primers , 2022 , 2 ( 1 ): 100 . DOI: 10.1038/s43586-022-00184-w http://dx.doi.org/10.1038/s43586-022-00184-w

Tenenbaum J B , De Silva V , Langford J C . A global geometric framework for nonlinear dimensionality reduction [J ] . Science , 2000 , 290 ( 5500 ): 2319 - 2323 . DOI: 10.1126/science.290.5500.2319 http://dx.doi.org/10.1126/science.290.5500.2319

Zhao Shuping , Zhang B , Yang Jian , et al . Linear discriminant analysis [J ] . Nature Reviews Methods Primers , 2024 , 4 ( 1 ): 70 . DOI: 10.1038/s43586-024-00346-y http://dx.doi.org/10.1038/s43586-024-00346-y

Dufrenois F , Jbilou K . A support vector machine formulation for linear and kernel discriminant analysis [J ] . Neurocomputing , 2025 , 622 : 129273 . DOI: 10.1016/j.neucom.2024.129273 http://dx.doi.org/10.1016/j.neucom.2024.129273

Li Fan , Li Yongming , Shen Yinghua , et al . Deep fuzzy envelope sample generation mechanism for imbalanced ensemble classification [J ] . IEEE Transactions on Fuzzy Systems , 2024 , 32 ( 3 ): 1248 - 1262 . DOI: 10.1109/tfuzz.2023.3321768 http://dx.doi.org/10.1109/tfuzz.2023.3321768

Ma Jie , Cheng Huan , Chen Hong , et al . Envelope rotation forest: A novel ensemble learning method for classification [J ] . Neurocomputing , 2025 , 618 : 129059 . DOI: 10.1016/j.neucom.2024.129059 http://dx.doi.org/10.1016/j.neucom.2024.129059

Mahadi M , Ballal T , Moinuddin M , et al . Regularized linear discriminant analysis using a nonlinear covariance matrix estimator [J ] . IEEE Transactions on Signal Processing , 2024 , 72 : 1049 - 1064 . DOI: 10.1109/tsp.2024.3361715 http://dx.doi.org/10.1109/tsp.2024.3361715

Chu Delin , Liao Lizhi , Ng M K . Sparse orthogonal linear discriminant analysis [J ] . SIAM Journal on Scientific Computing , 2012 , 34 ( 5 ): A2421 - A2443 . DOI: 10.1137/110851377 http://dx.doi.org/10.1137/110851377

Wang Jingyu , Wang Lin , Nie Feiping , et al . A novel formulation of trace ratio linear discriminant analysis [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2022 , 33 ( 10 ): 5568 - 5578 . DOI: 10.1109/tnnls.2021.3071030 http://dx.doi.org/10.1109/tnnls.2021.3071030

Ye Qiaolin , Yang Jie , Zheng Hao , et al . Convergence analysis on trace ratio linear discriminant analysis algorithms [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2025 , 36 ( 2 ): 3878 - 3881 . DOI: 10.1109/tnnls.2024.3355422 http://dx.doi.org/10.1109/tnnls.2024.3355422

Zhou Yang , Sun Shiliang . Manifold partition discriminant analysis [J ] . IEEE Transactions on Cybernetics , 2017 , 47 ( 4 ): 830 - 840 . DOI: 10.1109/tcyb.2016.2529299 http://dx.doi.org/10.1109/tcyb.2016.2529299

Nie Feiping , Wang Zheng , Wang Rong , et al . Adaptive local linear discriminant analysis [J ] . ACM Transactions on Knowledge Discovery from Data , 2020 , 14 ( 1 ): 9 . DOI: 10.1145/3369870 http://dx.doi.org/10.1145/3369870

Qu Jinlong , Zhao Xiaowei , Xiao Yun , et al . Adaptive manifold graph representation for two-dimensional discriminant projection [J ] . Knowledge-Based Systems , 2023 , 266 : 110411 . DOI: 10.1016/j.knosys.2023.110411 http://dx.doi.org/10.1016/j.knosys.2023.110411

Feng Wenyi , Wang Zhe , Cao Xiqing , et al . Discriminative sparse subspace learning with manifold regularization [J ] . Expert Systems with Applications , 2024 , 249 : 123831 . DOI: 10.1016/j.eswa.2024.123831 http://dx.doi.org/10.1016/j.eswa.2024.123831

Zhu Yufei , Lai Zhihui , Gao Can , et al . Generalized robust linear discriminant analysis for jointly sparse learning [J ] . Applied Intelligence , 2024 , 54 ( 19 ): 9508 - 9523 . DOI: 10.1007/s10489-024-05632-6 http://dx.doi.org/10.1007/s10489-024-05632-6

Zhou Jianhang , Zhang Qi , Zeng Shaoning , et al . Latent linear discriminant analysis for feature extraction via isometric structural learning [J ] . Pattern Recognition , 2024 , 149 : 110218 . DOI: 10.1016/j.patcog.2023.110218 http://dx.doi.org/10.1016/j.patcog.2023.110218

Li Zhengxin , Nie Feiping , Wu Danyang , et al . Sparse trace ratio LDA for supervised feature selection [J ] . IEEE Transactions on Cybernetics , 2024 , 54 ( 4 ): 2420 - 2433 . DOI: 10.1109/tcyb.2023.3264907 http://dx.doi.org/10.1109/tcyb.2023.3264907

Wen Jie , Fang Xiaozhao , Cui Jinrong , et al . Robust sparse linear discriminant analysis [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2019 , 29 ( 2 ): 390 - 403 . DOI: 10.1109/tcsvt.2018.2799214 http://dx.doi.org/10.1109/tcsvt.2018.2799214

Li Shuyi , Zhang Hengmin , Ma Ruijun , et al . Linear discriminant analysis with generalized kernel constraint for robust image classification [J ] . Pattern Recognition , 2023 , 136 : 109196 . DOI: 10.1016/j.patcog.2022.109196 http://dx.doi.org/10.1016/j.patcog.2022.109196

Liu Jingjing , Feng Manlong , Xiu Xianchao , et al . Towards robust and sparse linear discriminant analysis for image classification [J ] . Pattern Recognition , 2024 , 153 : 110512 . DOI: 10.1016/j.patcog.2024.110512 http://dx.doi.org/10.1016/j.patcog.2024.110512

Yang Xiaojun , Cao Chuanjie , Zhou Keyi , et al . A novel linear discriminant analysis based on alternate ratio sum minimization [J ] . Information Sciences , 2025 , 689 : 121444 . DOI: 10.1016/j.ins.2024.121444 http://dx.doi.org/10.1016/j.ins.2024.121444

Li Xuelong , Zhang Yunxing , Zhang Rui . Self-weighted unsupervised LDA [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2023 , 34 ( 3 ): 1627 - 1632 . DOI: 10.1109/tnnls.2021.3105196 http://dx.doi.org/10.1109/tnnls.2021.3105196

Ikotun A M , Ezugwu A E , Abualigah L , et al . K-means clustering algorithms: A comprehensive review, variants analysis, and advances in the era of big data [J ] . Information Sciences , 2023 , 622 : 178 - 210 . DOI: 10.1016/j.ins.2022.11.139 http://dx.doi.org/10.1016/j.ins.2022.11.139

Lam B S Y , Choy S K , Yu C K W . Linear discriminant analysis with trimmed and difference distribution modeling [J ] . Knowledge-Based Systems , 2024 , 299 : 112093 . DOI: 10.1016/j.knosys.2024.112093 http://dx.doi.org/10.1016/j.knosys.2024.112093

Frické M H . Data-information-knowledge-wisdom (DIKW) pyramid, framework, continuum [M ] //Schintler L A, McNeely C L. Encyclopedia of big data . Cham : Springer , 2018 : 1 - 4 . DOI: 10.1007/978-3-319-32001-4_331-1 http://dx.doi.org/10.1007/978-3-319-32001-4_331-1

Gazzaniga M S , Mangun G R , Blakemore S J . The Cognitive Neurosciences [M ] . 5th ed . Cambridge : MIT Press , 2014 . DOI: 10.7551/mitpress/9504.001.0001 http://dx.doi.org/10.7551/mitpress/9504.001.0001

Xia Shuyin , Dai Xiaochuan , Wang Guoyin , et al . An efficient and adaptive granular-ball generation method in classification problem [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2024 , 35 ( 4 ): 5319 - 5331 . DOI: 10.1109/tnnls.2022.3203381 http://dx.doi.org/10.1109/tnnls.2022.3203381

Peng Xiaoli , Wang Ping , Xia Shuyin , et al . VPGB: A granular-ball based model for attribute reduction and classification with label noise [J ] . Information Sciences , 2022 , 611 : 504 - 521 . DOI: 10.1016/j.ins.2022.08.066 http://dx.doi.org/10.1016/j.ins.2022.08.066

李帆 , 张小恒 , 李勇明 , 等 . 基于包络学习和分级结构一致性机制的不平衡集成算法 [J ] . 电子学报 , 2024 , 52 ( 3 ): 751 - 761 .

Li Fan , Zhang Xiaoheng , Li Yongming , et al . Imbalanced ensemble algorithm based on envelope learning and hierarchical structure consistency mechanism [J ] . Acta Electronica Sinica , 2024 , 52 ( 3 ): 751 - 761 . (in Chinese)

Li Yongming , Liu Chengyu , Wang Pin , et al . Envelope multi-type transformation ensemble algorithm of Parkinson speech samples [J ] . Applied Intelligence , 2023 , 53 ( 12 ): 15957 - 15978 . DOI: 10.1007/s10489-022-04345-y http://dx.doi.org/10.1007/s10489-022-04345-y

Schölkopf B , Smola A , Müller K R . Nonlinear component analysis as a kernel eigenvalue problem [J ] . Neural Computation , 1998 , 10 ( 5 ): 1299 - 1319 . DOI: 10.1162/089976698300017467 http://dx.doi.org/10.1162/089976698300017467

Gretton A , Borgwardt K M , Rasch M J , et al . A kernel two-sample test [J ] . The Journal of Machine Learning Research , 2012 , 13 : 723 - 773 .

Wright S J . Coordinate descent algorithms [J ] . Mathematical Programming , 2015 , 151 ( 1 ): 3 - 34 . DOI: 10.1007/s10107-015-0892-3 http://dx.doi.org/10.1007/s10107-015-0892-3

Nie Feiping , Yuan Jianjun , Huang Heng . Optimal mean robust principal component analysis [C ] // Proceedings of the 31st International Conference on Machine Learning . Cambridge : JMLR.org , 2014 : II-1062-II-1070.

Wang Jingyu , Wang Hongmei , Nie Feiping , et al . Ratio sum versus sum ratio for linear discriminant analysis [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022 , 44 ( 12 ): 10171 - 10185 . DOI: 10.1109/tpami.2021.3133351 http://dx.doi.org/10.1109/tpami.2021.3133351

Yi Jihai , Duan Huiyu , Wang Jikui , et al . Structure preserved fast dimensionality reduction [J ] . Applied Soft Computing , 2024 , 162 : 111817 . DOI: 10.1016/j.asoc.2024.111817 http://dx.doi.org/10.1016/j.asoc.2024.111817

Zhang Rui , Zhang Yunxing , Li Xuelong . Unsupervised feature selection via adaptive graph learning and constraint [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2022 , 33 ( 3 ): 1355 - 1362 . DOI: 10.1109/tnnls.2020.3042330 http://dx.doi.org/10.1109/tnnls.2020.3042330

Van Der Maaten L , Hinton G . Visualizing data using t-SNE [J ] . Journal of Machine Learning Research , 2008 , 9 ( 86 ): 2579 - 2605 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Research on Dynamic Continuous Emotional Recognition of EEG Based on Improved TCNN Algorithm

Manifold Nearest Neighbor Sample Envelope and Hierarchical Multitype Transform Algorithm for Ensemble Learning

Research on Lightweight Sensing Technology Based on Single Antenna and Multiple Subcarriers

Exploring Freshness Modeling for Encrypted Traffic Analysis: From “Data Retention” to “Value Accumulation”

Design of a Self-Biased Low-Jitter Low-Power Phase-Locked Loop for SerDes

Related Author

LI Yongming

ZHAO Wenqiang

LI Fan

ZHANG Xiaoheng

WANG Pin

JIE Li-lin

LIU Yong

WANG Ming-xun

Related Institution

School of Microelectronics and Communication Engineering, Chongqing University

School of Information Science and Engineering, Chongqing Jiaotong University

School of Instrument Science and Optoelectronic Engineering, Nanchang Hangkong University

School of Aviation Services and Music, Nanchang Hangkong University

School of Software, Jiangxi Agricultural University

⁰