Feature Masking and Contrastive Learning Integrating Multi-Dimensional Decorrelation in Sequential Recommendation

QIAN Zhongsheng; LIU Jinping; LI Yulong; FAN Fuyu; CHEN Chao

doi:10.12263/DZXB.20250551

您当前的位置：

首页 >

文章列表页 >

Feature Masking and Contrastive Learning Integrating Multi-Dimensional Decorrelation in Sequential Recommendation

PAPERS | 更新时间：2026-05-28

- Feature Masking and Contrastive Learning Integrating Multi-Dimensional Decorrelation in Sequential Recommendation
- ACTA ELECTRONICA SINICA Vol. 54, Issue 2, Pages: 875-898(2026)
- 作者机构：
  
  江西财经大学计算机与人工智能学院，江西南昌 330013
- 作者简介：
- 基金信息：
  
  Foundation Item(s):　Foundation Item(s): National Natural Science Foundation of China(62262025);Jiangxi Poyang Support Program for Talents of China-Major Discipline Academic and Technical Leaders Training Program of Jiangxi Province-Leading Talent (Academic)(20243BCE51024)
- DOI：10.12263/DZXB.20250551
  CLC： TP399;
- Received：24 June 2025，
  
  Accepted：26 December 2025，
  
  Published：25 February 2026
- 稿件说明：
移动端阅览
钱忠胜, 刘金平, 李玉龙, 等. 特征掩码与对比学习融合多维度去过度相关的序列推荐[J]. 电子学报, 2026, 54(02): 875-898.

QIAN Zhongsheng, LIU Jinping, LI Yulong, et al. Feature Masking and Contrastive Learning Integrating Multi-Dimensional Decorrelation in Sequential Recommendation[J]. Acta Electronica Sinica, 2026, 54(02): 875-898.
钱忠胜, 刘金平, 李玉龙, 等. 特征掩码与对比学习融合多维度去过度相关的序列推荐[J]. 电子学报, 2026, 54(02): 875-898. DOI：10.12263/DZXB.20250551

QIAN Zhongsheng, LIU Jinping, LI Yulong, et al. Feature Masking and Contrastive Learning Integrating Multi-Dimensional Decorrelation in Sequential Recommendation[J]. Acta Electronica Sinica, 2026, 54(02): 875-898. DOI：10.12263/DZXB.20250551

摘要

近年来，基于自注意力机制的序列推荐模型在用户行为建模中展现出了显著的效果，尤其是在处理长程依赖关系方面表现突出。然而，此类模型（如Transformer类方法）在深层编码过程中，高阶表示会因多层聚合而逐渐趋同，个性化信号被削弱，从而产生过度平滑问题，且该问题在此类模型中往往被忽视；与此同时，特征维度间的高度相关会带来冗余与噪声传播，削弱模型对重要特征的辨识能力，从而进一步限制模型的泛化能力。为此，本文提出特征掩码与对比学习融合多维度去过度相关的序列推荐模型（feature Masking and Contrastive learning integrating Multi-Dimensional decorrelation in Sequential Recommendation，MCMD-SR）。首先，设计自注意力感知的特征掩码机制，依据自注意力得分衡量各维度贡献，对低贡献且易导致表示相似化的特征进行针对性遮蔽，并提出随层数递减的对数掩码率衰减策略，使浅层施加强扰动以打破局部高相似特征，深层保持适度扰动以持续抑制过度聚合。进一步地，在掩码后的最终层表示与原始浅层表示之间构建对比学习任务，通过拉近同一序列的正样本对、推远不同序列或不同特征的负样本对，强化差异化与个性化语义，提升嵌入空间的区分能力。其次，提出多维度自适应去过度相关模块，在注意力掩码后的特征矩阵上分别从列间与层间计算皮尔逊相关系数（Pearson Correlation Coefficient，PCC），并依据相关强度自适应分配惩罚权重，在保持总体正则化强度可控的同时抑制冗余维度与冗余层间依赖，从局部（列间）与全局（层间）双视角降低特征冗余，提升关键特征辨识度。最后，将自注意力掩码机制、对比学习模块与多维度自适应去过度相关模块的损失进行多任务联合优化，使三类约束相互补充，稳定训练并提升嵌入质量与模型泛化性。在Beauty、Yelp、LastFM和ML-1M四个公开数据集上，本文模型与11个经典及最新的序列推荐模型进行了对比。实验结果表明，在命中率（Hit Ratio， HR）和归一化折损累积增益（Normalized Discounted Cumulative Gain，NDCG）两个指标上，所提模型MCMD-SR相对已有最优基线模型分别平均最少提升2.13%和1.67%，验证了本文模型在推荐性能上的有效性。此外，本文还通过消融实验和参数敏感性实验分析，验证了各模块的必要性及其协同有效性，进一步阐明了模型具有良好的泛化能力。

Abstract

Recently

self-attention-based sequential recommendation models have demonstrated remarkable effectiveness in user behavior modeling. However

these models tend to suffer from an over-smoothing problem during deep encoding. Repeated aggregation across multi layers makes high-order representations become increasingly similar

which gradually weakens personalized signals. Meanwhile

the high correlation among feature dimensions introduces redundancy and noise propagation

which weakens the model’s ability to identify important features and consequently limits its generalization capability. To address these challenges

this work proposes feature masking and contrastive learning integrating multi-dimensional decorrelation in sequential recommendation (MCMD-SR)

a feature masking and contrastive learning model integrating multi-dimensional decorrelation in sequential recommendation. Firstly

we design a feature masking mechanism based on self-attention. This mechanism measures the contribution of each feature dimension with attention scores. It then selectively masks features with low-contribution which are prone to inducing representation homogenization. In addition

we also introduce a logarithmic mask-rate decay strategy across layers. This strategy applies stronger perturbations in shallow layers to break high-similarity features locally. In deeper layers

it maintains moderate perturbations to continuously suppress excessive aggregation. Furthermore

a contrastive learning task is constructed between the masked final-layer representations and the original shallow-layer representations. The proposed method pulls together positive pairs from the same sequence and pushes apart negative pairs from different sequences or feature dimensions. The proposed method reinforces discriminative and personalized semantics. Thereby it improves the separability of the embedding space. Secondly

we propose a multi-dimensional adaptive decorrelation module. Based on the attention-masked feature matrix

Pearson correlation coefficients (PCC) are computed from both column-wise and layer-wise perspectives. Penalty weights are adaptively assigned according to the correlation strength. This suppresses redundant dimensions and inter-layer dependencies. Meanwhile

it keeps the overall regularization strength controllable. This dual-view decorrelation strategy reduces feature redundancy from both local (column-wise) and global (layer-wise) perspectives resulting in improving the identification of key features. Finally

the self-attention masking mechanism

the contrastive learning module

and the multi-dimensional adaptive decorrelation module are jointly optimized in a multi-task learning framework. These complementary constraints stabilize training and improve embedding quality as well as model generalization. Extensive experiments are conducted on 4 public datasets

where the proposed method is compared with 11 classical and state-of-the-art sequential recommendation models. Experimental results show that MCMD-SR achieves average improvements of 2.13% and 1.67% over the strongest baseline in terms of hit ratio (HR) and normalized discounted cumulative gain (NDCG)

respectively. In addition

ablation studies and parameter sensitivity analysis further verify the necessity of each module and their synergistic effectiveness

thereby further clarifying the strong generalization capability of our model.

关键词

Keywords

references

Shin Y , Choi J , Wi H , et al . An attentive inductive bias for sequential recommendation beyond the self-attention [C ] // Proceedings of the 38th AAAI Conference on Artificial Intelligence . Vancouver : AAAI Press , 2024 : 8984 - 8992 . DOI: 10.1609/aaai.v38i8.28747 http://dx.doi.org/10.1609/aaai.v38i8.28747

Fan Ziwei , Liu Zhiwei , Peng Hao , et al . Addressing the rank degeneration in sequential recommendation via singular spectrum smoothing [C ] // Proceedings of the ACM Conference on Recommender Systems . Woodstock : ACM , 2023 : 11986 .

Zhou Kun , Yu Hui , Zhao W X , et al . Filter-enhanced MLP is all you need for sequential recommendation [C ] // Proceedings of the ACM Web Conference 2022 . Lyon : ACM , 2022 : 2388 - 2399 . DOI: 10.1145/3485447.3512111 http://dx.doi.org/10.1145/3485447.3512111

Du Xinyu , Yuan Huanhuan , Zhao Pengpeng , et al . Frequency enhanced hybrid attention network for sequential recommendation [C ] // Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval . Taipei, China : ACM , 2023 : 78 - 88 . DOI: 10.1145/3539618.3591689 http://dx.doi.org/10.1145/3539618.3591689

Cui Ziqiang , Wu Haolun , He Bowei , et al . Context matters: Enhancing sequential recommendation with context-aware diffusion-based contrastive learning [C ] // Proceedings of the 33rd ACM International Conference on Information and Knowledge Management . Boise : ACM , 2024 : 404 - 414 . DOI: 10.1145/3627673.3679655 http://dx.doi.org/10.1145/3627673.3679655

Zhu Xiaofei , Li Liang , Liu Weidong , et al . Multi-level sequence denoising with cross-signal contrastive learning for sequential recommendation [J ] . Neural Networks , 2024 , 179 ( 6 ): 106480 . DOI: 10.1016/j.neunet.2024.106480 http://dx.doi.org/10.1016/j.neunet.2024.106480

Huang Y H , Lo L , Xie Hongxia , et al . Future sight and tough fights: Revolutionizing sequential recommendation with FENRec [C ] // Proceedings of the 39th AAAI Conference on Artificial Intelligence . Philadelphia : AAAI , 2025 : 11826 - 11834 . DOI: 10.1609/aaai.v39i11.33287 http://dx.doi.org/10.1609/aaai.v39i11.33287

钱忠胜 , 黄恒 , 万子珑 . 融合自注意力机制的多行为图对比学习推荐方法 [J ] . 电子学报 , 2024 , 52 ( 11 ): 3684 - 3698 .

Qian Zhongsheng , Huang Heng , Wan Zilong . The multi-behavior graph contrastive learning recommendation method with self-attention mechanism [J ] . Acta Electronica Sinica , 2024 , 52 ( 11 ): 3684 - 3698 . (in Chinese)

Luo Yanchen , Li Sihang , Sui Yongduo , et al . Masked graph modeling with multi- view contrast [C ] // Proceedings of 2024 IEEE 40th International Conference on Data Engineering . Utrecht : IEEE , 2024 : 2584 - 2597 . DOI: 10.1109/icde60146.2024.00203 http://dx.doi.org/10.1109/icde60146.2024.00203

Liu Chuang , Wang Yuyao , Zhan Yibing , et al . Where to mask: Structure-guided masking for graph masked autoencoders [C ] // Proceedings of the 33rd International Joint Conference on Artificial Intelligence . Jeju : ijcai.org , 2024 : 2180 - 2188 .

Fang Taoran , Xiao Zhiqiang , Wang Chunping , et al . DropMessage: Unifying random dropping for graph neural networks [C ] // Proceedings of the 37th AAAI Conference on Artificial Intelligence . Washington : AAAI Press , 2023 : 4267 - 4275 . DOI: 10.1609/aaai.v37i4.25545 http://dx.doi.org/10.1609/aaai.v37i4.25545

Liu Meng , Gao Hongyang , Ji Suiwang . Towards deeper graph neural networks [C ] // Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . New York : ACM , 2020 : 338 - 348 . DOI: 10.1145/3394486.3403076 http://dx.doi.org/10.1145/3394486.3403076

刘昕悦 , 尹海莲 , 臧亚磊 , 等 . 基于图插值和可变形卷积网络的序列推荐 [J ] . 计算机研究与发展 , 2025 , 62 ( 10 ): 2583 - 2594 .

Liu Xinyue , Yin Hailian , Zang Yalei , et al . A graph-based interpolation sequential recommender with deformable convolutional network [J ] . Journal of Computer Research and Development , 2025 , 62 ( 10 ): 2583 - 2594 . (in Chinese)

You Di , Lee K . Context-aware diffusion-based sequential recommendation [C ] // Proceedings of 2024 IEEE International Conference on Big Data . Washington : IEEE , 2024 : 670 - 679 . DOI: 10.1109/bigdata62323.2024.10826051 http://dx.doi.org/10.1109/bigdata62323.2024.10826051

He Zhankui , Zhao Handong , Wang Zhaowen , et al . Query-aware sequential recommendation [C ] // Proceedings of the 31st ACM International Conference on Information & Knowledge Management . Atlanta : ACM , 2022 : 4019 - 4023 . DOI: 10.1145/3511808.3557677 http://dx.doi.org/10.1145/3511808.3557677

Zhang Hao , Cheng Mingyue , Liu Zhiding , et al . Towards automatic sampling of user behaviors for sequential recommender systems [C ] // Proceedings of the 34th International Joint Conference on Artificial Intelligence . Montreal : ijcai.org , 2025 : 3624 - 3632 . DOI: 10.24963/ijcai.2025/403 http://dx.doi.org/10.24963/ijcai.2025/403

Pakdaman F , Gabbouj M . Channel-wise feature decorrelation for enhanced learned image compression [J ] . IEEE Signal Processing Letters , 2024 , 31 : 1635 - 1639 . DOI: 10.1109/lsp.2024.3411524 http://dx.doi.org/10.1109/lsp.2024.3411524

Hua Tianyu , Wang Wenxiao , Xue Zihui , et al . On feature decorrelation in self-supervised learning [C ] // Proceedings of 2021 IEEE/CVF International Conference on Computer vision . Montreal : IEEE , 2021 : 9578 - 9588 . DOI: 10.1109/iccv48922.2021.00946 http://dx.doi.org/10.1109/iccv48922.2021.00946

郭向星 , 周魏 , 杨正益 , 等 . 基于自监督图卷积和注意力机制实现隐式反馈降噪的社交推荐 [J ] . 电子学报 , 2025 , 53 ( 1 ): 151 - 162 .

Guo Xiangxing , Zhou Wei , Yang Zhengyi , et al . Denoising implicit feedback with self-supervised graph convolution network and attention mechanism for social recommendation [J ] . Acta Electronica Sinica , 2025 , 53 ( 1 ): 151 - 162 . (in Chinese)

Zhu Qiuyu , Wang Hao , Zu Xuewen , et al . Multi-stage feature decorrelation constraints for improving CNN classification performance [C ] // Proceedings of 2023 China Automation Congress . Chongqing : IEEE , 2023 : 9219 - 9224 . DOI: 10.1109/cac59555.2023.10451400 http://dx.doi.org/10.1109/cac59555.2023.10451400

Hidasi B , Karatzoglou A , Baltrunas L , et al . Session-based recommendations with recurrent neural networks [PP/OL ] . V4. arXiv ( 2016-03-29 )[ 2025-06-20 ] . https://arxiv.org/abs/1511.06939 https://arxiv.org/abs/1511.06939 .

Kang Wangcheng , McAuley J . Self-attentive sequential recommendation [C ] // Proceedings of 2018 IEEE International Conference on Data Mining . Singapore : IEEE , 2018 : 197 - 206 . DOI: 10.1109/icdm.2018.00035 http://dx.doi.org/10.1109/icdm.2018.00035

Ma Jianxin , Zhou Chang , Cui Peng , et al . Learning disentangled representations for recommendation [C ] // Proceedings of the 33rd International Conference on Neural Information Processing Systems . Vancouver : Curran Associates Inc. , 2019 : 513 .

钱忠胜 , 王亚惠 , 俞情媛 , 等 . 利用伪重叠判定机制的多层循环GCN跨域推荐 [J ] . 软件学报 , 2025 , 36 ( 9 ): 4327 - 4348 .

Qian Zhongsheng , Wang Yahui , Yu Qingyuan , et al . Multi-layer recurrent GCN cross-domain recommendation with pseudo-overlap detection mechanism [J ] . Journal of Software , 2025 , 36 ( 9 ): 4327 - 4348 . (in Chinese)

Zhou Haoyi , Zhang Shanghang , Peng Jieqi , et al . Informer: Beyond efficient transformer for long sequence time-series forecasting [C ] // Proceedings of the 35th AAAI Conference on Artificial Intelligence . AAAI Press , 2021 : 11106 - 11115 . DOI: 10.1609/aaai.v35i12.17325 http://dx.doi.org/10.1609/aaai.v35i12.17325

Zhu Yongchun , Chen Jingwu , Chen Ling , et al . AdaF 2 M 2 : Comprehensive learning and responsive leveraging features in recommendation system [C ] // Proceedings of the 30th International Conference on Database Systems for Advanced Applications . Singapore : Springer , 2026 : 332 - 343 . DOI: 10.1007/978-981-95-4158-4_23 http://dx.doi.org/10.1007/978-981-95-4158-4_23

Sang Lei , Wang Yu , Zhang Yiwen . Heterogeneous graph masked contrastive learning for robust recommendation [PP/OL ] . V1. arXiv ( 2025-05-30 )[ 2025-11-11 ] . https://arXiv.org/abs/2505.24172 https://arXiv.org/abs/2505.24172 . DOI: 10.1016/j.asoc.2025.113596 http://dx.doi.org/10.1016/j.asoc.2025.113596

Xia Xue , Eksombatchai P , Pancha N , et al . TransAct: Transformer-based realtime user action model for recommendation at pinterest [C ] // Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining . Long Beach : ACM , 2023 : 5249 - 5259 . DOI: 10.1145/3580305.3599918 http://dx.doi.org/10.1145/3580305.3599918

Zhou Xiaofan , Lee K . ID and graph view contrastive learning with multi-view attention fusion for sequential recommendation [C ] // Proceedings of 2024 IEEE International Conference on Big Data . Washington : IEEE , 2024 : 690 - 699 . DOI: 10.1109/bigdata62323.2024.10825602 http://dx.doi.org/10.1109/bigdata62323.2024.10825602

Qiu Ruihong , Huang Zi , Yin Hongzhi , et al . Contrastive learning for representation degeneration problem in sequential recommendation [C ] // Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining . New York : ACM , 2022 : 813 - 823 . DOI: 10.1145/3488560.3498433 http://dx.doi.org/10.1145/3488560.3498433

Peng Furong , Liu Kang , Lu Xuan , et al . TSC: A simple two-sided constraint against over-smoothing [C ] // Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining . Barcelona : ACM , 2024 : 2376 - 2387 . DOI: 10.1145/3637528.3671954 http://dx.doi.org/10.1145/3637528.3671954

Zhu Xiaotian , Zhou Wengang , Li Houqiang . Improving deep neural network sparsity through decorrelation regularization [C ] // Proceedings of the 27th International Joint Conference on Artificial Intelligence . Stockholm : IJCAI , 2018 : 3264 - 3270 . DOI: 10.24963/ijcai.2018/453 http://dx.doi.org/10.24963/ijcai.2018/453

Zeng Yuyuan , Dai Tao , Xia Shutao . Corrdrop: Correlation based dropout for convolutional neural networks [C ] // Proceedings of 2020 IEEE International Conference on Acoustics, Speech and Signal Processing . Barcelona : IEEE , 2020 : 3742 - 3746 . DOI: 10.1109/icassp40776.2020.9053605 http://dx.doi.org/10.1109/icassp40776.2020.9053605

Jin Wei , Liu Xiaorui , Ma Yao , et al . Feature overcorrelation in deep graph neural networks: A new perspective [C ] // Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining . Washington : ACM , 2022 : 709 - 719 . DOI: 10.1145/3534678.3539445 http://dx.doi.org/10.1145/3534678.3539445

Lin Guanyu , Luo Jinwei , Li Yinfeng , et al . Iterative sparse attention for long-sequence recommendation [C ] // Proceedings of the 39th AAAI Conference on Artificial Intelligence . Philadelphia : AAAI Press , 2025 : 12147 - 12155 . DOI: 10.1609/aaai.v39i11.33323 http://dx.doi.org/10.1609/aaai.v39i11.33323

Fei Ke , Zhang Xinyue , Li Jingjing . Entire-space variational information exploitation for post-click conversion rate prediction [C ] // Proceedings of the 39th AAAI Conference on Artificial Intelligence . Philadelphia : AAAI Press , 2025 : 11654 - 11662 . DOI: 10.1609/aaai.v39i11.33268 http://dx.doi.org/10.1609/aaai.v39i11.33268

Wu Wei , Wang Chao , Shen Dazhong , et al . AFDGCF: Adaptive feature de-correlation graph collaborative filtering for recommendations [C ] // Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval . Washington : ACM , 2024 : 1242 - 1252 . DOI: 10.1145/3626772.3657724 http://dx.doi.org/10.1145/3626772.3657724

Devlin J , Chang Mingwei , Lee K , et al . BERT: Pre-training of deep bidirectional transformers for language understanding [C ] // Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies . Minneapolis : ACL , 2019 : 4171 - 4186 . DOI: 10.18653/v1/n19-1423 http://dx.doi.org/10.18653/v1/n19-1423

Tang Jiaxi , Wang Ke . Personalized top-N sequential recommendation via convolutional sequence embedding [C ] // Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining . Marina Del Rey : ACM , 2018 : 565 - 573 . DOI: 10.1145/3159652.3159656 http://dx.doi.org/10.1145/3159652.3159656

Sun Fei , Liu Jun , Wu Jian , et al . BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer [C ] // Proceedings of the 28th ACM Iinternational Conference on Information and Knowledge Management . Beijing : ACM , 2019 : 1441 - 1450 . DOI: 10.1145/3357384.3357895 http://dx.doi.org/10.1145/3357384.3357895

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Generative Image Detection Based on Diffusion Artifact Contrast Learning

Leveraging Action Description Generation and Cross-Modal Semantic Alignment for Skeleton-Based Action Recognition

Vulnerability Knowledge Graph Construction and Completion with Dual-Modality Perception

RGE-Pipeline: Fast Evaluation for LLM-Based Retrieval-Augmented Generation Systems

A Priority-Weighted and Packet Arrival Time Based Scheduling Algorithm for MP-QUIC

Related Author

ZHU Liya

SUN Yana

REN Shuai

ZHANG Liukun

JIANG Donghua

YUAN Chengsheng

CHEN Jinrui

CAO Yi

Related Institution

School of Computer Science and Engineering, Sun Yat-sen University

School of Information Engineering, Chang’an University

School of Electronics and Control Engineering, Chang’an University

Institute of Artificial Intelligence, Guangzhou University

School of Cybersecurity and Informationization, Wuxi University

⁰