面向在线连续学习的特征融合引导的梯度重加权算法
Gradient Re-Weighting Guided by Feature Fusion for Online Continual Learning
- 电子学报 2025年53卷第11期 页码：3970-3982
- 作者机构：
电子科技大学信息与通信工程学院，四川成都 611731
- 作者简介：
[ "邱奔流 男，1999年5月生，湖南常德人.电子科技大学信息与通信工程专业博士研究生.主要研究方向为连续学习、迁移学习、多模态学习、目标检测、社交网络分析.E-mail: qbenliu@163.com" ]
[ "王岚晓 女，1997年3月生，山东淄博人.电子科技大学特聘副研究员.主要研究方向为多媒体信息处理、多模态场景解析.中国电子学会会员编号：E190091313M.E-mail: lanxiaowang@foxmail.com" ]
[ "邱荷茜 女，1994年10月生，山西运城人.电子科技大学信息与通信工程学院副教授.主要研究方向为计算机视觉、多媒体智能信息处理、目标检测与识别.E-mail: hqqiu@uestc.edu.cn" ]
[ "高翔宇 男，1998年2月生，贵州贵阳人.电子科技大学信息与通信工程专业博士研究生.主要研究方向为目标检测、开放词汇识别. E-mail: xygao@std.uestc.edu.cn" ]
[ "问海涛 男，1995年4月生，江苏淮安人.电子科技大学信息与通信工程学院博士研究生.主要研究方向为计算机视觉、机器学习、连续学习、增量学习. E-mail: haitaowen@std.uestc.edu.cn" ]
[ "李宏亮 男，1970年8月生，2005年获西安交通大学博士学位，目前为电子科技大学二级教授、博士生导师，国家杰出青年科学基金获得者.主要研究方向为多媒体智能、对象检测与分割、视觉感知模型以及机器学习等.电子学会会员编号：E190008433S. E-mail: hlli@uestc.edu.cn" ]
- 基金信息：新一代人工智能国家科技重大专项(2021ZD0112001)
DOI：10.12263/DZXB.20250413
中图分类号： TP181;TP391.4收稿：2025-05-24，
录用：2025-11-20，
纸质出版：2025-11-25
- 稿件说明：
移动端阅览
邱奔流, 王岚晓, 邱荷茜, 等. 面向在线连续学习的特征融合引导的梯度重加权算法[J]. 电子学报, 2025, 53(11): 3970-3982.
QIU Ben-liu, WANG Lan-xiao, QIU He-qian, et al. Gradient Re-Weighting Guided by Feature Fusion for Online Continual Learning[J]. Acta Electronica Sinica, 2025, 53(11): 3970-3982.
邱奔流, 王岚晓, 邱荷茜, 等. 面向在线连续学习的特征融合引导的梯度重加权算法[J]. 电子学报, 2025, 53(11): 3970-3982. DOI：10.12263/DZXB.20250413
QIU Ben-liu, WANG Lan-xiao, QIU He-qian, et al. Gradient Re-Weighting Guided by Feature Fusion for Online Continual Learning[J]. Acta Electronica Sinica, 2025, 53(11): 3970-3982. DOI：10.12263/DZXB.20250413
摘要
在线连续学习（Online Continual Learning，OCL）旨在从非平稳的数据流中以仅仅读取一次数据样本的方式学习知识，因此面临着学习不充分的问题.为缓解这一问题，本文提出了一种特征融合的方法.该方法提取每张图片的一系列增强后样本的特征作为锚点特征，并通过加权求和的操作来融合这些特征以获得融合特征.融合权值由锚点特征和选定的作为枢纽特征的图片特征之间的相似性来决定.优化这一融合特征的交叉熵损失能够促进学习进程，进而在当前新任务上取得更好的表现.另外，我们提出了一致性损失来限制融合特征和枢纽特征之间的均方误差，以进一步提高模型在新任务上的表现.最后，我们理论分析了交叉熵损失关于模型参数的梯度.这一分析揭示了特征融合和梯度重加权之间的关系.我们选择了在线连续学习的三个常用基准进行了大量的实验，包括CIFAR-10、CIFAR-100和Tiny-ImageNet.相比基准方法，本文方法的平均最终准确率在CIFAR-10上提高了至多7.00%，在CIFAR-100上提高了至多8.04%，在Tiny-ImageNet上提高了至多6.33%.实验结果表明了本文方法的有效性，并且其在线连续学习能力相比已有方法取得了实质性的提升.
Abstract
Online continual learning (OCL) aims at learning a non-stationary data stream in a way of reading each data sample only once
and hence suffers from insufficient learning. To address this problem
we propose a feature fusion method in this work. Our method leverages augmented samples of an image for producing anchor features
and incorporates them to obtain a fused feature via a weighted summation operation. The weights are determined by the similarity between anchor features and a pre-designated pivotal feature of the image. Optimizing the cross-entropy loss of this fused feature can accelerate the learning process
resulting in better performance on the current task. Additionally
we propose a consistency loss that restricts the mean-square error between the fused feature and the pivotal feature
which can further improve the performance on the current task. Finally
we provide a theoretical analysis about the gradients of cross-entropy loss to model parameters. This analysis reveals the relationship between the feature fusion and the gradient re-weighting. Extensive experiments are conducted on three benchmarks under OCL settings
including CIFAR-10
CIFAR-100 and Tiny-ImageNet. Our method surpasses baselines at most 7.00%
8.04%
6.33% for average end accuracy on CIFAR-10
CIFAR-100 and Tiny-ImageNet
respectively. Experimental results demonstrate the proposed method is effective
and achieves substantial improvement over previous methods for online continual learning.
关键词
Keywords
references
MA X Z , OUYANG W L , SIMONELLI A , et al . 3D object detection from images for autonomous driving: A survey [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2024 , 46 ( 5 ): 3537 - 3556 .
李博洋 , 刘思健 , 崔明月 , 等 . 基于最小回环检测的多车协同SLAM框架 [J ] . 电子学报 , 2021 , 49 ( 11 ): 2241 - 2250 .
LI B Y , LIU S J , CUI M Y , et al . Multi-vehicle collaborative SLAM framework for minimum loop detection [J ] . Acta Electronica Sinica , 2021 , 49 ( 11 ): 2241 - 2250 . (in Chinese)
周治国 , 马文浩 . 一种多层多模态融合3D目标检测方法 [J ] . 电子学报 , 2024 , 52 ( 3 ): 696 - 708 .
ZHOU Z G , MA W H . 3D object detection based on multilayer multimodal fusion [J ] . Acta Electronica Sinica , 2024 , 52 ( 3 ): 696 - 708 . (in Chinese)
GAO C , LIU S , CHEN J Y , et al . Room-object entity prompting and reasoning for embodied referring expression [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2024 , 46 ( 2 ): 994 - 1010 .
张磊 , 鲁凯 , 高春侠 , 等 . 基于变增益自抗扰技术的机器人轨迹跟踪控制方法 [J ] . 电子学报 , 2022 , 50 ( 1 ): 89 - 97 .
ZHANG L , LU K , GAO C X , et al . Path tracking control method of robot based on time-varying gain active disturbance rejection control [J ] . Acta Electronica Sinica , 2022 , 50 ( 1 ): 89 - 97 . (in Chinese)
金紫凤 , 潘思聪 , 危辉 . 可变环境下基于位姿变换矩阵的机器人无标定手眼协调方法 [J ] . 电子学报 , 2022 , 50 ( 10 ): 2318 - 2328 .
JIN Z F , PAN S C , WEI H . Uncalibrated hand eye coordination method for robot based on pose transformation matrix in variable environment [J ] . Acta Electronica Sinica , 2022 , 50 ( 10 ): 2318 - 2328 . (in Chinese)
VAN DE VEN G M , TUYTELAARS T , TOLIAS A S . Three types of incremental learning [J ] . Nature Machine Intelligence , 2022 , 4 ( 12 ): 1185 - 1197 .
CHAUDHRY A , ROHRBACH M , ELHOSEINY M , et al . On tiny episodic memories in continual learning [EB/OL ] . ( 2019-06-04 )[ 2025-10-20 ] . https://arXiv.org/abs/1902.10486 https://arXiv.org/abs/1902.10486 .
CHAUDHRY A , RANZATO M , ROHRBACH M , et al . Efficient lifelong learning with A-GEM [EB/OL ] . ( 2019-01-09 )[ 2025-10-10 ] . https://arXiv.org/abs/1812.00420 https://arXiv.org/abs/1812.00420 .
PRABHU A , TORR P H S , DOKANIA P K . GDumb: A simple approach that questions our progress in continual learning [M ] // Computer Vision-ECCV 2020 . Cham : Springer International Publishing , 2020 : 524 - 540 .
SHIM D , MAI Z D , JEONG J , et al . Online class-incremental continual learning with adversarial shapley value [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2021 , 35 ( 11 ): 9630 - 9638 .
CACCIA L , ALJUNDI R , ASADI N , et al . New insights on reducing abrupt representation change in online continual learning [EB/OL ] . ( 2022-05-02 )[ 2025-10-10 ] . https://arXiv.org/abs/2104.05025 https://arXiv.org/abs/2104.05025 .
BUZZEGA P , BOSCHINI M , PORRELLO A , et al . Dark experience for general continual learning: A strong, simple baseline [EB/OL ] . ( 2020-10-22 )[ 2025-10-20 ] . https://arXiv.org/abs/2004.07211 https://arXiv.org/abs/2004.07211 .
MAI Z D , LI R W , KIM H , et al . Supervised contrastive replay: Revisiting the nearest class mean classifier in online class-incremental continual learning [C ] // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops . Piscataway : IEEE , 2021 : 3584 - 3594 .
WEI Y J , YE J X , HUANG Z Z , et al . Online prototype learning for online continual learning [C ] // 2023 IEEE/CVF International Conference on Computer Vision . Piscataway : IEEE , 2024 : 18718 - 18728 .
GOU J P , YU B S , MAYBANK S J , et al . Knowledge distillation: A survey [J ] . International Journal of Computer Vision , 2021 , 129 ( 6 ): 1789 - 1819 .
HOU S H , PAN X Y , LOY C C , et al . Learning a unified classifier incrementally via rebalancing [C ] // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2020 : 831 - 839 .
WU Y , CHEN Y P , WANG L J , et al . Large scale incremental learning [C ] // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2020 : 374 - 382 .
MITTAL S , GALESSO S , BROX T . Essentials for class incremental learning [C ] // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops . Piscataway : IEEE , 2021 : 3508 - 3517 .
AHN H , KWAK J , LIM S , et al . SS-IL: Separated softmax for incremental learning [C ] // 2021 IEEE/CVF International Conference on Computer Vision . Piscataway : IEEE , 2022 : 824 - 833 .
GUO Y D , LIU B , ZHAO D Y . Online continual learning through mutual information maximization [C ] // International Conference on Machine Learning . Cambridge : PMLR , 2022 : 8109 - 8126 .
CHA H , LEE J , SHIN J . Co2 L: Contrastive continual learning[C ] // 2021 IEEE/CVF International Conference on Computer Vision . Piscataway : IEEE , 2022 : 9496 - 9505 .
ZHU F , CHENG Z , ZHANG X Y , et al . Class-incremental learning via dual augmentation [C ] // Proceedings of the 35th International Conference on Neural Information Processing Systems . New York : ACM , 2021 : 14306 - 14318 .
GU J Y , WANG K , JIANG W , et al . Summarizing stream data for memory-constrained online continual learning [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2024 , 38 ( 11 ): 12217 - 12225 .
YU R N , LIU S H , WANG X C . Dataset distillation: A comprehensive review [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2024 , 46 ( 1 ): 150 - 170 .
DE LANGE M , ALJUNDI R , MASANA M , et al . A continual learning survey: Defying forgetting in classification tasks [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022 , 44 ( 7 ): 3366 - 3385 .
ALJUNDI R , LIN M , GOUJAUD B , et al . Gradient based sample selection for online continual learning [C ] //Neural Information Processing Systems. Curran Associates Inc .: Red Hook , 2019 : 11817 - 11826 .
ALJUNDI R , BELILOVSKY E , TUYTELAARS T , et al . Online continual learning with maximal interfered retrieval [EB/OL ] . ( 2019-10-29 )[ 2025-10-10 ] . https://arXiv.org/abs/1908.04742 https://arXiv.org/abs/1908.04742 .
GUO Y D , LIU B , ZHAO D Y . Dealing with cross-task class discrimination in online continual learning [C ] // 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2023 : 11878 - 11887 .
WU Y C , WANG H , ZHAO P L , et al . Mitigating catastrophic forgetting in online continual learning by modeling previous task interrelations via pareto optimization [C ] // Proceedings of the 41st International Conference on Machine Learning . Cambridge : PMLR , 2024 : 8929 .
HE Y H , CHEN Y J , JIN Y H , et al . DYSON: Dynamic feature space self-organization for online task-free class incremental learning [C ] // 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2024 : 23741 - 23751 .
SEO M , KOH H , JEUNG W , et al . Learning equi-angular representations for online continual learning [C ] // 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2024 : 23933 - 23942 .
PAPYAN V , HAN X Y , DONOHO D L . Prevalence of neural collapse during the terminal phase of deep learning training [J ] . PNAS 2020 , 117 ( 40 ): 24652 - 24663 .
KRIZHEVSKY A . Learning multiple layers of features from tiny images [EB/OL ] . ( 2009-04-08 )[ 2025-10-10 ] . https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf .
LE Y , YANG X . Tiny ImageNet visual recognition challenge [EB/OL ] . ( 2015 )[ 2025-10-10 ] . https://cs231n.stanford.edu/reports/2015/pdfs/yle_project.pdf https://cs231n.stanford.edu/reports/2015/pdfs/yle_project.pdf .
HE K M , ZHANG X Y , REN S Q , et al . Deep residual learning for image recognition [C ] // 2016 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2016 : 770 - 778 .
MAATEN L , HINTON G . Visualizing data using t-SNE [J ] . Journal of Machine Learning Research , 2008 , 9 : 2579 - 2605 .
0
浏览量
2
下载量
0
CSCD
- 网络PDF下载
- Meta-XML下载
- BibTeX下载
- Endnote下载
- RefMan 下载
- NoteExpress下载
- NoteFirst下载
关联资源
相关文章
相关作者
相关机构
- 技术支持由北京北大方正电子有限公司提供 京ICP备09064830号-19
京公网安备11010802024621 - 本系统建议在Chrome、 IE9+ 以上版本浏览器阅读本站内容，360浏览器请切换至极速模式
- Cookies帮助我们提供服务并提供个性化体验。使用本网站，即表示您同意我们使用Cookies