Heterogeneous Online Multi-Source Transfer Learning with Cross-Domain Structure Preserving Projection

JIANG Xiao-ling; WU Ying-bo; CHEN Meng; QU Xiang-mou

doi:10.12263/DZXB.20210935

您当前的位置：

首页 >

文章列表页 >

Heterogeneous Online Multi-Source Transfer Learning with Cross-Domain Structure Preserving Projection

PAPERS | 更新时间：2025-12-08

- Heterogeneous Online Multi-Source Transfer Learning with Cross-Domain Structure Preserving Projection
- ACTA ELECTRONICA SINICA Vol. 51, Issue 8, Pages: 1983-1994(2023)
- 作者机构：
  
  重庆大学大数据与软件学院，重庆 400000
- 作者简介：
- 基金信息：
  
  Central University Basic Business Fee Project(2020CDCGRJ50)
- DOI：10.12263/DZXB.20210935
  CLC： TP391;
- Received：16 July 2021，
  
  Revised：2021-12-20，
  
  Published：25 August 2023
- 稿件说明：
移动端阅览
蒋晓玲,吴映波,陈蒙等.基于跨域结构保持投影的异构在线多源迁移学习方法[J].电子学报,2023,51(08):1983-1994.

JIANG Xiao-ling,WU Ying-bo,CHEN Meng,et al.Heterogeneous Online Multi-Source Transfer Learning with Cross-Domain Structure Preserving Projection[J].ACTA ELECTRONICA SINICA,2023,51(08):1983-1994.
蒋晓玲,吴映波,陈蒙等.基于跨域结构保持投影的异构在线多源迁移学习方法[J].电子学报,2023,51(08):1983-1994. DOI： 10.12263/DZXB.20210935.

JIANG Xiao-ling,WU Ying-bo,CHEN Meng,et al.Heterogeneous Online Multi-Source Transfer Learning with Cross-Domain Structure Preserving Projection[J].ACTA ELECTRONICA SINICA,2023,51(08):1983-1994. DOI： 10.12263/DZXB.20210935.

摘要

异构在线迁移学习使用异构源域的离线数据弥补目标域在线学习数据不足，从而提高目标域在线学习性能.现有方法通常假定源域是目标域特征空间子集或依赖特定的辅助数据.本文提出一种基于跨域结构保持投影的异构在线多源迁移学习方法.通过跨域结构保持投影，同时将每个源域与目标域的特征空间映射到公共子空间，并基于公共子空间中的跨域离线混合数据和目标域在线数据分别进行离线学习和在线学习，提出采用一种双层差异导向对冲集成策略，实现源域离线学习模型与目标域在线学习模型的两层集成融合和在线演化更新.基于本文方法设计实现了一种异构在线多源迁移多分类算法，且理论分析了该算法的分类错误上界.实验结果表明，本文方法能有效实现异构在线多源迁移学习并降低目标域在线多分类错误率，且优于同类的在线多源迁移学习方法.

Abstract

Heterogeneous online transfer learning aims to improve the online learning performance on target domain by making use of offline labeled instances from heterogeneous source domains to make up for the lack of online labeled date from target domain

where the feature space of source and target domains are different. Most existing approaches usually assume that the feature space of a source domain is a subset of that of the target domain

or rely on other auxiliary data. This paper proposes a new method called heterogeneous online multi-source transfer learning with cross-domain structure preserving projection. First

the original features of the data from every source and target domains are projected into a common subspace by the cross domain structure preserving projection algorithm

then we leverage the cross-domain offline hybrid data in the common subspace and online data from target domain to perform offline learning and online learning respectively. Furthermore

we propose the double-level mistake-driven hedge ensemble strategies to combine source and target learners by two-layer ensemble method and keep updating the combination method.Finally

we design and implement a heterogeneous online transfer learning algorithm for multi-class classification

and analyze the mistake bounds of the proposed algorithm theoretically. Experiment results show that our approach can reduce the mistake rate of online learning method on target domain

and outperforms the comparable online multi-source transfer learning approaches.

关键词

Keywords

references

YAN Y , WU Q , TAN M , et al . Online heterogeneous transfer by hedge ensemble of offline and online decisions [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2018 , 29 ( 7 ): 3252 - 3263 .

ZHAO P L , HOI S C H . OTL: A framework of online transfer learning [C ] // Proceedings of the 27th International Conference on International Conference on Machine Learning . New York : ACM , 2010 : 1231 - 1238 .

ZHAO P L , HOI S C H , WANG J L , et al . Online transfer learning [J ] . Artificial Intelligence , 2014 , 216 : 76 - 102 .

WU Q Y , WU H R , ZHOU X M , et al . Online transfer learning with multiple homogeneous or heterogeneous sources [J ] . IEEE Transactions on Knowledge and Data Engineering , 2017 , 29 ( 7 ): 1494 - 1507 .

CHEN Q , DU Y T , XU M , et al . HetEOTL: an algorithm for heterogeneous online transfer learning [C ] // 2018 IEEE 30th International Conference on Tools with Artificial Intelligence (ICTAI) . Piscataway : IEEE , 2018 : 350 - 357 .

WU H R , YAN Y G , YE Y Z , et al . Online Heterogeneous Transfer Learning by Knowledge Transaction [J ] . ACM Transaction on Intelligent Systems and Technology , 2019 , 10 ( 3 ): 26, 1 - 19 .

LUO Y , LIU T L , WEN Y G , et al . Online heterogeneous transfer metric learning [C ] // Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence . California : International Joint Conferences on Artificial Intelligence Organization , 2018 : 2525 - 2531 .

WANG Q , BRECKON T P . Cross-domain structure preserving projection for heterogeneous domain adaptation [EB/OL ] . ( 2020-04-26 )[ 2021-07-16 ] . https://arxiv.org/abs/2004.12427 https://arxiv.org/abs/2004.12427 .

CRAMMER K , DEKEL O , KESHET J , et al . Online passive-aggressive algorithms [J ] . Journal of Machine Learning Research , 2006 , 7 : 551 - 585 .

YOAV , FREUND , et al . A decision-theoretic generalization of on-line learning and an application to boosting [J ] . Journal of Computer and System Sciences , 1997 , 55 ( 1 ): 119 - 139 .

GONG B Q , SHI Y , SHA F , et al . Geodesic flow kernel for unsupervised domain adaptation [C ] // 2012 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2012 : 2066 - 2073 .

VENKATESWARA H , EUSEBIO J , CHAKRABORTY S , et al . Deep hashing network for unsupervised domain adaptation [C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2017 : 5385 - 5394 .

SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition [EB/OL ] . ( 2014-09-04 )[ 2021-07-16 ] . https://arxiv.org/abs/1409.1556 https://arxiv.org/abs/1409.1556 .

HE K M , ZHANG X Y , REN S Q , et al . Deep residual learning for image recognition [C ] // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2016 : 770 - 778 .

CHUA T S , TANG J H , HONG R C , et al . NUS-WIDE: A real-world web image database from National University of Singapore [C ] // Proceedings of the ACM International Conference on Image and Video Retrieval . New York : ACM , 2009 : 1 - 9 .

DENG J , DONG W , SOCHER R , et al . ImageNet: A large-scale hierarchical image database [C ] // 2009 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2009 : 248 - 255 .

KANG Z , YANG B , YANG S , et al . Online transfer learning with multiple source domains for multi-class classification [J ] . Knowledge-Based Systems , 2020 , 190 : 105149 .

WU Q Y , ZHOU X M , YAN Y G , et al . Online transfer learning by leveraging multiple source domains [J ] . Knowledge and Information Systems , 2017 , 52 ( 3 ): 687 - 707 .

季鼎承 , 蒋亦樟 , 王士同 . 基于域与样例平衡的多源迁移学习方法 [J ] . 电子学报 , 2019 , 47 ( 3 ): 692 - 699 .

JI D C , JIANG Y Z , WANG S T . Multi-source transfer learning method by balancing both the domains and instances [J ] . Acta Electronica Sinica , 2019 , 47 ( 3 ): 692 - 699 . (in Chinese)

赵鹏 , 王美玉 , 纪霞 , 等 . 基于张量表示的域适配的迁移学习中特征表示方法 [J ] . 电子学报 , 2020 , 48 ( 2 ): 359 - 368 .

ZHAO P , WANG M Y , JI X , et al . A novel feature representation based on tensor and domain adaption for transfer learning [J ] . Acta Electronica Sinica , 2020 , 48 ( 2 ): 359 - 368 . (in Chinese)

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Online Classification Algorithm for Concept Drift and Class Imbalance Data Stream

Semi-Supervised Video Object Segmentation Based on Foreground Perception Visual Attention

Gradient Re-Weighting Guided by Feature Fusion for Online Continual Learning

Concept Evolution Detection Method for Open Feature Space

Related Author

WU Ding-ming

LU Ke-zhong

CHEN Chao-fan

CAI Huan

ZHAO Yu

YAN Shao-xing

FU Li-hua

JIANG Han-xu

Related Institution

College of Computer Science and Software Engineering of Shenzhen University

School of Computer Science and Engineering, Beihang University

Faculty of Information Technology, Beijing University of Technology

School of Information and Communication Engineering, University of Electronic Science and Technology of China

Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education, Shanxi University

⁰