Tr-SLDA:一种面向交叉领域的迁移主题模型

doi:10.12263/DZXB.20200210

电子学报 ›› 2021, Vol. 49 ›› Issue (3): 605-613.DOI: 10.12263/DZXB.20200210

Tr-SLDA:一种面向交叉领域的迁移主题模型

唐焕玲^1,2,4, 郑涵², 刘艳红¹, 马思源², 窦全胜^1,3,4, 鲁明羽⁵

1. 山东工商学院计算机科学与技术学院, 山东烟台 264005;
2. 山东工商学院信息与电子工程学院, 山东烟台 264005;
3. 山东省高等学校协同创新中心:未来智能计算, 山东烟台 264005;
4. 山东省高校智能信息处理重点实验室(山东工商学院), 山东烟台 264005;
5. 大连海事大学信息科学技术学院, 辽宁大连 116026

收稿日期:2020-02-26 修回日期:2020-04-21 出版日期:2021-03-25
- 作者简介:
- 唐焕玲 女,教授,博士,硕士生导师,1970年生于山东龙口.2004年于清华大学获得工学硕士学位,2009年于大连海事大学大学获得工学博士学位.从事机器学习、人工智能、数据挖掘等方向的理论及应用研究.E-mail:thl01@163.com;郑涵男,1997年生于河南商丘.山东工商学院硕士研究生,主要研究方向为机器学习、人工智能、数据挖掘.E-mail:zhenghan0503@163.com;刘艳红 女,1995年生于山东烟台.山东工商学院硕士研究生,研究方向为机器学习与数据挖掘.E-mail:2669349709@qq.com;马思源 女,1994年生于河南周口.山东工商学院硕士研究生,主要研究方向为计算机视觉.E-mail:masiyuan423@163.com;窦全胜 男,教授,博士,硕士生导师.1971年生于黑龙江大庆.2001年、2005年于吉林大学分别获得理学硕士学位、工学博士学位.从事人工智能、机器学习、演化计算等方向的理论及应用研究.E-mail:li_dou@163.com;鲁明羽 男,教授,博士生导师.1963年生于黑龙江鸡西.1988年、2002年于清华大学分别获得工学硕士和工学博士学位,从事机器学习、人工智能、数据挖掘等方向的理论及应用研究.E-mail:lumingyu@dlmu.edu.cn
- 基金资助:
- 国家自然科学基金 (No.61976124，No.61976125，No.61773244，No.61772319，No.61873177，No.61972235）; 山东省高校科研计划 (No.J18KA385），烟台市重点研发计划 (No.2017ZH065，No.2019XDHZ081）

Tr-SLDA: A Transfer Topic Model for Cross-Domains

TANG Huan-ling^1,2,4, ZHENG Han², LIU Yan-hong¹, MA Si-yuan², DOU Quan-sheng^1,3,4, LU Ming-yu⁵

1. College of Computer Science and Technology, Shandong Technology and Business University, Yantai, Shandong 264005, China;
2. College of Information and Electronic Engineering, Shandong Technology and Business University, Yantai, Shandong 264005, China;
3. Co-innovation Center of Shandong Colleges and Universities:Future Intelligent Computing, Yantai, Shandong 264005, China;
4. Key Laboratory of Intelligent Information Processing in Universities of Shandong(Shandong Technology and Business University), Shandong Yantai, 264005, China;
5. Information Science and Technology College, Dalian Maritime University, Dalian Liaoning 116026, China

Received:2020-02-26 Revised:2020-04-21 Online:2021-03-25 Published:2021-03-25

摘要/Abstract

摘要： 当目标领域缺少足够多的标注数据时，迁移学习利用相关源领域的标注数据，辅助提升目标域的学习性能，但是目标域与源域的数据通常不满足独立同分布，容易导致"负迁移"问题.本文在有监督主题模型（Supervised LDA，SLDA）的基础上，融合迁移学习方法提出一种共享主题知识的迁移主题模型（Transfer SLDA，Tr-SLDA），提出Tr-SLDA-Gibbs主题采样新方法，在类别标签的约束下对不同领域文档中的词采取不同的采样策略，且无需指定主题个数.辅助源域与目标域共享潜在主题空间，Tr-SLDA通过发现潜在共享主题与不同领域类别之间的语义关联从源域迁移知识，可以有效解决"负迁移"问题.基于Tr-SLDA迁移主题模型提出Tr-SLDA-TC （Tr-SLDA Text Categorization）文本分类方法.对比实验表明，该方法可有效利用源域知识来提高目标领域的分类性能.

关键词: 文本分类, 主题模型, 吉布斯采样, 迁移学习, 负迁移

Abstract: With enough labeled data lacking in the target domain,it works well for transfer learning to use the labeled data of the related source domain and help improve the learning performance of the target domain.However,the data of these two domains usually do not satisfy the independently identically distribution,which easily leads to the problem of "negative transfer".Tr-SLDA(Transfer SLDA),a novel transfer topic model based on supervised topic model (Supervised LDA,SLDA) is proposed,which shares topic knowledge by integrating transfer learning.A new Tr-SLDA-Gibbs sampling method is proposed,under the constraints of category labels,different sampling strategies are adopted for words in the documents of different domains without specifying the number of topics.The source domain and target domain share the potential topic space,Tr-SLDA can effectively solve the problem of "negative transfer" by discovering the semantic correlation between the potential shared topics and categories of different domains.The Tr-SLDA-TC (Tr-SLDA-Text Categorization) text classification method is proposed based on the Tr-SLDA model.The comprehensive experiments show that the proposed method can effectively improve the performance of the classification by utilizing the knowledge from the source domain.

Key words: text categorization, topic model, Gibbs sampling, transfer learning, negative transfer

中图分类号:

TP181

唐焕玲, 郑涵, 刘艳红, 马思源, 窦全胜, 鲁明羽. Tr-SLDA:一种面向交叉领域的迁移主题模型[J]. 电子学报, 2021, 49(3): 605-613.

TANG Huan-ling, ZHENG Han, LIU Yan-hong, MA Si-yuan, DOU Quan-sheng, LU Ming-yu. Tr-SLDA: A Transfer Topic Model for Cross-Domains[J]. Acta Electronica Sinica, 2021, 49(3): 605-613.

参考文献

[1] Dai W,Jin O,Xue G R,et al.Eigentransfer:a unified framework for transfer learning[A].Proceedings of the 26th Annual International Conference on Machine Learning[C].New York,NY:Association for Computing Machinery,2009.193-200.
[2] Pan W,Yang Q.Transfer learning in heterogeneous collaborative filtering domains[J].Artificial Intelligence,2013,197(4):39-55.
[3] Long M,Wang J,Ding G,Shen D,Yang Q.Transfer learning with graph co-regularization[J].IEEE Transactions on Knowledge & Data Engineering,2014,26(7):1805-1818.
[4] Zhang X,Yang Q.Transfer hierarchical attention network for generative dialog system[J].International Journal of Automation and Computing,2019,16(6):720-736.
[5] Pan W,Yang Q,Duan Y,et al.Transfer learning for semi-supervised collaborative recommendation[J].The ACM Transactions on Interactive Intelligent Systems,2016,6(2):1-21.
[6] 吴斌,吉佳,孟琳,等.基于迁移学习的唐诗宋词情感分析[J].电子学报,2016,44(11):2780-2787. Wu B,Ji J,Meng L,et al.Transfer learning based sentiment analysis for poetry of the tang dynasty and song dynasty[J].Acta Electronica Sinica,2016,44(11):2780-2787.(in Chinese)
[7] Zhao G L,Xiang Y L,Li M Q,et al.A cross-region transfer learning method for classification of community service cases with small datasets[J].Knowledge-Based Systems,2020,193:105390.
[8] Blei D M,Ng A,Jordan M.Latent dirichlet allocation[J].Journal of Machine Learning Research,2003,3(4-5):993-1022.
[9] Wood J,Tan P,Wang W,et al.Source-LDA:enhancing probabilistic topic models using prior knowledge sources[A].International Conference on Data Engineering[C].San Diego,California:IEEE,2017.411-422.
[10] Zhang H,Zhong G.Improving short text classification by learning vector representations of both words and hidden topics[J].Knowledge-Based Systems,2016,102(6):76-86.
[11] Kandemir M,Kekeç T,Yeniterzi R.Supervising topic models with gaussian processes[J].Pattern Recognition,2018,77(5):226-236.
[12] 石敏,刘建勋,周栋,曹步清,等.基于多重关系主题模型的Web服务聚类方法[J].计算机学报,2019,42(04):820-836. Shi M,Liu J X,Zhou D,et al.Multi-relational topic model-based approach for web services clustering[J].Chinese Journal of Computers,2019,42(04):820-836.(in Chinese)
[13] 张雄,陈福才,黄瑞阳.基于双词主题模型的半监督实体消歧方法研究[J].电子学报,2018,46(3):607-613. Zhang X,Chen F C,Huang R Y.Semi-supervised entity disambiguation method research based on bi term topic model[J].Acta Electronica Sinica,2018,46(3):607-613.(in Chinese).
[14] Mei Q,Ling X,Wondra M,et al.Topic sentiment mixture:modeling facets and opinions in weblogs[A].Proceedings of the 16th International World Wide Web Conference (WWW'07).Association for Computing Machinery[C].New York,NY,USA,2007.171-180.
[15] Niu Z,Hua G,Wang L,et al.Knowledge based topic model for unsupervised object discovery and localization[J].IEEE Transactions on Image Processing,2018,27(1):50-63.
[16] Chen C,Zare A,Trinh H N,et al.Partial membership latent dirichlet allocation for soft image segmentation[J].IEEE Transactions on Image Processing,2017,26(12):5590-5602.
[17] 刘杰,杜军平.基于潜在主题融合的跨媒体图像语义标注[J].电子学报,2014,42(05):987-991. Liu J,Du J P.Latent topic fusion-based cross-media image semantic annotation[J].Acta Electronica Sinica,2014,42(05):987-991.(in Chinese)
[18] 唐焕玲,窦全胜,于立萍,宋英杰,鲁明羽.有监督主题模型的SLDA-TC文本分类新方法[J].电子学报,2019,47(06):1300-1308. Tang H L,Dou Q S,et al.SLDA-TC:a novel text categorization approach based on supervised topic model[J].Acta Electronica Sinica,2019,47(06):1300-1308.(in Chinese)
[19] 李永忠,郑滔.基于标签的半监督HDP文本分类主题模型[J].模式识别与人工智能,2017,30(12):1138-1148. LI Yongzhong,ZHENG Tao.Semi-supervised labeled hierarchical dirichlet process topic model for document categorization[J].Pattern Recognition and Artificial Intelligence,2017,30(12):1138-1148.(in Chinese)
[20] Pan S J,Yang Q.A survey on transfer learning[J].IEEE Transactions on Knowledge and Data Engineering,2010,22(10):1345-1359.

Tr-SLDA:一种面向交叉领域的迁移主题模型

Tr-SLDA: A Transfer Topic Model for Cross-Domains

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	邵志文, 周勇, 谭鑫, 马利庄, 刘兵, 姚睿. 基于深度学习的表情动作单元识别综述[J]. 电子学报, 2022, 50(8): 2003-2017.
[2]	李维刚, 甘平, 谢璐, 李松涛. 基于样本对元学习的小样本图像分类方法[J]. 电子学报, 2022, 50(2): 295-304.
[3]	陈文俊, 杨春玲. 图像压缩感知的特征域优化及自注意力增强神经网络重构算法[J]. 电子学报, 2022, 50(11): 2629-2637.
[4]	张小恒, 张馨月, 李勇明, 王品, 刘玉川. 面向帕金森病语音诊断的非监督两步式卷积稀疏迁移学习算法[J]. 电子学报, 2022, 50(1): 177-184.
[5]	李勇斌, 王彬, 邵高平, 邵帅. 一种基于RCGAN的水声通信信号降噪方法[J]. 电子学报, 2022, 50(1): 54-62.
[6]	张昱, 刘开峰, 张全新, 王艳歌, 高凯龙. 基于组合-卷积神经网络的中文新闻文本分类[J]. 电子学报, 2021, 49(6): 1059-1067.
[7]	李大鹏, 陈剑, 王晨, 闻英友, 赵大哲. 基于模板张量分解和双向LSTM的司法案件罪名认定[J]. 电子学报, 2021, 49(4): 760-767.
[8]	吴绿, 张馨月, 唐茉, 王梓, 王永安. Focus+Context语义表征的场景图像分割[J]. 电子学报, 2021, 49(3): 596-604.
[9]	张锦, 李阳, 任传伦, 黄炼, 王帅辉, 段晔鑫, 潘志松, 谢钧. 基于帧间高级特征差分的跨场景视频前景分割算法[J]. 电子学报, 2021, 49(10): 2032-2040.
[10]	王格格, 郭涛, 余游, 苏菡. 基于生成对抗网络的无监督域适应分类模型[J]. 电子学报, 2020, 48(6): 1190-1197.
[11]	张红斌, 蒋子良, 熊其鹏, 武晋鹏, 邬任重, 袁天, 姬东鸿. 基于改进的有效区域基因选择与跨模态语义挖掘的图像属性标注[J]. 电子学报, 2020, 48(4): 790-799.
[12]	赵鹏, 王美玉, 纪霞, 刘慧婷. 基于张量表示的域适配的迁移学习中特征表示方法[J]. 电子学报, 2020, 48(2): 359-368.
[13]	权宇, 李志欣, 张灿龙, 马慧芳. 融合深度扩张网络和轻量化网络的目标检测模型[J]. 电子学报, 2020, 48(2): 390-397.
[14]	吴玉佳, 李晶, 宋成芳, 常军. 基于高效用神经网络的文本分类方法[J]. 电子学报, 2020, 48(2): 279-284.
[15]	脱婷, 马慧芳, 李志欣, 赵卫中. 熵权约束稀疏表示的短文本分类算法[J]. 电子学报, 2020, 48(11): 2131-2137.