基于联邦学习的主动半监督短文本分类方法

孔德焱; 冀振燕; 杨燕燕; 刘洋; 刘吉强

doi:10.12263/DZXB.20230703

您当前的位置：

首页 >

文章列表页 >

基于联邦学习的主动半监督短文本分类方法

学术论文 | 更新时间：2026-05-07

- 基于联邦学习的主动半监督短文本分类方法
- An Active Semi-Supervised Short Text Classification Method Based on Federated Learning
- 电子学报 2024年52卷第10期页码：3517-3526
- 作者机构：
  
  1.北京交通大学软件学院，北京 100044
  2.北京交通大学网络空间安全学院智能交通数据安全与隐私保护北京市重点实验室，北京 100044
- 作者简介：
  
  [ "孔德焱男，1999年生于安徽省阜阳市.现为北京交通大学软件学院硕士研究生.主要研究方向为自然语言处理、图神经网络、弱监督学习.E-mail: 1402039615@qq.com" ]
  [ "冀振燕女，1972年生于河南省许昌市.现为北京交通大学软件学院教授.主要研究方向为人工智能与大数据、智能交通领域软件工程.中国电子学会会员编号：E190022451S.E-mail: zhyji@bjtu.edu.cn" ]
  [ "杨燕燕女，1986年3月出生于河南省郑州市.现为北京交通大学软件学院副教授.主要研究方向为机器学习、不确定性人工智能.E-mail: yangyy@bjtu.edu.cn" ]
  [ "刘洋男，2000年5月生于辽宁省沈阳市.现为北京交通大学软件学院硕士研究生.主要研究方向为自然语言处理.E-email: liuyang052630@163.com" ]
  [ "刘吉强男，1973年3月生于山东省海阳市.现为北京交通大学软件学院教授.主要研究方向为隐私保护、物联网安全、区块链、可信计算等.E-email: jqliu@bjtu.edu.cn" ]
- 基金信息：
  
  国家自然科学基金(52175493;51935002)
- DOI：10.12263/DZXB.20230703
  中图分类号： TP183;
- 收稿：2023-07-26，
  
  修回：2024-03-04，
  
  纸质出版：2024-10-25
- 稿件说明：
移动端阅览
孔德焱, 冀振燕, 杨燕燕, 等. 基于联邦学习的主动半监督短文本分类方法[J]. 电子学报, 2024, 52(10): 3517-3526.

KONG De-yan, JI Zhen-yan, YANG Yan-yan, et al. An Active Semi-Supervised Short Text Classification Method Based on Federated Learning[J]. Acta Electronica Sinica, 2024, 52(10): 3517-3526.
孔德焱, 冀振燕, 杨燕燕, 等. 基于联邦学习的主动半监督短文本分类方法[J]. 电子学报, 2024, 52(10): 3517-3526. DOI：10.12263/DZXB.20230703

KONG De-yan, JI Zhen-yan, YANG Yan-yan, et al. An Active Semi-Supervised Short Text Classification Method Based on Federated Learning[J]. Acta Electronica Sinica, 2024, 52(10): 3517-3526. DOI：10.12263/DZXB.20230703

摘要

短文本分类应用广泛，是当前的研究热点，但受到短文本标注数据稀缺和数据隐私保护不便集中训练的影响，分类效果不佳.针对上述问题，我们提出了基于联邦学习的主动半监督异质图注意力网络模型（Active Semi-Supervised Learning empowered Heterogeneous Graph ATtention network model based on Federated learning，Fed-ASSL-HGAT），通过设计新颖的主动半监督学习（Active Semi-Supervised Learning，ASSL）框架生成高质量标注样本赋能异质图注意力网络（Heterogeneous Graph ATttention network model，HGAT），引入联邦学习对部署在不同节点的模型进行联合训练以满足数据隐私保护需求.所提出的ASSL框架通过将主动学习的多类别标注转化成二元类别标注，可大大降低标注难度；设计基于信息增益的选择策略筛选软、硬标签，以防止信息损失；通过半监督学习选择高准确率、高稳定性的正负样本打伪标签以确保标注质量.实验结果表明，所提出的ASSL-HGAT（S）在AGNews、Snippets、TagMyNews数据集上相比HGAT基线模型

值分别提升2.45%、8.11%、7.46%.融合联邦学习所进一步提出的Fed-ASSL-HGAT模型可在不泄漏隐私数据的情况下满足性能要求.

Abstract

Short-text classification is broadly used and is a current hot research spot. However

the performance of short-text classification is hampered by the sca1rcity of annotated data for short texts and the challenges of centralized training for private data. To address these issues

we propose Fed-ASSL-HGAT (Active Semi-Supervised Heterogeneous Graph ATtention network model based on Federated learning)

an active semi-supervised heterogeneous graph attention network model based on federated learning. This model utilizes the innovative active semi-supervised learning (ASSL) framework to generate high-quality labeled samples for empowering the heterogeneous graph attention network (HGAT) model. Additionally

federated learning is introduced to facilitate the joint training of the models deployed on different nodes

thereby satisfying the requirements of data privacy protection. The proposed ASSL framework significantly reduces the annotation difficulty by transforming the multi-class annotation task into a binary classification task. To mitigate information loss

we employ a selection strategy based on information gain to filter soft and hard labels. Semi-supervised learning is employed to select positive and negative samples with high accuracy and stability for pseudo-labeling

thereby ensuring the labeling quality. Experimental results demonstrate that the proposed ASSL-HGAT (Active Semi-supervised Learning Empowered Heterogeneous Graph Attention Network) model achieves improvements of 2.45%

8.11%

and 7.46% in

scores comparing with the HGAT baseline model on the AGNews

Snippets

and TagMyNews datasets

respectively. By incorporating the federated learning

the Fed-ASSL-HGAT model can meet the performance requirements without scarifying data privacy.

关键词

Keywords

references

张昱 , 刘开峰 , 张全新 , 等 . 基于组合-卷积神经网络的中文新闻文本分类 [J ] . 电子学报 , 2021 , 49 ( 6 ): 1059 - 1067 .

ZHANG Y , LIU K F , ZHANG Q X , et al . A combined-convolutional neural network for Chinese news text classification [J ] . Acta Electronica Sinica , 2021 , 49 ( 6 ): 1059 - 1067 . (in Chinese)

李雪莹 , 王田路 , 梁鹏 , 等 . 基于系统模型的用户评论中非功能需求的自动分类 [J ] . 电子学报 , 2022 , 50 ( 9 ): 2079 - 2089 .

LI X Y , WANG T L , LIANG P , et al . Automatic classification of non-functional requirements in App user reviews based on system model [J ] . Acta Electronica Sinica , 2022 , 50 ( 9 ): 2079 - 2089 . (in Chinese)

YANG T C , HU L M , SHI C , et al . HGAT: Heterogeneous graph attention networks for semi-supervised short text classification [J ] . ACM Transactions on Information Systems , 39 ( 3 ): 32 .

MCMAHAN H B , MOORE E , RAMAGE D , et al . Communication-efficient learning of deep networks from decentralized data [EB/OL ] . ( 2016-02-17 )[ 2023-07-01 ] . http://arxiv.org/abs/1602.05629 http://arxiv.org/abs/1602.05629 .

ISCEN A , TOLIAS G , AVRITHIS Y , et al . Label propagation for deep semi-supervised learning [C ] // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2019 : 5065 - 5074 .

HAASE-SCHÜTZ C , STAL R , HERTLEIN H , et al . Iterative label improvement: Robust training by confidence based filtering and dataset partitioning [C ] // 2020 25th International Conference on Pattern Recognition (ICPR) . Piscataway : IEEE , 2021 : 9483 - 9490 .

BERTHELOT D , CARLINI N , Goodfellow I , et al . Mixmatch: A holistic approach to semi-supervised learning‍ [C ] // Proceedings of the 33rd International Conference on Neural Information Processing Systems . New York : Curran Associates Inc. , 2019 : 5049 - 5059 .

ZHOU T Y , WANG S J , BILMES J A . Time-consistent self-supervision for semi-supervised learning [C ] // Proceedings of the 37th International Conference on Machine Learning . New York : ACM , 2020 : 11523 - 11533 .

RIZVE M N , DUARTE K , RAWAT Y S , et al . In defense of pseudo-labeling: An uncertainty-aware pseudo-label selection framework for semi-supervised learning [EB/OL ] . ( 2021-01-15 )[ 2023-07-01 ] . http://arxiv.org/abs/2101.06329 http://arxiv.org/abs/2101.06329 .

李延超 , 肖甫 , 陈志 , 等 . 自适应主动半监督学习方法‍ [J ] . 软件学报 , 2020 , 31 ( 12 ): 3808 - 3822 .

LI Y C , XIAO F , CHEN Z , et al . Adaptive active learning for semi-supervised learning [J ] . Journal of Software , 2020 , 31 ( 12 ): 3808 - 3822 . (in Chinese)

ZHANG W Q , ZHU L , HALLINAN J , et al . BoostMIS: Boosting medical image semi-supervised learning with adaptive pseudo labeling and informative active annotation [C ] // 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2022 : 20634 - 20644 .

ZHANG W T , WANG Y X , YOU Z B , et al . Information gain propagation: A new way to graph active learning with soft labels [EB/OL ] . ( 2022-03-02 )[ 2023-07-01 ] . http://arxiv.org/abs/2203.01093 http://arxiv.org/abs/2203.01093 .

KONEČNÝ J , MCMAHAN H B , RAMAGE D , et al . Federated optimization: Distributed machine learning for on-device intelligence [EB/OL ] . ( 2016-10-08 )[ 2023-07-01 ] . http://arxiv.org/abs/1610.02527 http://arxiv.org/abs/1610.02527 .

HSU T M H , QI H , BROWN M . Federated visual classification with real-world data distribution [C ] // European Conference on Computer Vision . Cham : Springer , 2020 : 76 - 92 .

JEONG W , YOON J , YANG E , et al . Federated semi-supervised learning with inter-client consistency & disjoint learning [EB/OL ] . ( 2020-06-22 )[ 2023-07-01 ] . http://arxiv.‍org/abs/2006.12097 http://arxiv.‍org/abs/2006.12097 .

AHN J H , KIM K , KOH J , et al . Federated active learning (F-AL): An efficient annotation strategy for federated learning [EB/OL ] . ( 2022-02-01 )[ 2023-07-01 ] . http://arxiv.‍org/abs/2202.00195 http://arxiv.‍org/abs/2202.00195 .

GAL Y , GHAHRAMANI Z . Dropout as a Bayesian approximation: Representing model uncertainty in deep learning [C ] // Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48 . New York : ACM , 2016 : 1050 - 1059 .

TANG J , QU M , MEI Q Z . PTE: Predictive text embedding through large-scale heterogeneous text networks [C ] // Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . New York : ACM , 2015 : 1165 - 1174 .

VELIČKOVIĆ P , CUCURULL G , CASANOVA A , et al . Graph attention networks [EB/OL ] . ( 2017-10-30 )[ 2023-07-01 ] . http://arxiv.org/abs/1710.10903 http://arxiv.org/abs/1710.10903 .

WANG X , JI H Y , SHI C , et al . Heterogeneous graph attention network [C ] // WWW’19: The World Wide Web Conference. New York: ACM , 2019 : 2022 - 2032 .

YAO L , MAO C S , LUO Y . Graph convolutional networks for text classification [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2019 , 33 ( 1 ): 7370 - 7377 .

HSU T M H , QI H , BROWN M . Measuring the effects of non-identical data distribution for federated visual classification [EB/OL ] . ( 2019-09-13 )[ 2023-07-01 ] . https://arxiv.‍org/abs/1909.06335 https://arxiv.‍org/abs/1909.06335 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

面向在线生成式人工智能服务的隐私保护方法

基于跨视图上下文感知的高分辨率遥感图像半监督语义分割方法

基于区块链的分层联邦学习系统

基于联邦大模型的网络攻击检测方法研究