An Active Semi-Supervised Short Text Classification Method Based on Federated Learning

KONG De-yan; JI Zhen-yan; YANG Yan-yan; LIU Yang; LIU Ji-qiang

doi:10.12263/DZXB.20230703

您当前的位置：

首页 >

文章列表页 >

An Active Semi-Supervised Short Text Classification Method Based on Federated Learning

PAPERS | 更新时间：2026-05-07

- An Active Semi-Supervised Short Text Classification Method Based on Federated Learning
- ACTA ELECTRONICA SINICA Vol. 52, Issue 10, Pages: 3517-3526(2024)
- 作者机构：
  
  1.北京交通大学软件学院，北京 100044
  2.北京交通大学网络空间安全学院智能交通数据安全与隐私保护北京市重点实验室，北京 100044
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China(52175493;51935002)
- DOI：10.12263/DZXB.20230703
  CLC： TP183;
- Received：26 July 2023，
  
  Revised：2024-03-04，
  
  Published：25 October 2024
- 稿件说明：
移动端阅览
孔德焱, 冀振燕, 杨燕燕, 等. 基于联邦学习的主动半监督短文本分类方法[J]. 电子学报, 2024, 52(10): 3517-3526.

KONG De-yan, JI Zhen-yan, YANG Yan-yan, et al. An Active Semi-Supervised Short Text Classification Method Based on Federated Learning[J]. Acta Electronica Sinica, 2024, 52(10): 3517-3526.
孔德焱, 冀振燕, 杨燕燕, 等. 基于联邦学习的主动半监督短文本分类方法[J]. 电子学报, 2024, 52(10): 3517-3526. DOI：10.12263/DZXB.20230703

KONG De-yan, JI Zhen-yan, YANG Yan-yan, et al. An Active Semi-Supervised Short Text Classification Method Based on Federated Learning[J]. Acta Electronica Sinica, 2024, 52(10): 3517-3526. DOI：10.12263/DZXB.20230703

摘要

短文本分类应用广泛，是当前的研究热点，但受到短文本标注数据稀缺和数据隐私保护不便集中训练的影响，分类效果不佳.针对上述问题，我们提出了基于联邦学习的主动半监督异质图注意力网络模型（Active Semi-Supervised Learning empowered Heterogeneous Graph ATtention network model based on Federated learning，Fed-ASSL-HGAT），通过设计新颖的主动半监督学习（Active Semi-Supervised Learning，ASSL）框架生成高质量标注样本赋能异质图注意力网络（Heterogeneous Graph ATttention network model，HGAT），引入联邦学习对部署在不同节点的模型进行联合训练以满足数据隐私保护需求.所提出的ASSL框架通过将主动学习的多类别标注转化成二元类别标注，可大大降低标注难度；设计基于信息增益的选择策略筛选软、硬标签，以防止信息损失；通过半监督学习选择高准确率、高稳定性的正负样本打伪标签以确保标注质量.实验结果表明，所提出的ASSL-HGAT（S）在AGNews、Snippets、TagMyNews数据集上相比HGAT基线模型

值分别提升2.45%、8.11%、7.46%.融合联邦学习所进一步提出的Fed-ASSL-HGAT模型可在不泄漏隐私数据的情况下满足性能要求.

Abstract

Short-text classification is broadly used and is a current hot research spot. However

the performance of short-text classification is hampered by the sca1rcity of annotated data for short texts and the challenges of centralized training for private data. To address these issues

we propose Fed-ASSL-HGAT (Active Semi-Supervised Heterogeneous Graph ATtention network model based on Federated learning)

an active semi-supervised heterogeneous graph attention network model based on federated learning. This model utilizes the innovative active semi-supervised learning (ASSL) framework to generate high-quality labeled samples for empowering the heterogeneous graph attention network (HGAT) model. Additionally

federated learning is introduced to facilitate the joint training of the models deployed on different nodes

thereby satisfying the requirements of data privacy protection. The proposed ASSL framework significantly reduces the annotation difficulty by transforming the multi-class annotation task into a binary classification task. To mitigate information loss

we employ a selection strategy based on information gain to filter soft and hard labels. Semi-supervised learning is employed to select positive and negative samples with high accuracy and stability for pseudo-labeling

thereby ensuring the labeling quality. Experimental results demonstrate that the proposed ASSL-HGAT (Active Semi-supervised Learning Empowered Heterogeneous Graph Attention Network) model achieves improvements of 2.45%

8.11%

and 7.46% in

scores comparing with the HGAT baseline model on the AGNews

Snippets

and TagMyNews datasets

respectively. By incorporating the federated learning

the Fed-ASSL-HGAT model can meet the performance requirements without scarifying data privacy.

关键词

Keywords

references

张昱 , 刘开峰 , 张全新 , 等 . 基于组合-卷积神经网络的中文新闻文本分类 [J ] . 电子学报 , 2021 , 49 ( 6 ): 1059 - 1067 .

ZHANG Y , LIU K F , ZHANG Q X , et al . A combined-convolutional neural network for Chinese news text classification [J ] . Acta Electronica Sinica , 2021 , 49 ( 6 ): 1059 - 1067 . (in Chinese)

李雪莹 , 王田路 , 梁鹏 , 等 . 基于系统模型的用户评论中非功能需求的自动分类 [J ] . 电子学报 , 2022 , 50 ( 9 ): 2079 - 2089 .

LI X Y , WANG T L , LIANG P , et al . Automatic classification of non-functional requirements in App user reviews based on system model [J ] . Acta Electronica Sinica , 2022 , 50 ( 9 ): 2079 - 2089 . (in Chinese)

YANG T C , HU L M , SHI C , et al . HGAT: Heterogeneous graph attention networks for semi-supervised short text classification [J ] . ACM Transactions on Information Systems , 39 ( 3 ): 32 .

MCMAHAN H B , MOORE E , RAMAGE D , et al . Communication-efficient learning of deep networks from decentralized data [EB/OL ] . ( 2016-02-17 )[ 2023-07-01 ] . http://arxiv.org/abs/1602.05629 http://arxiv.org/abs/1602.05629 .

ISCEN A , TOLIAS G , AVRITHIS Y , et al . Label propagation for deep semi-supervised learning [C ] // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2019 : 5065 - 5074 .

HAASE-SCHÜTZ C , STAL R , HERTLEIN H , et al . Iterative label improvement: Robust training by confidence based filtering and dataset partitioning [C ] // 2020 25th International Conference on Pattern Recognition (ICPR) . Piscataway : IEEE , 2021 : 9483 - 9490 .

BERTHELOT D , CARLINI N , Goodfellow I , et al . Mixmatch: A holistic approach to semi-supervised learning‍ [C ] // Proceedings of the 33rd International Conference on Neural Information Processing Systems . New York : Curran Associates Inc. , 2019 : 5049 - 5059 .

ZHOU T Y , WANG S J , BILMES J A . Time-consistent self-supervision for semi-supervised learning [C ] // Proceedings of the 37th International Conference on Machine Learning . New York : ACM , 2020 : 11523 - 11533 .

RIZVE M N , DUARTE K , RAWAT Y S , et al . In defense of pseudo-labeling: An uncertainty-aware pseudo-label selection framework for semi-supervised learning [EB/OL ] . ( 2021-01-15 )[ 2023-07-01 ] . http://arxiv.org/abs/2101.06329 http://arxiv.org/abs/2101.06329 .

李延超 , 肖甫 , 陈志 , 等 . 自适应主动半监督学习方法‍ [J ] . 软件学报 , 2020 , 31 ( 12 ): 3808 - 3822 .

LI Y C , XIAO F , CHEN Z , et al . Adaptive active learning for semi-supervised learning [J ] . Journal of Software , 2020 , 31 ( 12 ): 3808 - 3822 . (in Chinese)

ZHANG W Q , ZHU L , HALLINAN J , et al . BoostMIS: Boosting medical image semi-supervised learning with adaptive pseudo labeling and informative active annotation [C ] // 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2022 : 20634 - 20644 .

ZHANG W T , WANG Y X , YOU Z B , et al . Information gain propagation: A new way to graph active learning with soft labels [EB/OL ] . ( 2022-03-02 )[ 2023-07-01 ] . http://arxiv.org/abs/2203.01093 http://arxiv.org/abs/2203.01093 .

KONEČNÝ J , MCMAHAN H B , RAMAGE D , et al . Federated optimization: Distributed machine learning for on-device intelligence [EB/OL ] . ( 2016-10-08 )[ 2023-07-01 ] . http://arxiv.org/abs/1610.02527 http://arxiv.org/abs/1610.02527 .

HSU T M H , QI H , BROWN M . Federated visual classification with real-world data distribution [C ] // European Conference on Computer Vision . Cham : Springer , 2020 : 76 - 92 .

JEONG W , YOON J , YANG E , et al . Federated semi-supervised learning with inter-client consistency & disjoint learning [EB/OL ] . ( 2020-06-22 )[ 2023-07-01 ] . http://arxiv.‍org/abs/2006.12097 http://arxiv.‍org/abs/2006.12097 .

AHN J H , KIM K , KOH J , et al . Federated active learning (F-AL): An efficient annotation strategy for federated learning [EB/OL ] . ( 2022-02-01 )[ 2023-07-01 ] . http://arxiv.‍org/abs/2202.00195 http://arxiv.‍org/abs/2202.00195 .

GAL Y , GHAHRAMANI Z . Dropout as a Bayesian approximation: Representing model uncertainty in deep learning [C ] // Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48 . New York : ACM , 2016 : 1050 - 1059 .

TANG J , QU M , MEI Q Z . PTE: Predictive text embedding through large-scale heterogeneous text networks [C ] // Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . New York : ACM , 2015 : 1165 - 1174 .

VELIČKOVIĆ P , CUCURULL G , CASANOVA A , et al . Graph attention networks [EB/OL ] . ( 2017-10-30 )[ 2023-07-01 ] . http://arxiv.org/abs/1710.10903 http://arxiv.org/abs/1710.10903 .

WANG X , JI H Y , SHI C , et al . Heterogeneous graph attention network [C ] // WWW’19: The World Wide Web Conference. New York: ACM , 2019 : 2022 - 2032 .

YAO L , MAO C S , LUO Y . Graph convolutional networks for text classification [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2019 , 33 ( 1 ): 7370 - 7377 .

HSU T M H , QI H , BROWN M . Measuring the effects of non-identical data distribution for federated visual classification [EB/OL ] . ( 2019-09-13 )[ 2023-07-01 ] . https://arxiv.‍org/abs/1909.06335 https://arxiv.‍org/abs/1909.06335 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Building Privacy Shield in Online Generative AI Services

Cross-View Context-Aware Semi-Supervised Semantic Segmentation for High-Resolution Remote Sensing Images

Blockchain-Based Hierarchical Federated Learning System

Research on Network Attack Detection Method Based on Federated Large Model

Related Author

QI Tao

WANG Huili

YANG Peiru

WANG Wendan

TAN Zhipeng

HUANG Yongfeng

WANG Shangguang

XU Hongyan

Related Institution

School of Information Science, Beijing Forestry University

Wuhan National Optoelectronics Research Center, Huazhong University of Science and Technology

Department of Electronic Engineering, Tsinghua University

State Key Laboratory of Networking and Switching Technology, School of Computer Science, Beijing University of Posts and Telecommunications

School of Software, Shandong University

⁰