基于特征异常检测与伪标签回归的无监督对抗域适应

潘杰; 刘波; 邹筱瑜

doi:10.12263/DZXB.20240074

您当前的位置：

首页 >

文章列表页 >

基于特征异常检测与伪标签回归的无监督对抗域适应

学术论文 | 更新时间：2025-12-08

- 基于特征异常检测与伪标签回归的无监督对抗域适应
- Feature Anomaly Detection and Pseudo-Label Regression for Adversarial Domain Adaptation
- 电子学报 2025年53卷第1期页码：128-140
- 作者机构：
  
  1.中国矿业大学信息与控制工程学院，江苏徐州 221116
  2.中国矿业大学机电工程学院，江苏徐州 221116
- 作者简介：
  
  [ "潘杰男，1986年12月出生于江苏省徐州市.现为中国矿业大学信息与控制工程学院副教授、硕士生导师.主要研究方向为机器学习与网络生理学." ]
  [ "刘波男，1997年10月出生于湖南省邵阳市.2023年毕业于中国矿业大学信息与控制工程学院.现为中国电信股份有限公司长沙分公司工程师.从事算法设计与云网设施维护工作.E-mail: liubo7556@163.com" ]
  [ "邹筱瑜女，1990年5月出生于四川省自贡市.现为中国矿业大学机电工程学院副教授、硕士生导师.主要研究方向为机器学习与装备运维." ]
- 基金信息：
  
  国家自然科学基金(62176258;62273349;61806207);中央高校基本科研业务费专项资金项目(2021YCPY0111)
- DOI：10.12263/DZXB.20240074
  中图分类号： TP181
- 收稿：2024-01-15，
  
  修回：2024-09-06，
  
  纸质出版：2025-01-25
- 稿件说明：
移动端阅览
潘杰, 刘波, 邹筱瑜. 基于特征异常检测与伪标签回归的无监督对抗域适应[J]. 电子学报, 2025, 53(01): 128-140.

PAN Jie, LIU Bo, ZOU Xiao-yu. Feature Anomaly Detection and Pseudo-Label Regression for Adversarial Domain Adaptation[J]. Acta Electronica Sinica, 2025, 53(01): 128-140.
潘杰, 刘波, 邹筱瑜. 基于特征异常检测与伪标签回归的无监督对抗域适应[J]. 电子学报, 2025, 53(01): 128-140. DOI：10.12263/DZXB.20240074

PAN Jie, LIU Bo, ZOU Xiao-yu. Feature Anomaly Detection and Pseudo-Label Regression for Adversarial Domain Adaptation[J]. Acta Electronica Sinica, 2025, 53(01): 128-140. DOI：10.12263/DZXB.20240074

摘要

无监督域适应任务中源域和目标域通常不满足独立同分布假设. 为生成目标域可用标签，经典域适应方法选择分类器预测概率最大的类别作为目标样本伪标签，使伪标签中可能包含噪声信息，造成域适应“负迁移”. 此外，传统对抗域适应方法往往考虑对齐领域间全局分布，较少关注样本类别信息，如何在域适应任务中提取判别性类别级特征至关重要. 为此，本文提出一种基于特征异常检测与伪标签回归的无监督对抗域适应方法. 通过分类器预测同类别目标样本组成目标域类别子域，引入高斯均匀混合模型检测与类均值特征距离异常的子域样本，计算样本后验概率并以此度量子域中样本伪标签的正确性，作为损失因子限制伪标签在训练中对模型的影响. 同时，采用伪标签回归函数减小分类器预测标签与高置信度伪标签差异，对无标签目标域进行类别约束，提高特征类别可辨别性. 实验表明，所提方法在数据集Office-31、Image-CLEF和Office-Home上平均识别精度分别为90.2%、89.6%和69.5%，较相关主流算法均有提升.

Abstract

In unsupervised domain adaptation tasks

the source and target domains usually do not satisfy the independent and identical distribution assumption. In order to generate the usable labels for the target domain

classical domain adaptation methods select the category with the highest prediction probability of the classifier as the pseudo-label of the target sample. Thus

the pseudo-label inevitably contains certain noise information

which may cause negative transfer to the domain adaptation model. In addition

traditional adversarial domain adaptation methods usually consider the global distribution between domains and ignore the category information of samples. How to extract discriminative category-level features in domain adaptation tasks is also an important problem. Therefore

an unsupervised adversarial domain adaptation method is proposed using feature anomaly detection and pseudo-label regression. The target samples of the same class predicted by the classifier are formed into the category subdomain within the target domain. The Gaussian uniform mixture model is used to detect the subdomain samples with abnormal distance from the class mean. The posterior probability of the samples is calculated and the correctness of the sample pseudo-labels in the subdomain is measured

which is used as a loss factor to limit the influence of pseudo-labels on the model in training. Meanwhile

the pseudo-label regression function is used to reduce the difference between the predicted label and the high-confidence pseudo-label of the classifier. The category constraint of the unlabeled target domain is adopted to improve the distinguishability of feature categories. Experimental results show that the average recognition accuracy of the proposed method on datasets Office-31

Image-CLEF

and Office-Home are 90.2%

89.6%

and 69.5%

respectively

which are all higher than the related popular algorithms.

关键词

Keywords

references

韩冲 , 王俊丽 , 吴雨茜 , 等 . 基于神经进化的深度学习模型研究综述 [J ] . 电子学报 , 2021 , 49 ( 2 ): 372 - 379 .

HAN C , WANG J L , WU Y X , et al . A review of deep learning models based on neuroevolution [J ] . Acta Electronica Sinica , 2021 , 49 ( 2 ): 372 - 379 . (in Chinese)

XIANG X , ABDEIN R , LI W , et al . Deep scene flow learning: From 2D images to 3D point clouds [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2024 , 46 ( 1 ): 185 - 208 .

HE K , ZHANG X , REN S , et al . Deep residual learning for image recognition [C ] // IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2016 : 770 - 778 .

MITSUZUMI Y , IRIE G , KIMURA A , et al . Phase randomization: A data augmentation for domain adaptation in human action recognition [J ] . Pattern Recognition , 2024 , 146 ( 1 ): 1 - 11 .

PAN S J , YANG Q . A survey on transfer learning [J ] . IEEE Transactions on Knowledge and Data Engineering , 2010 , 22 ( 10 ): 1345 - 1359 .

PAN S J , TSANG I W , KWOK J T , et al . Domain adaptation via transfer component analysis [J ] . IEEE Transactions on Neural Networks , 2010 , 22 ( 2 ): 199 - 210 .

AZIZZADENESHELI K . Importance weight estimation and generalization in domain adaptation under label shift [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022 , 44 ( 10 ): 6578 - 6584 .

范苍宁 , 刘鹏 , 肖婷 , 等 . 深度域适应综述: 一般情况与复杂情况 [J ] . 自动化学报 , 2021 , 47 ( 3 ): 515 - 548 .

FAN C N , LIU P , XIAO T , et al . A review of deep domain adaptation: General situation and complex situation [J ] . Acta Automatica Sinica , 2021 , 47 ( 3 ): 515 - 548 . (in Chinese)

孙晨峰 , 吕卫民 , 戴洪德 , 等 . 一种基于TimeGAN和OCSVM的多元退化设备小子样数据增广方法 [J ] . 电子学报 , 2022 , 50 ( 11 ): 2678 - 2687 .

SUN C F , LÜ W M , DAI H D , et al . A small sample data augmentation method for multivariate degradation equipment based on TimeGAN and OCSVM [J ] . Acta Electronica Sinica , 2022 , 50 ( 11 ): 2678 - 2687 . (in Chinese)

WANG W , LI H , DING Z , et al . Rethinking maximum mean discrepancy for visual domain adaptation [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2023 , 34 ( 1 ): 264 - 277 .

TZENG E , HOFFMAN J , ZHANG N , et al . Deep domain confusion: Maximizing for domain invariance [EB/OL ] . [2024 ] . https://arxiv.org/abs/1412.3474 https://arxiv.org/abs/1412.3474 .

LONG M , CAO Y , WANG J , et al . Learning transferable features with deep adaptation networks [C ] // Proceedings of the International Conference on Machine Learning . New York : ILMS , 2015 : 97 - 105 .

SUN B , SAENKO K . Deep CORAL: Correlation alignment for deep domain adaptation [C ] // Proceedings of the European Conference on Computer Vision . Amsterdam : Springer , 2016 : 443 - 450 .

ZELLINGER W , GRUBINGER T , LUGHOFER E , et al . Central moment discrepancy for domain-invariant representation learning [C ] // Proceedings of the 5th International Conference on Learning Representations . Toulon : Springer , 2017 : 1 - 11 .

ANDÉOL L , KAWAKAMI Y , WADA Y , et al . Learning domain invariant representations by joint Wasserstein distance minimization [J ] . Neural Networks , 2023 , 167 ( 1 ): 233 - 243 .

王格格 , 郭涛 , 余游 , 等 . 基于生成对抗网络的无监督域适应分类模型 [J ] . 电子学报 , 2020 , 48 ( 6 ): 1190 - 1197 .

WANG G G , GUO T , YU Y , et al . Unsupervised domain adaptation classification model based on generative adversarial network [J ] . Acta Electronica Sinica , 2020 , 48 ( 6 ): 1190 - 1197 . (in Chinese)

GANIN Y , LEMPITSKY V S . Unsupervised domain adaptation by backpropagation [C ] // International Conference on Machine Learning . Lille : IMLS , 2015 : 1180 - 1189 .

ISMAEL S F , KAYABOL K , APTOULA E . Unsupervised domain adaptation for the semantic segmentation of remote sensing images via one-shot image-to-image translation [J ] . IEEE Geoscience and Remote Sensing Letters , 2023 , 20 ( 1 ): 1 - 12 .

ZHAO S , YUE X , ZHANG S , et al . A review of single-source deep unsupervised visual domain adaptation [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2022 , 33 ( 2 ): 473 - 493 .

PEI Z , CAO Z , LONG M . Multi-adversarial domain adaptation [C ] // Proceedings of the Thirty-Second Conference on Artificial Intelligence . Louisiana : AAAI , 2018 : 3934 - 3941 .

SAFFARI M , KHODAYAR M . Low-rank sparse generative adversarial unsupervised domain adaptation for multi-target traffic scene semantic segmentation [J ] . IEEE Transactions on Industrial Informatics , 2024 , 20 ( 2 ): 2564 - 2576 .

LONG M , CAO Z , WANG J , et al . Conditional adversarial domain adaptation [C ] // Proceedings of the Annual Conference on Neural Information Processing Systems . New York : ACM , 2018 : 1647 - 1657 .

DAN J , JIN T , CHI H , et al . Trust-aware conditional adversarial domain adaptation with feature norm alignment [J ] . Neural Networks , 2023 , 168 ( 1 ): 518 - 530 .

CHO J , KIM D , JUNG Y , et al . MCDAL: Maximum classifier discrepancy for active learning [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2023 , 34 ( 11 ): 8753 - 8763 .

SAITO K , KIM D , SCLAROFF S , et al . Semi-supervised domain adaptation via minimax entropy [C ] // IEEE/CVF International Conference on Computer Vision . Piscataway : IEEE , 2019 : 8050 - 8058 .

SAITO K , USHIKU Y , HARADA T . Asymmetric tri-training for unsupervised domain adaptation [C ] // Proceedings of the International Conference on Machine Learning . New York : IMLS , 2017 : 2988 - 2997 .

XIE B , LI S , LI M , et al . SePiCo: Semantic-guided pixel contrast for domain adaptive semantic segmentation [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2023 , 45 ( 7 ): 9004 - 9021 .

ZHANG W , OUYANG W , LI W , et al . Collaborative and adversarial network for unsupervised domain adaptation [C ] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 3801 - 3809 .

ZHANG W , XU D , OUYANG W , et al . Self-paced collaborative and adversarial network for unsupervised domain adaptation [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021 , 43 ( 6 ): 2047 - 2061 .

BENGIO Y , COURVILLE A , VINCENT P . Representation learning: A review and new perspectives [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2013 , 35 ( 8 ): 1798 - 1828 .

HARARY S , SCHWARTZ E , ARBELLE A , et al . Unsupervised domain generalization by learning a bridge across domains [C ] // IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2022 : 5270 - 5280 .

TZENG E , HOFFMAN J , SAENKO K , et al . Adversarial discriminative domain adaptation [C ] // IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2017 : 2962 - 2971 .

GU X , SUN J , XU Z . Spherical space domain adaptation with robust pseudo-label loss [C ] // IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2020 : 9098 - 9107 .

CHEN S , JIA X , HE J , et al . Semi-supervised domain adaptation based on dual-level domain mixing for semantic segmentation [C ] // IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2021 : 11013 - 11022 .

LATHUILIÈRE S , MESEJO P , ALAMEDA-PINEDA X , et al . Deepgum: Learning deep robust regression with a Gaussian-uniform mixture model [C ] // European Conference on Computer Vision . Cham : Springer , 2018 : 202 - 217 .

HUANG J , XIAO N , ZHAN L . Balancing transferability and discriminability for unsupervised domain adaptation [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2024 , 35 ( 4 ): 5807 - 5814 .

ZHANG J , DING Z , LI W , et al . Importance weighted adversarial nets for partial domain adaptation [C ] // IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 8156 - 8164 .

MA A , LI J , LU K , et al . Adversarial entropy optimization for unsupervised domain adaptation [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2022 , 33 ( 11 ): 6263 - 6274 .

SAENKO K , KULIS B , FRITZ M . Adapting visual category models to new domains [C ] // European Conference on Computer Vision . Berlin : Springer , 2010 : 213 - 226 .

WANG S , ZHANG L , ZUO W , et al . Class-specific reconstruction transfer learning for visual recognition across domains [J ] . IEEE Transactions on Image Processing , 2019 , 29 ( 1 ): 2424 - 2438 .

VENKATESWARA H , EUSEBIO J , CHAKRABORTY S , et al . Deep hashing network for unsupervised domain adaptation [C ] // IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2017 : 5018 - 5027 .

RUSSAKOVSKY O , DENG J , SU H , et al . Imagenet large scale visual recognition challenge [J ] . International Journal of Computer Vision , 2015 , 115 ( 3 ): 211 - 252 .

LONG M , ZHU H , WANG J , et al . Deep transfer learning with joint adaptation networks [C ] // Proceedings of the International Conference on Machine Learning . New York : IMLS , 2017 : 2208 - 2217 .

ZHU Y , ZHUANG F , WANG J , et al . Deep subdomain adaptation network for image classification [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2020 , 32 ( 4 ): 1713 - 1722 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

背景感知机制的图像分类网络

基于坐标重要性池化和解耦类别对齐蒸馏的图像分类算法

基于双模糊学习的鲁棒无监督特征选择算法

基于图表征知识蒸馏的图像分类方法