Category Expansion Based Generalized Zero-Shot Image Classification

ZHANG Jie; LIAO Sheng-bin; ZHANG Hao-feng; CHEN De-bao

doi:10.12263/DZXB.20220036

您当前的位置：

首页 >

文章列表页 >

Category Expansion Based Generalized Zero-Shot Image Classification

PAPERS | 更新时间：2025-12-08

- Category Expansion Based Generalized Zero-Shot Image Classification
- ACTA ELECTRONICA SINICA Vol. 51, Issue 4, Pages: 1068-1080(2023)
- 作者机构：
  
  1.南京理工大学计算机科学与工程学院，江苏南京 210094
  2.华中师范大学国家数字化学习工程技术研究中心，湖北武汉 430079
  3.淮北师范大学计算机科学与技术学院，安徽淮北 235000
- 作者简介：
- 基金信息：
- DOI：10.12263/DZXB.20220036
  CLC： TP391.4;TP37
- Received：03 January 2022，
  
  Revised：2022-02-28，
  
  Published：25 April 2023
- 稿件说明：
移动端阅览
张杰,廖盛斌,张浩峰等.基于类别扩展的广义零样本图像分类方法[J].电子学报,2023,51(04):1068-1080.

ZHANG Jie,LIAO Sheng-bin,ZHANG Hao-feng,et al.Category Expansion Based Generalized Zero-Shot Image Classification[J].ACTA ELECTRONICA SINICA,2023,51(04):1068-1080.
张杰,廖盛斌,张浩峰等.基于类别扩展的广义零样本图像分类方法[J].电子学报,2023,51(04):1068-1080. DOI： 10.12263/DZXB.20220036.

ZHANG Jie,LIAO Sheng-bin,ZHANG Hao-feng,et al.Category Expansion Based Generalized Zero-Shot Image Classification[J].ACTA ELECTRONICA SINICA,2023,51(04):1068-1080. DOI： 10.12263/DZXB.20220036.

摘要

在传统的零样本图像分类方法中，语义属性通常被用作辅助信息来描述各类别的视觉特征.然而，单一的语义属性并不能对类内多样性的视觉特征进行全面的描述.为提高语义属性对类别内部多样性的表示能力，同时也为了帮助模型提高对各类别的描述能力，本文通过属性自编码器的方式在视觉以及语义空间上对类别进行扩展.此外，为了缓解传统生成性方法因无法直接计算生成空间到真实空间的变换而带来的模型次优解问题，本文采用了生成流网络作为基础网络，通过可逆变换直接计算两个空间之间的变换来开展对零样本学习任务的研究.本文使用解码器网络将逆生成流网络为测试样本生成的原型特征解耦成视觉原型及语义原型信息，然后根据这两个原型信息实现将测试样本预分类到可见类集或不可见类集中，最终在这两个子分类空间中根据样本的特点分别进行监督分类和零样本分类任务以提高模型的整体性能表现.本文在五个数据集上通过大量的实验验证了本文所提方法的有效性.

Abstract

In traditional zero-shot image classification methods

semantic attributes are usually used as auxiliary information to describe the visual features of each class. However

a single semantic attribute cannot fully describe the diverse visual features within a single class. To improve the ability of semantic attributes to express the diversity within the class

and to help the model improve the description ability for each category

we use the semantic auto-encoder to expand the categories in visual and semantic space. In addition

to alleviate the suboptimal solution problem of the model caused by the inability to directly calculate the transformation from the generation space to the real space by the traditional generative methods

we employ the generative flow as the basic network in this paper to directly calculate the transformation between the two spaces. Furthermore

we exploit the decoder network to decouple the prototype features generated by the inverse generative flow network for the test samples into visual prototypes and semantic prototypes

and then realize the pre-classification of the test samples into seen or unseen classes. Finally

in the two sub-classification domains

supervised classification and zero-shot classification are performed separately to improve the overall performance. Extensive experiments are conducted on five popular datasets to verify the effectiveness of the proposed method.

关键词

Keywords

references

LAROCHELLE H , ERHAN D , BENGIO Y . Zero-data learning of new tasks [C ] // Proceedings of the 23rd national conference on Artificial intelligence-Volume 2 . Chicago : AAAI Press , 2008 : 646 - 651 .

CHAO W L , CHANGPINYO S , GONG B Q , et al . An empirical study and analysis of generalized zero-shot learning for object recognition in the wild [C ] // European Conference on Computer Vision . Amsterdam : Springer , 2016 : 52 - 68 .

LAMPERT C H , NICKISCH H , HARMELING S . Attribute-based classification for zero-shot visual object categorization [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2014 , 36 ( 3 ): 453 - 465 .

程玉虎 , 乔雪 , 王雪松 . 基于混合属性的零样本图像分类 [J ] . 电子学报 , 2017 , 45 ( 6 ): 1462 - 1468 .

CHENG Y H , QIAO X , WANG X S . Hybrid attribute-based zero-shot image classification [J ] . Acta Electronica Sinica , 2017 , 45 ( 6 ): 1462 - 1468 . (in Chinese)

赵鹏 , 汪纯燕 , 张思颖 , 等 . 一种基于融合重构的子空间学习的零样本图像分类方法 [J ] . 计算机学报 , 2021 , 44 ( 2 ): 409 - 421 .

ZHAO P , WANG C Y , ZHANG S Y , et al . A zero-shot image classification method based on subspace learning with the fusion of reconstruction [J ] . Chinese Journal of Computers , 2021 , 44 ( 2 ): 409 - 421 . (in Chinese)

BAI H Y , ZHANG H F , WANG Q . Dual discriminative auto-encoder network for zero shot image recognition [J ] . Journal of Intelligent & Fuzzy Systems , 2021 , 40 ( 3 ): 5159 - 5170 .

ZHANG H F , LIU L , LONG Y , et al . Deep transductive network for generalized zero shot learning [J ] . Pattern Recognition , 2020 , 105 : 107370 .

冀中 , 汪浩然 , 于云龙 , 等 . 零样本图像分类综述: 十年进展 [J ] . 中国科学: 信息科学 , 2019 , 49 ( 10 ): 1299 - 1320 .

JI Z , WANG H R , YU Y L , et al . A decadal survey of zero-shot image classification [J ] . Scientia Sinica Informationis , 2019 , 49 ( 10 ): 1299 - 1320 . (in Chinese)

KODIROV E , XIANG T , GONG S G . Semantic autoencoder for zero-shot learning [C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu : IEEE , 2017 : 4447 - 4456 .

GOODFELLOW I , POUGET-ABADIE J , MIRZA M , et al . Generative adversarial networks [J ] . Communications of the ACM , 2020 , 63 ( 11 ): 139 - 144 .

KINGMA D P , WELLIING M . Auto-encoding variational bayes [C ] // In Proceedings of the International Conference on Learning Representations . Banff : ICLR , 2014 : 1 - 14 .

DINH L , KRUEGER D , BENGIO Y . Nice: Non-linear independent components estimation [C ] // Proceedings of the International Conference on Learning Representations . San Diego : ICLR , 2015 : 1 - 14 .

CHEN Z , LUO Y D , WANG S , et al . Mitigating generation shifts for generalized zero-shot learning [C ] // Proceedings of the 29th ACM International Conference on Multimedia . Virtual Conference : ACM , 2021 : 844 - 852 .

SHEN Y M , QIN J , HUANG L , et al . Invertible zero-shot recognition flows [C ] // European Conference on Computer Vision . Virtual Conference : Springer , 2020 : 614 - 631 .

LIU J R , FU L Y , ZHANG H F , et al . Learning discriminative and representative feature with cascade GAN for generalized zero-shot learning [J ] . Knowledge-Based Systems , 2022 , 236 : 107780 .

ZHANG H F , WANG Y D , LONG Y , et al . Modality independent adversarial network for generalized zero shot image classification [J ] . Neural Networks: The Official Journal of the International Neural Network Society , 2021 , 134 : 11 - 22 .

ZHANG H F , BAI H Y , LONG Y , et al . A plug-in attribute correction module for generalized zero-shot learning [J ] . Pattern Recognition , 2021 , 112 : 107767 .

XIAN Y Q , LORENZ T , SCHIELE B , et al . Feature generating networks for zero-shot learning [C ] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE , 2018 : 5542 - 5551 .

SARIYILDIZ M B , CINBIS R G . Gradient matching generative networks for zero-shot learning [C ] // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach : IEEE , 2019 : 2163 - 2173 .

SCHÖNFELD E , EBRAHIMI S , SINHA S , et al . Generalized zero- and few-shot learning via aligned variational autoencoders [C ] // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach : IEEE , 2019 : 8239 - 8247 .

WANG W L , PU Y C , VERMA V , et al . Zero-shot learning via class-conditioned deep generative models [C ] // Proceedings of the AAAI Conference on Artificial Intelligence . New Orleans : AAAI Press , 2018 , 32 ( 1 ): 4211 - 4218 .

ARDIZZONE L , KRUSE J , ROTHER C , et al . Analyzing inverse problems with invertible neural networks [C ] // Proceedings of the International Conference on Learning Representations . Vancouver : ICLR , 2018 : 1 - 20

王格格 , 郭涛 , 余游 , 等 . 基于生成对抗网络的无监督域适应分类模型 [J ] . 电子学报 , 2020 , 48 ( 6 ): 1190 - 1197 .

WANG G G , GUO T , YU Y , et al . Unsupervised domain adaptation classification model based on generative adversarial network [J ] . Acta Electronica Sinica , 2020 , 48 ( 6 ): 1190 - 1197 . (in Chinese)

席亮 , 刘涵 , 樊好义 , 等 . 基于深度对抗学习潜在表示分布的异常检测模型 [J ] . 电子学报 , 2021 , 49 ( 7 ): 1257 - 1265 .

XI L , LIU H , FAN H Y , et al . Deep adversarial learning latent representation distribution model for anomaly detection [J ] . Acta Electronica Sinica , 2021 , 49 ( 7 ): 1257 - 1265 . (in Chinese)

CHEN X Y , LAN X G , SUN F C , et al . A boundary based out-of-distribution classifier for generalized zero-shot learning [C ] // European Conference on Computer Vision . Virtual Conference : Springer , 2020 : 572 - 588 .

DAVIDSON T R , FALORSI L , DE CAO N , et al . Hyperspherical variational auto-encoders [C ] // Proceeding of the Conference on Uncertainty in Artificial Intelligence 2018 . California : AUAI , 2018 : 856 - 865 .

ATZMON Y , CHECHIK G . Adaptive confidence smoothing for generalized zero-shot learning [C ] // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach : IEEE , 2019 : 11663 - 11672 .

MIN S B , YAO H T , XIE H T , et al . Domain-aware visual bias eliminating for generalized zero-shot learning [C ] // 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle : IEEE , 2020 : 12661 - 12670 .

QUEEN J MAC . Some methods for classification and analysis of multivariate observations [C ] // Proceedings of the Berkeley Symposium on Mathematical Statistics and Probability . California : The Regents of the University of California , 1967 : 281 - 297 .

BARTELS R H , STEWART G W . Solution of the matrix equation AX + XB = C [F 4 ] [J ] . Communications of the ACM , 1972 , 15 ( 9 ): 820 - 826 .

KINGMA D P , DHARIWAL P . Glow: Generative flow with invertible 1×1 convolutions [C ] // Proceedings of the Anuual Conference on Neural Information Processing Systems . Montreal : NIPS , 2018 : 10236 - 10245 .

XIAN Y Q , LAMPERT C H , SCHIELE B , et al . Zero-shot learning—A comprehensive evaluation of the good, the bad and the ugly [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2019 , 41 ( 9 ): 2251 - 2265 .

WELINDER P , BRANSON S , MITA T , et al . The caltech-ucsd birds-200-2011 dataset [R ] . California : California Institute of Technology, Technical Report: CNS-TR-2010-001 , 2010 .

PATTERSON G , XU C , SU H , et al . The SUN attribute database: Beyond categories for deeper scene understanding [J ] . International Journal of Computer Vision , 2014 , 108 ( 1/2 ): 59 - 81 .

NILSBACK M E , ZISSERMAN A . Automated flower classification over a large number of classes [C ] // 2008 Sixth Indian Conference on Computer Vision , Graphics & Image Processing . Bhubaneswar : IEEE , 2008 : 722 - 729 .

LAMPERT C H , NICKISCH H , HARMELING S . Learning to detect unseen object classes by between-class attribute transfer [C ] // 2009 IEEE Conference on Computer Vision and Pattern Recognition . Miami : IEEE , 2009 : 951 - 958 .

CHURCH K W . WORD2VEC [J ] . Natural Language Engineerring , 2017 , 23 ( 1 ): 155 - 162 .

JIA Z , ZHANG Z , WANG L , et al . Deep unbiased embedding transfer for zero-shot learning [J ] . IEEE Transactions on Image Processing , 2019 , 29 : 1958 - 1971 .

HAN Z Y , FU Z Y , YANG J . Learning the redundancy-free features for generalized zero-shot object recognition [C ] // 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle : IEEE , 2020 : 12862 - 12871 .

VYAS M R , VENKATESWARA H , PANCHANATHAN S . Leveraging seen and unseen semantic relationships for generative zero-shot learning [C ] // European Conference on Computer Vision . Virtual Conference : Springer , 2020 : 70 - 86 .

HUANG S , LIN J K , HUANGFU L W . Class-prototype discriminative network for generalized zero-shot learning [J ] . IEEE Signal Processing Letters , 2020 , 27 : 301 - 305 .

KESHARI R , SINGH R , VATSA M . Generalized zero-shot learning via over-complete distribution [C ] // 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle : IEEE , 2020 : 13297 - 13305 .

LI X Y , XU Z , WEI K , et al . Generalized zero-shot learning via disentangled representation [C ] // Proceedings of the AAAI Conference on Artificial Intelligence . Virtual Conference : AAAI Press , 2021 , 35 ( 3 ): 1966 - 1974 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰