基于关键局部语义对齐的小样本图像分类算法

孙哲; 郑旺; 郭朋飞

doi:10.12263/DZXB.20240209

您当前的位置：

首页 >

文章列表页 >

基于关键局部语义对齐的小样本图像分类算法

学术论文 | 更新时间：2024-09-23

- 基于关键局部语义对齐的小样本图像分类算法
- Few-Shot Image Classification Algorithm Based on Key Local Semantic Alignment
- 电子学报 2024年页码：1-10
- 作者机构：
  
  燕山大学信息科学与工程学院，河北秦皇岛 066004
- 作者简介：
  
  [ "孙哲女，1989年12月出生于河北省邯郸市.现为燕山大学信息科学与工程学院副教授、博士生/硕士生导师.分别于2013年和2018年在燕山大学获得学士和博士学位.2017年9月—2018年3月，于澳大利亚纽卡斯尔大学博士联合培养.主要研究方向包括多模态信号处理、小样本学习、视频理解，在国内外发表学术论文40余篇. E-mail: sunzhe_ysu@163.com" ]
  [ "郑旺男，1996年2月出生于新疆维吾尔自治区巴音郭楞蒙古自治州.现为燕山大学信息科学与工程学院硕士.主要研究方向为小样本图像分类.E-mail: 18196480916@163.com" ]
  [ "郭朋飞男，2000年12月出生于河南省南阳市.现为燕山大学信息科学与工程学院硕士.主要研究方向为小样本图像分类.E-mail: gpf_xx@163.com" ]
- 基金信息：
  
  国家自然科学基金(62001413);河北省高等学校科学技术研究项目——青年拔尖人才项目(BJK2023117);燕山大学基础创新科研培育重点项目(2023LGZD006)
- DOI：10.12263/DZXB.20240209
  中图分类号： TP391.4;
- 收稿：2024-03-04，
  
  修回：2024-06-17，
  
  网络出版：2024-09-23，
  
  纸质出版：2024-10-25
- 稿件说明：
移动端阅览
孙哲, 郑旺, 郭朋飞. 基于关键局部语义对齐的小样本图像分类算法[J]. 电子学报, 2024, 52(10): 1-10.

SUN Zhe, ZHENG Wang, GUO Peng-fei. Few-Shot Image Classification Algorithm Based on Key Local Semantic Alignment[J]. Acta Electronica Sinica, 2024, 52(10): 1-10.
孙哲, 郑旺, 郭朋飞. 基于关键局部语义对齐的小样本图像分类算法[J]. 电子学报, 2024, 52(10): 1-10. DOI：10.12263/DZXB.20240209

SUN Zhe, ZHENG Wang, GUO Peng-fei. Few-Shot Image Classification Algorithm Based on Key Local Semantic Alignment[J]. Acta Electronica Sinica, 2024, 52(10): 1-10. DOI：10.12263/DZXB.20240209

摘要

小样本分类旨在从少量标记样本中学习识别新类.目前基于局部描述符的小样本分类方法因考虑了局部特征在可见类和不可见类中的一致性取得了较好的分类性能.然而，基于局部描述符的表示方法存在邻近表示信息冗余、部分表示与图像语义无关、可解释性差等问题.鉴于此，本文提出一种基于关键局部语义对齐的小样本图像分类算法（Key Local Semantic Alignment Network，KLSANet），该方法通过对齐局部语义来实现图像到类的度量以完成分类.为了减轻图像语义无关局部对分类的影响，本文进一步设计了关键局部筛选模块并通过设置阈值筛选出关键局部块.KLSANet在三个广泛使用的基准数据集上均表现出较好的分类性能，尤其在1-shot和5-shot设置上比最优的对比方法平均提高了3.95%和2.56%.本文的代码公布在：

https：//github.com/ZitZhengWang/KLSANet

https://github.com/ZitZhengWang/KLSANet

Abstract

Few-shot classification aims to recognize new classes with a limited number of labeled samples. Currently

methods based on local descriptors achieve good performance by leveraging the consistency of local features in both visible and unseen classes. However

these methods often suffer from issues such as redundant neighboring representations

irrelevance to image semantics

and poor interpretability. To address the above problems

this paper proposes a Key Local Semantic Alignment Network(KLSANet)

a few-shot image classification approach based on key local semantic alignment network

which improves few-shot image classification by aligning local semantics for image-to-class measurement. To minimize the impact of semantically irrelevant local parts

we design a key local screening module that filters out non-essential blocks using set thresholds. KLSANet demonstrates superior performanc

e on three benchmark datasets

outperforming the best comparison methods by 3.95% and 2.56% in the 1-shot and 5-shot settings

respectively. The code is available at:

https://github.com/ZitZhengWang/KLSANet

关键词

Keywords

references

WANG Y Q , YAO Q M , KWOK J T , et al . Generalizing from a few examples: A survey on few-shot learning [J ] . ACM Computing Surveys , 2021 , 53 ( 3 ): 1 - 34 .

SONG Y S , WANG T , CAI P Y , et al . A comprehensive survey of few-shot learning: Evolution, applications, challenges, and opportunities [J ] . ACM Computing Surveys , 2023 , 55 ( 13 s): 1 - 40 .

LU J , GONG P H , YE J P , et al . A survey on machine learning from few samples [J ] . Pattern Recognition , 2023 , 139 : 109480 .

葛轶洲 , 刘恒 , 王言 , 等 . 小样本困境下的深度学习图像识别综述 [J ] . 软件学报 , 2022 , 33 ( 1 ): 193 - 210 .

GE Y Z , LIU H , WANG Y , et al . Survey on deep learning image recognition in dilemma of small samples [J ] . Journal of Software , 2022 , 33 ( 1 ): 193 - 210 . (in Chinese)

FINN C , ABBEEL P , LEVINE S . Model-agnostic meta-learning for fast adaptation of deep networks [C ] // Proceedings of the 34th International Conference on Machine Learning . New York : ACM , 2017 : 1126 - 1135 .

RAVI S , LAROCHELLE H . Optimization as a model for few-shot learning [C ] // International Conference on Learning Representations . Vancouver : Openreview.net , 2017 : 1 - 11 .

VINYALS O , BLUNDELL C , LILLICRAP T , et al . Matching networks for one shot learning [J ] . Advances in Neural Information Processing Systems , 2016 : 3637 - 3645 .

CHEN Y B , LIU Z , XU H J , et al . Meta-baseline: Exploring simple meta-learning for few-shot learning [C ] // 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2021 : 9042 - 9051 .

CAO K , BRBIC M , LESKOVEC J . Concept learners for few-shot learning [C ] // International Conference on Learning Representations . Vancouver : Openreview.net , 2021 : 1 - 17 .

赵凯琳 , 靳小龙 , 王元卓 . 小样本学习研究综述 [J ] . 软件学报 , 2021 , 32 ( 2 ): 349 - 369 .

ZHAO K L , JIN X L , WANG Y Z . Survey on few-shot learning [J ] . Journal of Software , 2021 , 32 ( 2 ): 349 - 369 . (in Chinese)

RUSU A A , RAO D , SYGNOWSKI J , et al . Meta-learning with latent embedding optimization [EB/OL ] . ( 2019-03-26 )[ 2024-03-02 ] . https://arxiv.org/abs/1807.05960 https://arxiv.org/abs/1807.05960 .

LI Z G , ZHOU F W , CHEN F , et al . Meta-sgd: Learning to learn quickly for few-shot learning [EB/OL ] . ( 2017-09-28 )[ 2024-03-02 ] . https://arxiv.org/abs/1707.09835 https://arxiv.org/abs/1707.09835 .

GRANT E , FINN C , LEVINE S , et al . Recasting gradient-based meta-learning as hierarchical bayes [EB/OL ] . ( 2018-01-26 )[ 2024-03-02 ] . https://arxiv.org/abs/1801.08930 https://arxiv.org/abs/1801.08930 .

OH J , YOO H , KIM C H , et al . Boil: Towards representation change for few-shot learning [EB/OL ] . ( 2021-03-03 )[ 2024-03-02 ] . https://arxiv.org/abs/2008.08882 https://arxiv.org/abs/2008.08882 .

LAI N , KAN M N , HAN C R , et al . Learning to learn adaptive classifier-predictor for few-shot learning [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2020 , 32 ( 8 ): 3458 - 3470 .

SANTORO A , BARTUNOV S , BOTVINICK M , et al . Meta-learning with memory-augmented neural networks [C ] // Proceedings of The 33rd International Conference on Machine Learning . New York : ACM , 2016 : 1842 - 1850 .

MUNKHDALAI T , YU H . Meta networks [C ] // Proceedings of the 34th International Conference on Machine Learning . New York : ACM , 2017 : 2554 - 2563 .

RAMALHO T , GARNELO M . Adaptive posterior learning: Few-shot learning with a surprise-based memory module [EB/OL ] . ( 2019-02-07 )[ 2024-03-02 ] . https://arxiv.org/abs/1902.02527 https://arxiv.org/abs/1902.02527 .

PARNAMI A , LEE M . Learning from few examples: A summary of approaches to few-shot learning [EB/OL ] . ( 2022-03-07 )[ 2024-03-02 ] . https://arxiv.org/abs/2203.04291 https://arxiv.org/abs/2203.04291 .

WANG W J , DUAN L J , WANG Y X , et al . Remember the difference: Cross-domain few-shot semantic segmentation via meta-memory transfer [C ] // 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2022 : 7055 - 7064 .

SNELL J , SWERSKY K , ZEMEL R . Prototypical networks for few-shot learning [C ] . Proceedings of the 31st International Conference on Neural Information Processing Systems . Red Hook : Curran Associates Inc. , 2017 : 4080 - 4090 .

GARCIA V , BRUNA J . Few-shot learning with graph neural networks [C ] // International Conference on Learning Representations . Vancouver : Openreview.net , 2018 .

TSENG H Y , LEE H Y , Huang J B , et al . Cross-domain few-shot classification via learned feature-wise transformation [C ] // International Conference on Learning Representations . Vancouver : Openreview.net , 2020 .

XIE J T , LONG F , LV J M , et al . Joint distribution matters: Deep Brownian distance covariance for few-shot classification [C ] // 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2022 : 7962 - 7971 .

LI W H , LIU X L , BILEN H . Cross-domain few-shot learning with task-specific adapters [C ] // 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2022 : 7151 - 7160 .

李维刚 , 甘平 , 谢璐 , 等 . 基于样本对元学习的小样本图像分类方法 [J ] . 电子学报 , 2022 , 50 ( 2 ): 295 - 304 .

LI W G , GAN P , XIE L , et al . A few-shot image classification method by pairwise-based meta learning [J ] . Acta Electronica Sinica , 2022 , 50 ( 2 ): 295 - 304 . (in Chinese)

LI Y H , LI H X , CHEN H X , et al . Hierarchical representation based query-specific prototypical network for few-shot image classification [EB/OL ] . ( 2021-03-21 )[ 2024-03-02 ] . http://arxiv.org/abs/2103.11384 http://arxiv.org/abs/2103.11384 .

LI W B , WANG L , XU J L , et al . Revisiting local descriptor based image-to-class measure for few-shot learning [C ] // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2019 : 7253 - 7260 .

SUNG F , YANG Y X , ZHANG L , et al . Learning to compare: Relation network for few-shot learning [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 1199 - 1208 .

CHENG J , HAO F S , LIU L , et al . Imposing semantic consistency of local descriptors for few-shot learning [J ] . IEEE Transactions on Image Processing , 2022 , 31 : 1587 - 1600 .

HAO F S , HE F X , CHENG J , et al . Collect and select: Semantic alignment metric learning for few-shot learning [C ] // 2019 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2019 : 8459 - 8468 .

DONG C Q , LI W B , HUO J , et al . Learning task-aware local representations for few-shot learning [C ] // Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence . California : Morgan Kaufmann , 2020 : 716 - 722 .

LI W B , XU J L , HUO J , et al . Distribution consistency based covariance metric networks for few-shot learning [C ] // Proceedings of the AAAI Conference on Artificial Intelligence . New York : ACM , 2019 , 33 ( 1 ): 8642 - 8649 .

LI H , YANG L , GAO F . More attentional local descriptors for few-shot learning [C ] //FARKAŠ I, MASULLI P, WERMTER S. International Conference on Artificial Neural Networks . Cham : Springer , 2020 : 419 - 430 .

QI Y , SUN H , LIU N Z , et al . A task-aware dual similarity network for fine-grained few-shot learning [C ] // Pacific Rim International Conference on Artificial Intelligence . Cham : Springer , 2022 : 606 - 618 .

HUANG H W , WU Z K , LI W B , et al . Local descriptor-based multi-prototype network for few-shot Learning [J ] . Pattern Recognition , 2021 , 116 : 107935 .

ZHENG Z J , FENG X , YU H Q , et al . BDLA: Bi-directional local alignment for few-shot learning [J ] . Applied Intelligence , 2023 , 53 ( 1 ): 769 - 785 .

ABDELAZIZ M , ZHANG Z P . Learn to aggregate global and local representations for few-shot learning [J ] . Multimedia Tools and Applications , 2023 , 82 ( 21 ): 32991 - 33014 .

齐妍 , 孙涵 . 基于判别性特征增强的小样本细粒度图像识别 [J ] . 计算机技术与发展 , 2024 , 34 ( 1 ): 44 - 51 .

QI Y , SUN H . Few-shot fine-grained image recognition based on discriminative feature enhancement [J ] . Computer Technology and Development , 2024 , 34 ( 1 ): 44 - 51 . (in Chinese)

CHEN T , KORNBLITH S , NOROUZI M , et al . A simple framework for contrastive learning of visual representations [C ] // Proceedings of the 37th International Conference on Machine Learning . New York : ACM , 2020 : 1597 - 1607 .

HE K M , FAN H Q , WU Y X , et al . Momentum contrast for unsupervised visual representation learning [C ] // 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2020 : 9726 - 9735 .

ORESHKIN B , RODRÍGUEZ LÓPEZ P , LACOSTE A . TADAM: Task dependent adaptive metric for improved few-shot learning [C ] . Proceedings of the 32nd International Conference on Neural Information Processing Systems . Red Hook : Curran Associates Inc. , 2018 : 719 - 729 .

WAH C , BRANSON S , WELINDER P , et al . The caltech-UCSD birds-200-2011 dataset [DB/OL ] . ( 2021 )[ 2024-03-02 ] . https://authors.library.caltech.edu/records/cvm3y-5hh21 https://authors.library.caltech.edu/records/cvm3y-5hh21 .

KHOSLA A , JAYADEVAPRAKASH N , YAO B P , et al . Novel dataset for fine-grained image categorization: Stanford dogs [C ] // Proceedings of CVPR Workshop on Fine-Grained Visual Categorization (FGVC) . Piscataway : IEEE , 2011 , 2 ( 1 ): 1 - 2 .

KRAUSE J , STARK M , JIA D , et al . 3D object representations for fine-grained categorization [C ] // 2013 IEEE International Conference on Computer Vision Workshops . Piscataway : IEEE , 2013 : 554 - 561 .

HILLIARD N , PHILLIPS L , HOWLAND S , et al . Few-shot learning with metric-agnostic conditional embeddings [EB/OL ] . ( 2018-02-12 )[ 2024-03-02 ] . https://arxiv.org/abs/1802.04376 https://arxiv.org/abs/1802.04376 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

一种基于LSB序列局部特征的通用隐写检测方法