基于改进的有效区域基因选择与跨模态语义挖掘的图像属性标注

doi:10.3969/j.issn.0372-2112.2020.04.021

电子学报 ›› 2020, Vol. 48 ›› Issue (4): 790-799.DOI: 10.3969/j.issn.0372-2112.2020.04.021

基于改进的有效区域基因选择与跨模态语义挖掘的图像属性标注

张红斌¹, 蒋子良¹, 熊其鹏¹, 武晋鹏¹, 邬任重¹, 袁天², 姬东鸿³

1. 华东交通大学软件学院, 江西南昌 330013;
2. 华东交通大学信息工程学院, 江西南昌 330013;
3. 武汉大学国家网络安全学院, 湖北武汉 430072

收稿日期:2018-07-02 修回日期:2019-12-20 出版日期:2020-04-25 发布日期:2020-04-25
作者简介:张红斌 男,1979年10月生,江苏如皋人.华东交通大学软件学院副教授、硕士生导师.主要研究方向:计算机视觉、自然语言处理、推荐系统.E-mail:zhanghongbin@whu.edu.cn;蒋子良 男,1997年4月生,江西婺源人.华东交通大学软件学院硕士研究生.主要研究方向:机器学习、图像材质识别.E-mail:ziiliangjiang@163.com
基金资助:
国家自然科学基金（No.61762038，No.61861016）；教育部人文社会科学研究规划基金项目（No.17YJAZH117）；江西省自然科学基金（No.20171BAB202023）；江西省科技厅重点研发计划（No.20171BBG70093，No.20192BBE50071）；江西省教育厅科技项目（No.GJJ190323）

Image Attribute Annotation via a Modified Effective Range Based Gene Selection and Cross-Modal Semantics Mining

ZHANG Hong-bin¹, JIANG Zi-liang¹, XIONG Qi-peng¹, WU Jin-peng¹, WU Ren-zhong¹, YUAN Tian², JI Dong-hong³

1. School of Software, East China Jiaotong University, Nanchang, Jiangxi 330013, China;
2. School of Information Engineering, East China Jiaotong University, Nanchang, Jiangxi 330013, China;
3. School of Cyber Science and Engineering, Wuhan University, Wuhan, Hubei 430072, China

Received:2018-07-02 Revised:2019-12-20 Online:2020-04-25 Published:2020-04-25

摘要/Abstract

摘要： 图像属性标注是一种更细化的图像标注，它能缩小认知与特征间"语义鸿沟".现有研究多基于单特征且未挖掘属性蕴含的深层语义，故无法准确刻画图像内容.改进有效区域基因选择算法融合图像特征，并设计迁移学习策略，实现材质属性标注；基于判别相关分析挖掘特征间跨模态语义，以改进相对属性模型，标注材质属性蕴含的深层语义-实用属性.实验表明：材质属性标注精准度达63.11%，较最强基线提升1.97%；实用属性标注精准度达59.15%，较最强基线提升2.85%；层次化的标注结果能全面刻画图像内容.

关键词: 图像标注, 有效区域基因选择, 相对属性, 迁移学习, 跨模态语义, 判别相关分析

Abstract: Image attribute annotation is a refined method of image annotation.It can narrow the "semantic gap" between cognition and features.However,a single feature is used to characterize images and the deep-level semantics are not fully explored.So annotations cannot depict images comprehensively.The traditional effective range based gene selection algorithm is modified to complete feature fusion.And transfer learning strategy is designed to complete material annotation.The cross-modal semantics among features are mined by the discriminant correlation analysis algorithm.So the relative attribute model is optimized to complete deep-level semantics (practical attributes) annotation.Experimental results demonstrate:Material attributes annotation accuracy reaches 63.11%,which is improved by 1.97% compared with baseline.Practical attributes annotation accuracy reaches 59.15%,which is improved by 2.85% compared with baseline.The proposed hierarchical annotation mechanism can more comprehensively depict images.

Key words: image annotation, effective range based gene selection, relative attribute, transfer learning, cross-modal semantics, discriminant correlation analysis

中图分类号:

TP391

张红斌, 蒋子良, 熊其鹏, 武晋鹏, 邬任重, 袁天, 姬东鸿. 基于改进的有效区域基因选择与跨模态语义挖掘的图像属性标注[J]. 电子学报, 2020, 48(4): 790-799.

ZHANG Hong-bin, JIANG Zi-liang, XIONG Qi-peng, WU Jin-peng, WU Ren-zhong, YUAN Tian, JI Dong-hong. Image Attribute Annotation via a Modified Effective Range Based Gene Selection and Cross-Modal Semantics Mining[J]. Acta Electronica Sinica, 2020, 48(4): 790-799.

参考文献

[1] 李倩玉,蒋建国,齐美彬,等.基于改进深层网络的人脸识别算法[J].电子学报,2017,45(3):619-625. LI Qian-yu,JIANG Jian-guo,QI Mei-bin et al.Face recognition algorithm based on improved deep networks[J].Acta Electronica Sinica,2017,45(3):619-625.(in Chinese)
[2] ZHU X Y,LIU H,LEI Z,et al.Large-scale bisample learning on id versus spot face recognition[J].International Journal of Computer Vision,2019,127(6-7):684-700.
[3] 周家琪,刘丽,崔晓萍,等.基于相似性迁移学习的图像标注[J].山东师范大学学报(自然科学版),2016,31(2):22-26. ZHOU Jia-qi,LIU Li,CUI Xiao-ping,et al.An image annotation method based on similarity transfer learning[J].Journal of ShanDong Normal University (Natural Science),2016,31(2):22-26.(in Chinese)
[4] Chen C Y,GRAUMAN K.Inferring analogous attributes[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)[C].Piscataway,NJ:IEEE,2014.200-207.
[5] JAYARAMAN D,GRAUMAN K.Zero-shot recognition with unreliable attributes[A].Proceedings of the Advances in Neural Information Processing Systems (NIPS)[C].New York:Curran Associates,2014.3464-3472.
[6] 乔雪,彭晨,段贺,张钰尧.基于共享特征相对属性的零样本图像标注[J].电子与信息学报,2017,39(7):1563-1570. QIAO Xue,PENG Chen,DUAN He,ZHANG Yu-yao.Shared features based relative attributes for zero-shot image classification[J].Journal of Electronics & Information Technology,2017,39(7):1563-1570.(in Chinese)
[7] 张峰,钟宝江.基于兴趣目标的图像检索[J].电子学报,2018,44(8):1915-1923. ZHANG Feng,ZHONG Bao-jiang.Image retrieval based on interested objects[J].Acta Electronica Sinica,2018,44(8):1915-1923.(in Chinese)
[8] ZHAO B,FENG J,WU X,et al.Memory-augmented attribute manipulation networks for interactive fashion search[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)[C].Piscataway,NJ:IEEE,2017.1520-1528.
[9] YU A,GRAUMAN K.Thinking outside the pool:active training image creation for relative attributes[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)[C].Piscataway,NJ:IEEE,2019.708-718.
[10] KOVASHKA A,PARIKH D,GRAUMAN K.WhittleSearch:interactive image search with relative attribute feedback[J].International Journal on Computer Vision,2015,115(2):185-210.
[11] YU A,GRAUMAN K.Just noticeable differences in visual attributes[A].Proceedings of the IEEE International Conference on Computer Vision (ICCV)[C].Piscataway,NJ:IEEE,2015.2416-2424.
[12] SINGH K K,LEE Y J.End-to-end localization and ranking for relative attributes[A].Proceedings of the European Conference on Computer Vision (ECCV)[C].Charm,Switzerland:Springer,2016.753-769.
[13] DUBEY A,AGARWAL S.Modeling image virality with pairwise spatial transformer networks[A].Proceedings of the 25th ACM International Conference on Multimedia (MM)[C].New York:ACM,2017.663-671.
[14] KRIZHEVSKY A,SUTSKEVER I,HINTON G E.ImageNet classification with deep convolutional neural networks[A].Proceedings of the Advances in Neural Information Processing Systems (NIPS)[C].New York:Curran Associates,2012.1097-1105.
[15] CHANDRA B,GUPTA M.An efficient statistical feature selection approach for classification of gene expression data[J].Journal of Biomedical Informatics,2011,44(4):529-535.
[16] 王晓梅,林晓惠,黄鑫.基于特征有效范围的前向特征选择及融合分类算法[J].小型微型计算机系统,2016,37(6):1159-1163. WANG Xiao-mei,LIN Xiao-hui,HUANG Xin.Algorithm of forward feature selection and aggregation of classifiers based on feature effective range[J].Journal of Chinese Computer Systems,2016,37(6):1159-1163.(in Chinese)
[17] SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-Scale Image Recognition[OL].https://arxiv.org/abs/1409.1556,2014/2019-12-18.
[18] HAGHIGHAT M,ABDEL-MOTTALEB M,ALHALABI W.Discriminant correlation analysis:real-time feature level fusion for multimodal biometric recognition[J].IEEE Transactions on Information Forensics and Security,2016,11(9):1984-1996.
[19] ZHANG H B,QIU D D,WU R Z,et al.Novel framework for image attribute annotation with gene selection XGBoost algorithm and relative attribute model[J].Applied Soft Computing,2019,80:57-79.
[20] 李航.统计学习方法[M].北京:清华大学出版社,2012.151-152.
[21] Chen T Q,Guestrin C.XGBoost:a scalable tree boosting system[A].Proceedings of ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD)[C].New York:ACM,2016.785-794.
[22] FARHADI A,ENDRES I,HOIEM D,et al.Describing objects by their attributes[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)[C].Piscataway,NJ:IEEE,2009.1778-1785.
[23] HAMKER F H.Predictions of a model of spatial attention using sum-and max-pooling functions[J].Neurocomputing,2018,56(1):329-343.
[24] WU L,WANG Y,LI X,et al.Deep attention-based spatially recursive networks for fine-grained visual recognition[J].IEEE Transactions on Cybernetics,2018,49(5):1791-1802.
[25] KARRAS T,AILA T,LAINE S,et al.Progressive Growing of Gans for Improved Quality,Stability,and Variation[OL].https://arxiv.org/abs/1710.10196,2017/2019-12-18.
[26] BROCK A,DONAHUE J,SIMONYAN K.Large Scale Gan Training for High Fidelity Natural Image Synthesis[OL].https://arxiv.org/abs/1809.11096,2018/2019-12-18.
[27] WANG J,YU L,ZHANG W,et al.IRGAN:a minimax game for unifying generative and discriminative information retrieval models[A].In Proceedings of ACM SIGIR[C].New York:ACM,2017.515-524.

基于改进的有效区域基因选择与跨模态语义挖掘的图像属性标注

Image Attribute Annotation via a Modified Effective Range Based Gene Selection and Cross-Modal Semantics Mining

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	李维刚, 甘平, 谢璐, 李松涛. 基于样本对元学习的小样本图像分类方法[J]. 电子学报, 2022, 50(2): 295-304.
[2]	张小恒, 张馨月, 李勇明, 王品, 刘玉川. 面向帕金森病语音诊断的非监督两步式卷积稀疏迁移学习算法[J]. 电子学报, 2022, 50(1): 177-184.
[3]	李勇斌, 王彬, 邵高平, 邵帅. 一种基于RCGAN的水声通信信号降噪方法[J]. 电子学报, 2022, 50(1): 54-62.
[4]	唐焕玲, 郑涵, 刘艳红, 马思源, 窦全胜, 鲁明羽. Tr-SLDA:一种面向交叉领域的迁移主题模型[J]. 电子学报, 2021, 49(3): 605-613.
[5]	史先进, 曹爽, 张重生, 陶月锋, 吕灵灵, 沈夏炯. 基于锚点的字符级甲骨图像自动标注算法研究[J]. 电子学报, 2021, 49(10): 2020-2031.
[6]	张锦, 李阳, 任传伦, 黄炼, 王帅辉, 段晔鑫, 潘志松, 谢钧. 基于帧间高级特征差分的跨场景视频前景分割算法[J]. 电子学报, 2021, 49(10): 2032-2040.
[7]	王格格, 郭涛, 余游, 苏菡. 基于生成对抗网络的无监督域适应分类模型[J]. 电子学报, 2020, 48(6): 1190-1197.
[8]	赵鹏, 王美玉, 纪霞, 刘慧婷. 基于张量表示的域适配的迁移学习中特征表示方法[J]. 电子学报, 2020, 48(2): 359-368.
[9]	权宇, 李志欣, 张灿龙, 马慧芳. 融合深度扩张网络和轻量化网络的目标检测模型[J]. 电子学报, 2020, 48(2): 390-397.
[10]	刘真, 田靖玉, 苑宝鑫, 孙永奇. 基于知识聚合和迁移的跨领域推荐算法[J]. 电子学报, 2020, 48(10): 1928-1932.
[11]	季鼎承, 蒋亦樟, 王士同. 基于域与样例平衡的多源迁移学习方法[J]. 电子学报, 2019, 47(3): 692-699.
[12]	余游, 冯林, 王格格, 徐其凤. 一种基于伪标签的半监督少样本学习模型[J]. 电子学报, 2019, 47(11): 2284-2291.
[13]	任卓君, 陈光, 卢文科. 基于N-gram特征的恶意代码可视化方法[J]. 电子学报, 2019, 47(10): 2108-2115.
[14]	胡根生, 查慧敏, 梁栋, 鲍文霞. 结合分类与迁移学习的薄云覆盖遥感图像地物信息恢复[J]. 电子学报, 2017, 45(12): 2855-2862.
[15]	韩敏, 杨雪. 改进贝叶斯ARTMAP的迁移学习遥感影像分类算法[J]. 电子学报, 2016, 44(9): 2248-2253.