Focus+Context语义表征的场景图像分割

doi:10.12263/DZXB.20200161

电子学报 ›› 2021, Vol. 49 ›› Issue (3): 596-604.DOI: 10.12263/DZXB.20200161

Focus+Context语义表征的场景图像分割

吴绿¹, 张馨月², 唐茉², 王梓³, 王永安⁴

1. 武汉理工大学宽带无线通信与传感器网络湖北省重点实验室, 湖北武汉 430070;
2. 武汉大学资源与环境科学学院, 湖北武汉 430079;
3. 中国人民解放军 95028 部队, 湖北武汉 430070;
4. 中国电子科技集团公司第五十四研究所, 河北石家庄 050081

收稿日期:2020-02-20 修回日期:2020-08-11 出版日期:2021-03-25
- 作者简介:
- 吴绿女,1983年1月出生,湖北潜江人.武汉理工大学通信与信息系统专业博士,现任武汉理工大学信息学院讲师.主要研究方向为机器视觉、模式识别.E-mail:wulv@whut.edu.cn;张馨月 女,1994年7月出生,吉林双辽人.武汉大学资源与环境科学学院博士研究生.
- 基金资助:
- "十三五"国家重点研发计划 (No.2017YFB0503500）

Focus+Context Semantic Representation in Scene Segmentation

WU Lü¹, ZHANG Xin-yue², TANG Mo², WANG Zi³, WANG Yong-an⁴

1. Key Laboratory of Broadband Wireless Communication and Sensor Networks, Wuhan University of Technology, Wuhan, Hubei 430070, China;
2. School of Resource and Environmental Sciences, Wuhan University, Wuhan, Hubei 430079, China;
3. A Unit of the Chinese People's Liberation Army, Wuhan, Hubei 430070, China;
4. The 54 th Research Institute of China Electronics Technology Group Corporation, Shijiazhuang, Hebei 050081, China

Received:2020-02-20 Revised:2020-08-11 Online:2021-03-25 Published:2021-03-25

摘要/Abstract

摘要： 场景图像分割一直是机器视觉学习中较为复杂的重难点问题.本文在机器视觉注意力机制学习方法的基础上，融合人类对事物个体的认知，提出场景对象的Focus+Context语义表征，将对象类别信息带入图像底层特征学习中，运用概率统计理论，在抽象层上建模局部区域对象，再联合上下文语义信息推理全局与局部区域对象之间的关系，以实现类内焦点对象（Focus）突出的场景语义分割.实验验证，基于Focus+Context的语义表征和建模能够增加对象的识别率，尤其是在小样本环境下，所提出的方法能极大地简化场景的理解.

关键词: 场景图像分割, Focus+Context, 语义表征, 主题模型

Abstract: Scene segmentation has always been a key and complicated problem in machine learning.In order to understand the scene and recognize the objects more accurately,this paper adopts human attention mechanism,takes the category semantic information into consideration and merges it into the image feature learning.The Focus+Context semantic representation is proposed,where the context describes the relationship between the focus and different objects in the scene,and the focus shared among the same category are composed of similar clusters.The probabilistic topic model is used to compute the local features as well as their semantic information.The experimental results show that the Focus+Context method increases the recognition rate of the scene objects,and specially,the proposed method,in a local and global understanding way,can simplify the scene recognition greatly under a small sample size.

Key words: scene segmentation, Focus+Context, semantic representation, topic model

中图分类号:

TP391.7

吴绿, 张馨月, 唐茉, 王梓, 王永安. Focus+Context语义表征的场景图像分割[J]. 电子学报, 2021, 49(3): 596-604.

WU Lü, ZHANG Xin-yue, TANG Mo, WANG Zi, WANG Yong-an. Focus+Context Semantic Representation in Scene Segmentation[J]. Acta Electronica Sinica, 2021, 49(3): 596-604.

参考文献

[1] BADUEA C,GUIDOLINIA R,RAPHAEL C,et al.Self-driving Cars:A Survey[OL].https://arxiv.org/abs/1901.04407v1,2019-01-14.
[2] PORCHERON M,FISCHER J E,REEVES S,et al.Voice interfaces in everyday life[A].Proceedings of the CHI Conference on Human Factors in Computing Systems[C].Montreal QC,Canada:ACM,2018.1-12.
[3] WANG Q,ZHANG L,BERTINETTO L.Fast online object tracking and segmentation:a unifying approach[A].2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition[C].Long Beach,USA:IEEE,2019.1328-1338.
[4] 田艳玲,张维桐,张锲石,等.图像场景分类技术综述[J].电子学报,2019,47(4):915-926. TIAN Yan-ling,ZHANG Wei-tong,ZHANG Qie-shi,et al.Review on image scene classification technology[J].Acta Electronica Sinica,2019,47(4):915-926.(in Chinese)
[5] 罗会兰,张云.基于深度网络的图像语义分割综述[J].电子学报,2019,47(10):2211-2220. LUO Hui-lan,ZHANG Yun.A survey of image semantic segmentation based on deep network[J].Acta Electronica Sinica,2019,47(10):2211-2220.(in Chinese)
[6] HOCK H S,GREGORY P G,WHITEHURST R.Contextual relations:The influence of familiarity,physical plausibility,and belongingness[J].Attention Perception & Psychophysics,1974,16(1):4-8.
[7] 钱晓亮,白臻,陈渊,等.协同视觉显著性检测方法综述[J].电子学报,2019,47(6):1352-1365. QIAN Xiao-liang,BAI Zhen,CHEN Yuan,et al.A review of co-saliency detection[J].Acta Electronica Sinica,2019,47(6):1352-1365.(in Chinese)
[8] LIU T,YUAN Z,SUN J,et al.Learning to detect a salient object[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2011,33(2):353-367.
[9] CHEN X W,ZHENG A L,LI J,et al.Look,perceive and segment:finding the salient objects in images via two-stream fixation semantic CNNs[A].IEEE International Conference on Computer Vision[C].Venice,Italy:IEEE,2017.1050-1058.
[10] WANG W G,SHEN J B,DONG X P,et al.Salient object detection driven by fixation prediction[A].2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition[C].Salt Lake City,Utah:IEEE,2018.1711-1720.
[11] CHEN S H,TAN X L,WANG B,et al.Reverse attention for salient object detection[A].2018 European Conference on Computer Vision[C].Munich,Germany:Springer,2018.236-252.
[12] ZHOU Y Z,SUN X Y,ZHA Z J,et al.Context-reinforced semantic segmentation[A].2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition[C].Long Beach,CA,USA:IEEE,2019.4041-4050.
[13] WANG T H.Context propagation from proposals for semantic video object segmentation[A].The 25th IEEE International Conference on Image Processing[C].Athens:IEEE,2018.256-260.
[14] MOTTAGHI R,CHEN X J,LIU X B,et al.The role of context for object detection and semantic segmentation in the wild[A].2014 IEEE Conference on Computer Vision and Pattern Recognition[C].Columbus,OH:IEEE,2014.891-898.
[15] BANSAL A,SHEIKHY,RAMANAN D.Shapes and context:In-the-wild image synthesis & manipulation[A].2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition[C].Long Beach,USA:IEEE,2019.2312-2321.
[16] 雷涛,张肖,加小红,等.基于模糊聚类的图像分割研究进展[J].电子学报,2019,47(8):1776-1791. LEI Tao,ZHANG Xiao,JIA Xiao-hong,et al.Research progress on image segmentation based on fuzzy clustering[J].Acta Electronica Sinica,2019,47(8):1776-1791.(in Chinese)
[17] ZHOU Z H.A brief introduction to weakly supervised learning[J].National Science Review,2018,5(1):44-53.
[18] 杨盛春.知识表征研究述评[J].图书情报导刊,2012,22(19):145-147. YANG Sheng-chun.A review of the research on knowledge tepresentation[J].Tech Information Development & Economy,2012,22(19):145-147.(in Chinese)
[19] BLEI D M,NG A Y,JORDAN M I,et al.Latent dirichlet allocation[J].Journal of Machine Learning Research,2003,3:993-1022.
[20] WU L,LIU Q.Weakly supervised object co-localization via zharing parts based on a joint bayesian model[J].Symmetry,2018,10(5):142.
[21] MATAS J,CHUMO,URBAN M,et al.Robust wide baseline stereo from maximally stable extremal regions[J].Image and Vision Computing,2004,22(10):761-767.
[22] VEDALDI A,FULKERSON B.An Open and Portable Library of Computer Vision Algorithms[OL].http://www.vlfeat.org/,2018.
[23] NIU Z X,HUA G,GAO X B,et al.Spatial-discLDA for visual recognition[A].2011 IEEE Conference on Computer Vision and Pattern Recognition[C].Providence,RI:IEEE,2011.1769-1776.
[24] PASCAL VOC 2012数据库[DB/OL].https://pjreddie.com/projects/pascal-voc-dataset-mirror/,2020.
[25] WANG C,BLEI D M,LI F F.Simultaneous image classification and annotation[A].2009 IEEE Conference on Computer Vision and Pattern Recognition[C].Miami,FL:IEEE,2009.1903-1910.
[26] WANG Y,Mori G.Max-margin latent dirichlet allocation for image classification and annotation[J].Lecture Notes in Computer Science,2011,1674(1):39-48.
[27] ZHENG Y,ZHANG Y J,LAROCHELLE H.Topic modeling of multimodal data:An autoregressive approach[A].2014 IEEE Conference on Computer Vision and Pattern Recognition[C].Columbus,OH:IEEE,2014.1370-1377.
[28] ZHENG Y,ZHANG Y J,LAROCHELLE H.A deep and autoregressive approach for topic modeling of multimodal data[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,38(6):1056-1069.
[29] LAROCHELLE H,LAULY S.A neural autoregressive topic model[A].Proceedings of the 25th International Conference on Neural Information Processing Systems-Volume 2[C].New York,USA:ACM,2012.2708-2716.

Focus+Context语义表征的场景图像分割

Focus+Context Semantic Representation in Scene Segmentation

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 11

编辑推荐

Metrics

[1]	仇韫琦, 王元卓, 白龙, 尹芷仪, 沈华伟, 白硕. 面向知识库问答的问句语义解析研究综述[J]. 电子学报, 2022, 50(9): 2242-2264.
[2]	唐焕玲, 郑涵, 刘艳红, 马思源, 窦全胜, 鲁明羽. Tr-SLDA:一种面向交叉领域的迁移主题模型[J]. 电子学报, 2021, 49(3): 605-613.
[3]	唐焕玲, 窦全胜, 于立萍, 宋英杰, 鲁明羽. 有监督主题模型的SLDA-TC文本分类新方法[J]. 电子学报, 2019, 47(6): 1300-1308.
[4]	张雄, 陈福才, 黄瑞阳. 基于双词主题模型的半监督实体消歧方法研究[J]. 电子学报, 2018, 46(3): 607-613.
[5]	孙小兵, 刘湘月, 李斌, 张伟佳. 基于相关主题模型的程序网络自动构建与分析[J]. 电子学报, 2017, 45(5): 1052-1056.
[6]	吕品, 汪鑫, 罗宜元, 计春雷. 基于主题模型的(Aspect,Rating)摘要生成方法研究[J]. 电子学报, 2016, 44(12): 3036-3043.
[7]	田刚, 何克清, 王健, 孙承爱, 徐建建. 面向领域标签辅助的服务聚类方法[J]. 电子学报, 2015, 43(7): 1266-1274.
[8]	丁宇新, 燕泽权, 冯威, 薛成龙, 周迪. 基于有监督主题模型的排序学习算法[J]. 电子学报, 2015, 43(2): 333-337.
[9]	刘杰, 杜军平. 基于潜在主题融合的跨媒体图像语义标注[J]. 电子学报, 2014, 42(5): 987-991.
[10]	李晓旭, 李睿凡, 冯方向, 曹洁, 王小捷. 多视图有监督的LDA模型[J]. 电子学报, 2014, 42(10): 2040-2044.
[11]	王李冬, 魏宝刚, 袁杰. 基于概率主题模型的文档聚类[J]. 电子学报, 2012, 40(11): 2346-2350.