基于深度学习的小目标检测研究与应用综述

doi:10.3969/j.issn.0372-2112.2020.03.024

电子学报 ›› 2020, Vol. 48 ›› Issue (3): 590-601.DOI: 10.3969/j.issn.0372-2112.2020.03.024

基于深度学习的小目标检测研究与应用综述

刘颖^1,2,3, 刘红燕³, 范九伦³, 公衍超^1,3, 李莹华³, 王富平^1,3, 卢津^1,3

1. 电子信息现场勘验应用技术公安部重点实验室, 陕西西安 710121;
2. 陕西省无线通信与信息处理技术国际合作研究中心, 陕西西安 710121;
3. 西安邮电大学图像与信息处理研究所, 陕西西安 710121

收稿日期:2019-09-29 修回日期:2019-11-20 出版日期:2020-03-25
- 通讯作者:
- 刘颖
- 作者简介:
- 刘红燕 女,主要研究方向为小目标检测.E-mail:hy_liu1314@163.com;范九伦 男,主要研究方向为模糊信息处理、图像处理、信息安全等.E-mail:jiulunf@xupt.edu.cn;公衍超 男,主要研究方向为视频编码及码率控制等.E-mail:gongyanchao@xupt.edu.cn;李莹华 女,主要研究方向为图像安全与图像超分辨率重建等.E-mail:liyinghua@xupt.edu.cn;王富平 男,主要研究方向为图像特征提取及图像检索等.E-mail:wfp1608@163.com;卢津男,主要研究方向为三维痕迹图像数据采集及处理等.E-mail:lujin@xupt.edu.cn
- 基金资助:
- 陕西省国际合作交流项目 (No.2018KW-003）; 国家自然科学基金 (No.61671377）; 国家重点研发计划 (No.2017YFC0803805）

A Survey of Research and Application of Small Object Detection Based on Deep Learning

LIU Ying^1,2,3, LIU Hong-yan³, FAN Jiu-lun³, GONG Yan-chao^1,3, LI Ying-hua³, WANG Fu-ping^1,3, LU Jin^1,3

1. Key Laboratory of Electronic Information Application Technology for Scene Investigation, Ministry of Public Security, Xi'an, Shaanxi 710121, China;
2. International Joint Research Center for Wireless Communication and Information Processing, Shaanxi, Xi'an, Shaanxi 710121, China;
3. Center for Image and Information Processing, Xi'an University of Posts and Telecommunications, Xi'an, Shaanxi 710121, China

Received:2019-09-29 Revised:2019-11-20 Online:2020-03-25 Published:2020-03-25
- Corresponding author:
- LIU Ying
- Supported by:
- Project of International Coorperation and Exchanges of Shaanxi Province (No.2018KW-003); National Natural Science Foundation of China (No.61671377); National Key Research and Development Program of China (No.2017YFC0803805)

摘要/Abstract

摘要： 目标检测在基于传统手工特征及深度学习算法上已经取得较大发展，然而针对小目标检测的研究近几年才开始出现，研究成果较少，且大都是在已有目标检测算法基础上进行改进，以提高小目标检测的检测精度.小目标像素点少，本身携带的特征少，多次下采样后就更难进行特征提取，因而小目标检测面临极大挑战.小目标检测在自动驾驶、遥感图像检测、刑侦等领域都有广泛应用需求，对于小目标检测技术的研究有重要的实用价值.本文对小目标检测的现有研究成果进行了详细综述.首先，将现有算法按照检测需要的阶段数分为一阶段、二阶段、多阶段，描述了RetinaNet、CornerNet-Lite、特征金字塔网络（Feature Pyramid Network，FPN）等算法的原理并进行了对比分析.其次，本文描述了小目标检测技术在不同领域的应用情况，并汇总了MS COCO、PASCAL VOC、DOTA、KITTI等数据集及算法性能评价指标.最后，总结了小目标检测面临的挑战，并展望了未来的研究方向.

关键词: 小目标检测, 尺度变换, 特征金字塔, 深度学习, 特征提取, 卷积神经网络

Abstract: Object detection has been well studied based on the traditional manual features and deep learning algorithm. However, the research on small object detection has just begun in recent years and there are little research outcome available. Furthermore, most of the methods proposed are based on the traditional object detection algorithms with certain modifications so as to improve the accuracy of small object detection. Small object contains fewer pixels and has less features, and it is even harder to extract object features after down sampling. Hence, small object detection is a challenging task. Small object detection has a wide range of application requirements in the fields of automatic driving, remote sensing image detection, and criminal investigation. It has important practical value for the research of small object detection technology. In this paper, the existing research results of small object detection are summarized. Firstly, the existing algorithms are classified into one stage, two stages and multi-stages according to the number of stages for detection. The principles of RetinaNet、CornerNet-Lite、feature pyramid network (FPN) and other algorithms are described and compared. Secondly, this paper describes the application of small object detection technology in different fields, and summarizes the data sets such as MS COCO、PASCAL VOC、DOTA、KITTI and algorithm performance evaluation indicators. Finally, the challenges faced by small object detection are concluded, and the future research directions are prospected.

Key words: small object detection, scale variation, feature pyramid, deep learning, feature extraction, convolutional neural network

中图分类号:

刘颖, 刘红燕, 范九伦, 等. 基于深度学习的小目标检测研究与应用综述[J]. 电子学报, 2020, 48(3): 590-601.

LIU Ying, LIU Hong-yan, FAN Jiu-lun, et al. A Survey of Research and Application of Small Object Detection Based on Deep Learning[J]. Acta Electronica Sinica, 2020, 48(3): 590-601.

参考文献

[1] KISANTAL M,WOJNA Z,MURAWSKI J,et al.Augmentation for Small Object Detection[DB/OL].arXiv:1902.07296v1,2019.
[2] wq604887956.小目标检测文章总结[EB/OL].https://blog.csdn.net/wq604887956/article/details/83053927,2018-10-16/2019-09-02.
[3] LIN T Y,MAIRE M,BELONGIE S,et al.Microsoft COCO:Common objects in context[A].The 13th European Conference on Computer Vision(ECCV)[C].Cham:Springer,2014.740-755.
[4] BODLA N,SINGH B,CHELLAPPA R,et al.Soft-NMS-Improving object detection with one line of code[A].2017 IEEE International Conference on Computer Vision[C].New York,USA:IEEE,2017.5561-5569.
[5] LI Y H,CHEN Y T,WANG N Y,et al.Scale-aware Trident Networks for Object Detection[DB/OL].arXiv:1901.01892,2019.
[6] 王海屹.基于特征融合的小物体目标检测研究与应用[D].北京:北京邮电大学,2018.
[7] EGGERT C,BREHM S,WINSCHEL A,et al.A closer look:Small object detection in faster R-CNN[A].2017 IEEE International Conference on Multimedia and Expo (ICME)[C].Hong Kong,China:IEEE Computer Society,2017.421-426.
[8] 刘晓楠,王正平,贺云涛,等.基于深度学习的小目标检测研究综述[J].战术导弹技术,2019,193(01):106-113. LIU X N,WANG Z P,HE Y T,et al.Review of small object detection based on deep learning[J].Tactical Missile Technology,2019,193(01):106-113.(in Chinese)
[9] TAKEKI A,Tu T T,YOSHIHASHI R,et al. Combining deep features for object detection at various scales:finding small birds in landscape images[J].IPSJ Transactions on Computer Vision & Applications,2016,8(1):5.
[10] KAMPFFMEYER M,Salberg A B,Jenssen R.Semantic segmentation of small objects and modeling of uncertainty in urban remote sensing images using deep convolutional neural networks[A].2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops[C].Las Vegas,NV:IEEE,2016.680-688.
[11] MUNDHENK T N,Konjevod G,Sakla W A,et al.A large contextual dataset for classification detection and counting of cars with deep learning[A].The 14th European Conference on Computer Vision[C].Cham:Springer,2016.785-800.
[12] RENY,ZHU C R,XIAO S P.Small object detection in optical remote sensing images via modified faster R-CNN[J].Applied Sciences,2018,8(5):813.
[13] EGGERT C,ZECHA D,BREHM S,et al.Improvingsmall object proposals for company logo detection[A].Proceedings of The 2017 ACM on International Conference on Multimedia Retrieval[C].New York,USA:ACM,2017.167-174.
[14] 于进勇,丁鹏程,王超.卷积神经网络在目标检测中的应用综述[J].计算机科学,2018,45(S2):27-36. YU J Y,DING P C,WANG C.Summary of application of convolutional neural network in object detection[J].Computer Science,2018,45(S2):27-36.(in Chinese)
[15] HARIHARAN B,Arbeláez P,GIRSHICK R,et al.Hypercolumns forobject segmentation and fine-grained localization[A].2014 IEEE Conference on Computer Vision and Pattern Recognition[C].Boston,MA,USA:IEEE,2014.447-456.
[16] 李旭冬,叶茂,李涛.基于卷积神经网络的目标检测研究综述[J].计算机应用研究,2017,34(10):2881-2886. LI X D,YE M,LI T.Summary of research on object detection based on convolutional neural network[J].Computer Applied Research,2017,34(10):2881-2886.(in Chinese)
[17] 周俊宇,赵艳明.卷积神经网络在图像分类和目标检测应用综述[J].计算机工程与应用,2017,53(13):34-41. ZHOU J Y,ZHAO Y M.Summary ofapplication of convolutional neural network in image classification and object detection[J].Computer Engineering and Application,2017,53(13):34-41.(in Chinese)
[18] 崔广新,李殿奎.基于自编码算法的深度学习综述[J].计算机系统应用,2018,27(09):47-51. CUI G X,LI D K.A survey of deep learning based on self coding algorithm[J].Computer System Application,2018,27(09):47-51.(in Chinese)
[19] 徐毅,李蓓蓓,宋威.改进的深度置信网络分类算法研究[J].计算机科学与探索,2019,13(04):596-607. XU Y,LI B B,SONG W.Research on improved depth confidence network classification algorithm[J].Computer Science and Exploration,2019,13(04):596-607.(in Chinese)
[20] HE K M,ZHANG X Y,REN S Q,et al.Deep residual learning for image recognition[A].Proceedings of The 2016 IEEE Conference on Computer Vision and Pattern Recognition[C].USA:IEEE Computer Society,2016.770-778.
[21] NEWELL A,YANG K,DENG J.Stacked hourglass networks for human pose estimation[A].The 14th European Conference on Computer Vision[C].Cham:Springer,2016.483-499.
[22] REDMON J,DIVVALA S,and GIRSHICK R,et al.You only look once:unified,real-time object detection[A].2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)[C].Las Vegas,NV:IEEE,2016.779-788.
[23] LIU W,ANGUELOV D,ERHAN D,et al.SSD:Single shot multiBox detector[A].The 14th European Conference on Computer Vision[C].Cham:Springer,2016.21-37.
[24] FU C Y,LIU W,RANGA A,et al.DSSD:Deconvolutional Single Shot Detector[DB/OL].arXiv:1701.06659,2017.
[25] LIN T Y,GOYAL P,GIRSHICK R,et al.Focalloss for dense object detection[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2017,PP(99):2999-3007.
[26] LAW H,TENG Y,RUSSAKOVSKY O,et al.CornerNet-Lite:Efficient Keypoint Based Object Detection[DB/OL].arXiv:1904.08900,2019.
[27] LIN T Y,Dollár,and PIOTR,et al.Feature pyramid networks for object detection[A].2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)[C].Honolulu,HI:IEEE Computer Society,2017.2117-2125.
[28] HUANG J,RATHOD V,SUN C,et al.Speed/accuracy trade-offs for modern convolutional object detectors[A].2016 IEEE Conference on Computer Vision and Pattern Recognition[C].New York,USA:IEEE,2016.3296-3297.
[29] DAI J F,QI H Z,XIONG Y W,et al.Deformable Convolutional Networks[DB/OL].arXiv:1703.06211,2017.
[30] HE K M,GKIOXARI G,Dollár P,et al.Mask R-CNN[A].2017 IEEE International Conference on Computer Vision[C].Venice:IEEE Computer Society,2017.2980-2988.
[31] LI Z M,PENG C,YU G,et al.DetNet:A Backbone Network for Object Detection[DB/OL].arXiv:1804.06215,2018.
[32] SINGH B and DAVIS L S.An analysis of scale invariance in object detection-SNIP[A].2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)[C].New York,USA:IEEE Computer Society,2018.3578-3587.
[33] SINGH B,NAJIBI M,DAVIS L S.SNIPER:Efficientmulti-scale Training[DB/OL].arXiv:1805.09300,2018.
[34] CAI Z W,VASCONCELOS N.Cascade R-CNN:Delving into high quality object detection[A].2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)[C].New York,USA:IEEE Computer Society,2018.6154-6162.
[35] CAI Z W,VASCONCELOS N.Cascade R-CNN:High Quality Object Detection and Instance Segmentation[DB/OL].arXiv:1906.09756,2019.
[36] 张亚须.视频目标检测与跟踪算法研究及其在公安刑侦系统中的应用[D].云南:云南师范大学,2018.
[37] KRISHNA H,JAWAHAR C V.Improving small object detection[A].20174th IAPR Asian Conference on Pattern Recognition[C].Nanjing:Nanjing University of Science and Technology (NJUST),2017.340-345.
[38] HU P Y,RAMANAN D.Finding tiny faces[A].2017 IEEE Conference on Computer Vision and Pattern Recognition[C].Honolulu,HI:IEEE,2017.1522-1530.
[39] LIU W,LIAO S C,HU W D,et al.Learning efficient single-stage pedestrian detectors by asymptotic localization fitting[A].European Conference on Computer Vision[C].Munich,Germany:Springer,2018.618-634.
[40] cLI J N,LIANG X D,WEI Y C,et al.Perceptual generative adversarial networks for small object detection[A].2017 IEEE Conference on Computer Vision and Pattern Recognition[C].New York,USA:IEEE,2017.1951-1959.
[41] YI K,JIAN Z Q,CHEN S T,et al.Knowledge-based Recurrent Attentive Neural Network for Traffic Sign Detection[DB/OL].arXiv:1803.05263,2018.
[42] 西安邮电大学图像处理团队小目标检测图像数据库[DB/OL].http://www.xuptciip.com.cn/show.html?database-xksjk8,2018-11-01/2019-09-02.
[43] EVERINGHAM M,WINN J.The PASCALvisual object classes challenge 2007(VOC2007) development kit[J].International Journal of Computer Vision,2006,111(1):98-136.
[44] XIA G S,BAI X,DING J,et al.DOTA:A Large-scale Dataset for Object Detection in Aerial Images[DB/OL].arXiv:1711.10398,2017.
[45] ZHU H G,CHEN X G,DAI W Q,et al.Orientation robust object detection in aerial images using deep convolutional neural network[A].2015 IEEE International Conference on Image Processing (ICIP)[C].Quebec City,QC,Canada:IEEE,2015.3735-3739.
[46] CHENG G,HAN J W,ZHOU P C,et al.Multi-class geospatial object detection and geographic image classification based on collection of part detectors[J].ISPRS Journal of Photogrammetry and Remote Sensing,2014,98(1):119-132.
[47] XIAO Z F,LIU Q,TANG G F,et al.Elliptic fourier transformation-based histograms of oriented gradients for rotationally invariant object detection in remote-sensing images[J].International Journal of Remote Sensing,2015,36(2):618-644.
[48] ZHU Z,LIANG D,ZHANG S H,et al.Traffic-sign detection and classification in the wild[A].2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)[C].Las Vegas,NV,USA:IEEE Computer Society,2016.2110-2118.
[49] LWJjing.Kitti数据集简介[EB/OL].https://blog.csdn.net/weixin_43229119/article/details/89927707,2019-05-07/2019-09-02.
[50] ROMBERG S,PUEYO L G,LIENHART R,et al.Scalable logo recognition in real-world images[A].Proceedings of the 1st International Conference on Multimedia Retrieval(ICMR)[C].New York,USA:ACM,2011.25.
[51] BELL S,ZITNICK C L,BALA K,et al.Inside-outside net:Detecting objects in context with skip pooling and recurrent neural networks[A].2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)[C].Las Vegas,NV,USA:IEEE,2016.2874-2883.
[52] LONG J,SHELHAMER E,DARRELL T.Fullyconvolutional networks for semantic segmentation[J].2014 IEEE Transactions on Pattern Analysis & Machine Intelligence,2014,39(4):640-651.
[53] SHEN Z Q,SHI H H,YU J H,et al.Learning Object Detectors from Scratch with Gated Recurrent Feature Pyramids[DB/OL].arXiv:1712.00886,2017.
[54] YU F,KOLUN V.Multi-scale Context Aggregation by Dilated Convolutions[DB/OL].arXiv:1511.07122,2015.
[55] ZOPH B,CUBUK E D,GHIASI G,et al.Learning Data Augmentation Strategies for Object Detection[DB/OL].arXiv:1906.11172,2019.
[56] KHOREVA A,BENENSON R,ILG E,et al.Lucid data dreaming for video object segmentation[J].International Journal of Computer Vision,2019,127(9):1175-1197.

基于深度学习的小目标检测研究与应用综述

A Survey of Research and Application of Small Object Detection Based on Deep Learning

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	李钦, 刘伟, 牛朝阳, 宝音图, 惠周勃. 低信噪比下基于分裂EfficientNet网络的雷达信号调制方式识别[J]. 电子学报, 2023, 51(3): 675-686.
[2]	黄赟, 张帆, 郭威, 陈立, 羊光. 一种基于数据标准差的卷积神经网络量化方法[J]. 电子学报, 2023, 51(3): 639-647.
[3]	范兵兵, 何庭建, 张聪炫, 陈震, 黎明. 联合遮挡约束与残差补偿的特征金字塔光流计算方法[J]. 电子学报, 2023, 51(3): 648-657.
[4]	吕杭, 蒋明峰, 李杨, 张鞠成, 王志康. 基于混合时频域特征的卷积神经网络心律失常分类方法的研究[J]. 电子学报, 2023, 51(3): 701-711.
[5]	张聿远, 闫文君, 张立民. 基于多模态特征融合网络的空时分组码识别算法[J]. 电子学报, 2023, 51(2): 489-498.
[6]	许新征, 李杉. 基于特征膨胀卷积模块的轻量化技术研究[J]. 电子学报, 2023, 51(2): 355-364.
[7]	丁琪, 田萱, 孙国栋. 基于注意力增强的热点感知新闻推荐模型[J]. 电子学报, 2023, 51(1): 93-104.
[8]	张永梅, 孙捷. 基于动静态特征双输入神经网络的咳嗽声诊断COVID-19算法[J]. 电子学报, 2023, 51(1): 202-212.
[9]	李滔, 董秀成, 林宏伟. 基于深监督跨尺度注意力网络的深度图像超分辨率重建[J]. 电子学报, 2023, 51(1): 128-138.
[10]	郭晓轩, 冯其波, 冀振燕, 郑发家, 杨燕燕. 多线激光光条图像缺陷分割模型研究[J]. 电子学报, 2023, 51(1): 172-179.
[11]	贾童瑶, 卓力, 李嘉锋, 张菁. 基于深度学习的单幅图像去雾研究进展[J]. 电子学报, 2023, 51(1): 231-245.
[12]	柳长源, 张玉亮, 毕晓君. 基于多阶段提议稀疏区域卷积网络的城市交通目标检测[J]. 电子学报, 2023, 51(1): 26-31.
[13]	何滢婕, 刘月峰, 边浩东, 郭威, 张小燕. 基于Informer的电池荷电状态估算及其稀疏优化方法[J]. 电子学报, 2023, 51(1): 50-56.
[14]	袁海英, 成君鹏, 曾智勇, 武延瑞. Mobile_BLNet：基于Big-Little Net的轻量级卷积神经网络优化设计[J]. 电子学报, 2023, 51(1): 180-191.
[15]	王神龙, 雍宇, 吴晨睿. 基于伪孪生神经网络的低纹理工业零件6D位姿估计[J]. 电子学报, 2023, 51(1): 192-201.