尺度敏感损失与特征融合的快速小目标检测方法

doi:10.12263/DZXB.20210530

摘要/Abstract

摘要：

现有通用深度学习目标检测方法对中、大目标有着较好的检测精度，而对小目标检测精度较低，主要原因在于小目标训练数据少以及下采样后特征图分辨率过低.针对上述问题，一方面，提出一种尺度敏感损失函数用于分类热图的训练，使小目标能够对模型更新产生更大的影响；另一方面，利用反卷积与可变形卷积提出一种自上而下的特征融合方法，获得高分辨率、强语义的特征图来检测目标.在上述两个方面的基础上，设计一种尺度敏感与特征融合的小目标检测方法.在PASCAL VOC数据集上，对提出方法进行了实验验证，实验结果表明：相比于现有目标检测方法，本文方法在保持较快检测速度的同时，提升了小目标检测的精度.

关键词: 深度学习, 小目标检测, 尺度敏感损失, 特征融合

Abstract:

The existing general deep learning target detection methods have good detection accuracy for medium and large targets, but the detection accuracy for small targets is low, mainly due to few data for small target training and low resolution of feature map after down sampling. To solve the above problems, on the one hand, a scale sensitive loss function is proposed for the training of classified heatmaps, so that small targets can have a greater impact on model updating; on the other hand, a top-down feature fusion method is proposed by using deconvolution and deformable convolution to obtain high-resolution and strong semantic feature map for target detection. On the basis of the above two aspects, a small target detection method based on scale sensitivity and feature fusion is designed. Experimental results on PASCAL VOC dataset show that compared with the existing target detection methods, the proposed method can maintain a faster detection speed and improve the accuracy of small object detection.

Key words: deep learning, small object detection, scale-sensitivity loss, feature fusion

中图分类号:

TP391.4

琚长瑞, 秦晓燕, 袁广林, 李豪, 朱虹. 尺度敏感损失与特征融合的快速小目标检测方法[J]. 电子学报, 2022, 50(9): 2119-2126.

JU Chang-rui, QIN Xiao-yan, YUAN Guang-lin, LI Hao, ZHU Hong. Fast Small Object Detection Method with Scale-Sensitivity Loss and Feature Fusion[J]. Acta Electronica Sinica, 2022, 50(9): 2119-2126.

图/表 10

图1 PASCAL VOC数据图像(其中黄色框为标注目标,红色框内是未标注的小目标)

图 2 尺度敏感权重函数图像

图 3 特征融合模块

图 4 征融合网络结构

图 5 目标图像与特征图特征图特征图

表 1 不同k值的检测结果

k值	mAP	AP $50 : 95 l a r g e$	AP $50 : 95 m e d i u m$	AP $50 : 95 s m a l l$
k=1	77.1	80.4	75.6	25.5
k=2	75.5	77.5	73.5	28.6
k=3	73.4	75.3	70.6	29.1
k=4	70.2	73.3	69.1	29.3

表 1 不同k值的检测结果

k值	mAP	AP $50 : 95 l a r g e$	AP $50 : 95 m e d i u m$	AP $50 : 95 s m a l l$
k=1	77.1	80.4	75.6	25.5
k=2	75.5	77.5	73.5	28.6
k=3	73.4	75.3	70.6	29.1
k=4	70.2	73.3	69.1	29.3

表2 不同深度ResNet网络特征融合检测结果

骨干网络	mAP	AP $50 : 95 l a r g e$	AP $50 : 95 m e d i u m$	AP $50 : 95 s m a l l$	FPS
ResNet-18	73.2	75.0	72.5	26.9	142
ResNet-50	77.1	79.4	76.6	24.5	65
ResNet-101	80.5	82.4	78.5	23.4	45
FF-ResNet-18	65.1	67.9	60.2	27.6	122
FF-ResNet-50	76.8	77.1	76.4	27.9	50
FF-ResNet-101	78.5	79.0	77.5	28.8	29

表2 不同深度ResNet网络特征融合检测结果

骨干网络	mAP	AP $50 : 95 l a r g e$	AP $50 : 95 m e d i u m$	AP $50 : 95 s m a l l$	FPS
ResNet-18	73.2	75.0	72.5	26.9	142
ResNet-50	77.1	79.4	76.6	24.5	65
ResNet-101	80.5	82.4	78.5	23.4	45
FF-ResNet-18	65.1	67.9	60.2	27.6	122
FF-ResNet-50	76.8	77.1	76.4	27.9	50
FF-ResNet-101	78.5	79.0	77.5	28.8	29

表 3 综合影响实验结果

骨干网络	k	mAP	AP $50 : 95 l a r g e$	AP $50 : 95 m e d i u m$	AP $50 : 95 s m a l l$
FF-ResNet-18	2	64.9	67.2	60.0	28.3
FF-ResNet-50	2	76.5	76.9	74.5	30.6
FF-ResNet-101	2	77.4	78.3	74.6	31.4

表 3 综合影响实验结果

骨干网络	k	mAP	AP $50 : 95 l a r g e$	AP $50 : 95 m e d i u m$	AP $50 : 95 s m a l l$
FF-ResNet-18	2	64.9	67.2	60.0	28.3
FF-ResNet-50	2	76.5	76.9	74.5	30.6
FF-ResNet-101	2	77.4	78.3	74.6	31.4

表 4 本文方法与各类方法对比

方法类型	方法	网络结构	mAP	AP $50 : 95 s m a l l$	FPS
两阶段方法	Faster-RCNN^[20]	ResNet-50	74.5	24.3	15
	Cascade RCNN^[21]	ResNet-50	75.6	26.2	17
	R-FCN^[22]	ResNet-50	78.4	24.5	12
	FPN^[9]	ResNet-50	77.5	29.4	9
	CenterNet2^[26]	ResNet-50	80.3	29.1	27
单阶段方法	SSD321^[7]	ResNet-50	77.2	27.5	23
	SSD513^[7]	VGG16	78.8	26.6	33
	DSSD^[24]	ResNet-50	78.4	27.1	8
	YOLOv4^[23]	DarkNet-53	79.5	29.8	52
	YOLOv5	YOLOv5-l	78.3	23.9	121
	CornerNet^[25]	ResNet-50	75.3	23.6	26
	CenterNet^[18]	ResNet-50	77.1	25.5	65
	SSLFF-Det	ResNet-50	76.5	30.6	50

表 4 本文方法与各类方法对比

方法类型	方法	网络结构	mAP	AP $50 : 95 s m a l l$	FPS
两阶段方法	Faster-RCNN^[20]	ResNet-50	74.5	24.3	15
	Cascade RCNN^[21]	ResNet-50	75.6	26.2	17
	R-FCN^[22]	ResNet-50	78.4	24.5	12
	FPN^[9]	ResNet-50	77.5	29.4	9
	CenterNet2^[26]	ResNet-50	80.3	29.1	27
单阶段方法	SSD321^[7]	ResNet-50	77.2	27.5	23
	SSD513^[7]	VGG16	78.8	26.6	33
	DSSD^[24]	ResNet-50	78.4	27.1	8
	YOLOv4^[23]	DarkNet-53	79.5	29.8	52
	YOLOv5	YOLOv5-l	78.3	23.9	121
	CornerNet^[25]	ResNet-50	75.3	23.6	26
	CenterNet^[18]	ResNet-50	77.1	25.5	65
	SSLFF-Det	ResNet-50	76.5	30.6	50

图6 不同检测方法检测结果

参考文献 26

1	WANG X J, ZHANG S L, YU Z R, et al. Scale-equalizing pyramid convolution for object detection[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE, 2020: 13356-13365.
2	GUO C X, FAN B, ZHANG Q, et al. AugFPN: improving multi-scale feature learning for object detection[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE, 2020: 12592-12601.
3	TAN M X, PANG R M, LE Q V. EfficientDet: scalable and efficient object detection[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2020: 10778-10787.
4	刘颖, 刘红燕, 范九伦, 等. 基于深度学习的小目标检测研究与应用综述[J]. 电子学报, 2020, 48(3): 590-601.
	LIU Y, LIU H Y, FAN J L, et al. A survey of research and application of small object detection based on deep learning[J]. Acta Electronica Sinica, 2020, 48(3): 590-601. (in Chinese)
5	TONG K, WU Y Q, ZHOU F. Recent advances in small object detection based on deep learning: A review[J]. Image and Vision Computing, 2020, 97: 103910.
6	YANG F, CHOI W, LIN Y Q. Exploit all the layers: Fast and accurate CNN object detector with scale dependent pooling and cascaded rejection classifiers[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 2129-2137.
7	LIU W, ANGUELOV D, ERHAN D, et al. SSD: Single Shot MultiBox Detector[C]//European Conference on Computer Vision. Cham: Springer, 2016: 21-37.
8	CAI Z W, FAN Q F, FERIS R S, et al. A unified multi-scale deep convolutional neural network for fast object detection[C]//European Conference on Computer Vision. Cham: Springer, 2016: 354-370.
9	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 936-944.
10	LIANG Z W, SHAO J, ZHANG D Y, et al. Small object detection using deep feature pyramid networks[C]//Pacific Rim Conference on Multimedia. Cham: Springer, 2018: 554-564.
11	GHIASI G, LIN T Y, LE Q V. NAS-FPN: Learning scalable feature pyramid architecture for object detection[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2019: 7029-7038.
12	WU J X, LIU S T, HUANG D, et al. Multi-Scale Positive Sample Refinement for Few-Shot Object Detection[C]//Computer Vision – ECCV 2020. Cham: Springer International Publishing, 2020: 456-472.
13	李雅倩, 盖成远, 肖存军, 等. 基于细化多尺度深度特征的目标检测网络[J]. 电子学报, 2020, 48(12): 2360-2366.
	LI Y Q, GAI C Y, XIAO C J, et al. Object detection networks based on refined multi-scale depth feature[J]. Acta Electronica Sinica, 2020, 48(12): 2360-2366. (in Chinese)
14	ZHANG S F, ZHU X Y, LEI Z, et al. S3FD: single shot scale-invariant face detector[C]//2017 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017: 192-201.
15	KISANTAL M, WOJNA Z, MURAWSKI J, et al. Augmentation for small object detection[EB/OL]. (2019-02-19). .
16	CHEN Y K, ZHANG P Z, LI Z M, et al. Dynamic scale training for object detection[EB/OL]. (2020-04-26). .
17	李宝奇, 贺昱曜, 强伟, 等. 基于并行附加特征提取网络的SSD地面小目标检测模型[J]. 电子学报, 2020, 48(1): 84-91.
	LI B Q, HE Y Y, QIANG W, et al. SSD with parallel additional feature extraction network for ground small target detection[J]. Acta Electronica Sinica, 2020, 48(1): 84-91. (in Chinese)
18	ZHOU X Y, WANG D Q, Philipp Krähenbühl,et al.Objects as Points [EB/OL]. (2019-04-16). .
19	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//2017 IEEE International Conference on Computer Vision(ICCV). Piscataway: IEEE, 2017: 2999-3007.
20	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017,39(6): 1137-1149.
21	CAI Z W, VASCONCELOS N. Cascade R-CNN: Delving into high quality object detection[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 6154-6162.
22	DAI J, LI Y, HE K, et al. R-FCN: object detection via region-based fully convolutional networks[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc, 2016: 379-387.
23	BOCHKOVSKIY A, WANG C Y, LIAO H, et al. YOLOv4: Optimal speed and accuracy of object detection[EB/OL]. (2020-04-23). .
24	FU C Y, LIU W, RANGA A, et al. Dssd: Deconvolutional single shot detector[EB/OL]. (2017-01-23). .
25	LAW H, DENG J. CornerNet: Detecting objects as paired keypoints[C]//Computer Vision – ECCV 2018. Cham: Springer International Publishing, 2018: 765-781.
26	ZHOU X Y, KOLTUN V, KRÄHENBÜHL P, et al. Probabilistic two-stage detection[EB/OL]. (2021-03-12). .

[1]	吴靖, 叶晓晶, 黄峰, 陈丽琼, 王志锋, 刘文犀. 基于深度学习的单帧图像超分辨率重建综述[J]. 电子学报, 2022, 50(9): 2265-2294.
[2]	李雪莹, 王田路, 梁鹏, 王翀. 基于系统模型的用户评论中非功能需求的自动分类[J]. 电子学报, 2022, 50(9): 2079-2089.
[3]	张志昌, 于沛霖, 庞雅丽, 朱林, 曾扬扬. SMGN：用于对话状态跟踪的状态记忆图网络[J]. 电子学报, 2022, 50(8): 1851-1858.
[4]	张亚洲, 俞洋, 朱少林, 陈锐, 戎璐, 梁辉. 一种量子概率启发的对话讽刺识别网络模型[J]. 电子学报, 2022, 50(8): 1885-1893.
[5]	王飞扬, 冀鹏欣, 孙笠, 危倩, 李根, 张忠宝. 一种基于深度学习的动态社交网络用户对齐方法[J]. 电子学报, 2022, 50(8): 1925-1936.
[6]	徐兴荣, 刘聪, 李婷, 郭娜, 任崇广, 曾庆田. 基于双向准循环神经网络和注意力机制的业务流程剩余时间预测方法[J]. 电子学报, 2022, 50(8): 1975-1984.
[7]	李超, 黄新宇, 王凯. 基于特征融合和自学习锚框的高分辨率图像小目标检测算法[J]. 电子学报, 2022, 50(7): 1684-1695.
[8]	欧阳与点, 谢鲲, 谢高岗, 文吉刚. 面向大规模网络测量的数据恢复算法：基于关联学习的张量填充[J]. 电子学报, 2022, 50(7): 1653-1663.
[9]	裴炤, 邱文涛, 王淼, 马苗, 张艳宁. 基于Transformer动态场景信息生成对抗网络的行人轨迹预测方法[J]. 电子学报, 2022, 50(7): 1537-1547.
[10]	彭闯, 王伦文, 胡炜林. 融合深度特征的电磁频谱异常检测算法[J]. 电子学报, 2022, 50(6): 1359-1369.
[11]	杨伟超, 杜宇, 文伟, 侯舒维, 徐常志, 张建华. 基于多重分形谱智能分析的卫星信号调制识别研究[J]. 电子学报, 2022, 50(6): 1336-1343.
[12]	李政伟, 李佳树, 尤著宏, 聂茹, 赵欢, 钟堂波. 基于异质图注意力网络的miRNA与疾病关联预测算法[J]. 电子学报, 2022, 50(6): 1428-1435.
[13]	张波, 陆云杰, 秦东明, 邹国建. 一种卷积自编码深度学习的空气污染多站点联合预测模型[J]. 电子学报, 2022, 50(6): 1410-1427.
[14]	廖勇, 李玉杰. 一种轻量化低复杂度的FDD大规模MIMO系统CSI反馈方法[J]. 电子学报, 2022, 50(5): 1211-1217.
[15]	冀振燕, 韩梦豪, 宋晓军, 冯其波. 面向激光光条图像修复的循环相似度映射网络[J]. 电子学报, 2022, 50(5): 1234-1242.