电子学报 ›› 2022, Vol. 50 ›› Issue (4): 887-899.DOI: 10.12263/DZXB.20210842
杨曦1, 张鑫1, 郭浩远1, 王楠楠1, 高新波1,2
收稿日期:
2021-07-05
修回日期:
2021-09-26
出版日期:
2022-04-25
发布日期:
2022-04-25
作者简介:
基金资助:
YANG Xi1, ZHANG Xin1, GUO Hao-yuan1, WANG Nan-nan1, GAO Xin-bo1,2
Received:
2021-07-05
Revised:
2021-09-26
Online:
2022-04-25
Published:
2022-04-25
摘要:
由于域偏移的存在,多源图像舰船目标检测任务面临着不同源传感器带来的图像风格差异难题.另外,为特定数据源训练特定的检测模型会消耗大量的计算资源,严重限制了其在军民用领域的工程应用.因此,设计一个通用网络以有效检测来自不同源遥感数据的舰船目标成了当下的研究热点.针对该需求,本文提出了一种基于不变特征的通用舰船目标检测方法,通过充分利用多源数据之间的共享知识实现通用遥感目标的网络检测.本方法由2部分组成:图像级的风格转换网络和特征级的域自适应网络.具体地,前者采用风格转换网络生成接近真实分布的伪多源图像,拉近多源数据之间的分布,在图像层面上学习多源数据的不变特征;为学习特征层面上多源数据的不变特征,后者通过适应网络对多源特征进行信息解耦,通过域注意力网络的自适应权重分配实现特征重组.本文在NWPU VHR-10,SSDD,HRSC和SAR-Ship-Dataset数据集上进行实验验证,结果表明:所提方法通过不变特征之间的信息互补,缓解了域偏移问题,可有效检测多源遥感数据.本文方法在上述多源数据集上的平均mAP为90.8%,相比现有主流舰船目标检测方法可以提高1.4%~10.6%.
中图分类号:
杨曦, 张鑫, 郭浩远, 王楠楠, 高新波. 基于不变特征的多源遥感图像舰船目标检测算法[J]. 电子学报, 2022, 50(4): 887-899.
YANG Xi, ZHANG Xin, GUO Hao-yuan, WANG Nan-nan, GAO Xin-bo. Invariant Features Based Ship Detection Model for Multi-source Remote Sensing Images[J]. Acta Electronica Sinica, 2022, 50(4): 887-899.
图像级 | 特征级 | NWPU VHR-10/% | SSDD/% | Avg/% |
---|---|---|---|---|
No | No | 72.7 | 93.6 | 83.2 |
Yes | No | 73.2 | 94.3 | 83.7 |
Yes | Yes | 74.1 | 95.6 | 84.9 |
表1 多源数据集上消融实验mAP
图像级 | 特征级 | NWPU VHR-10/% | SSDD/% | Avg/% |
---|---|---|---|---|
No | No | 72.7 | 93.6 | 83.2 |
Yes | No | 73.2 | 94.3 | 83.7 |
Yes | Yes | 74.1 | 95.6 | 84.9 |
Methods | NWPU VHR-10/% | SSDD/% | Avg/% | Runtime/s |
---|---|---|---|---|
Faster R-CNN | 71.8 | 91.3 | 81.6 | 0.068 |
Cascade R-CNN | 71.9 | 93.4 | 82.8 | 0.087 |
Libra R-CNN | 72.3 | 93.9 | 83.1 | 0.071 |
RetinaNet | 73.1 | 90.5 | 81.8 | 0.059 |
EFGRNet | 72.2 | 92.4 | 82.3 | 0.033 |
CenterNet | 72.7 | 93.6 | 83.2 | 0.031 |
CenterNet++ | 73.4 | 94.3 | 83.9 | 0.033 |
Universal DA | 73.1 | 88.8 | 81.0 | 0.122 |
DA Faster R-CNN | 74.1 | 88.1 | 81.1 | 0.069 |
Ours (N=1) | 73.7 | 94.8 | 84.2 | 0.037 |
Ours (N=2) | 73.9 | 95.0 | 84.5 | 0.037 |
Ours (N=3) | 74.1 | 95.6 | 84.9 | 0.037 |
Ours (N=4) | 73.8 | 94.6 | 84.2 | 0.038 |
表2 各种算法在SSDD和NWPU VRH-10数据集上的mAP和Runtime
Methods | NWPU VHR-10/% | SSDD/% | Avg/% | Runtime/s |
---|---|---|---|---|
Faster R-CNN | 71.8 | 91.3 | 81.6 | 0.068 |
Cascade R-CNN | 71.9 | 93.4 | 82.8 | 0.087 |
Libra R-CNN | 72.3 | 93.9 | 83.1 | 0.071 |
RetinaNet | 73.1 | 90.5 | 81.8 | 0.059 |
EFGRNet | 72.2 | 92.4 | 82.3 | 0.033 |
CenterNet | 72.7 | 93.6 | 83.2 | 0.031 |
CenterNet++ | 73.4 | 94.3 | 83.9 | 0.033 |
Universal DA | 73.1 | 88.8 | 81.0 | 0.122 |
DA Faster R-CNN | 74.1 | 88.1 | 81.1 | 0.069 |
Ours (N=1) | 73.7 | 94.8 | 84.2 | 0.037 |
Ours (N=2) | 73.9 | 95.0 | 84.5 | 0.037 |
Ours (N=3) | 74.1 | 95.6 | 84.9 | 0.037 |
Ours (N=4) | 73.8 | 94.6 | 84.2 | 0.038 |
Methods | NWPU VHR-10 | SSDD | HRSC | SAR-Ship-Dataset | Avg |
---|---|---|---|---|---|
Faster R-CNN | 70.2 | 84.1 | 85.2 | 94.9 | 83.6 |
Cascade R-CNN | 70.0 | 86.8 | 85.5 | 94.8 | 84.3 |
Libra R-CNN | 75.2 | 85.6 | 84.8 | 95.7 | 85.3 |
RetinaNet | 69.8 | 91.3 | 84.9 | 96.4 | 85.6 |
EFGRNet | 79.7 | 91.6 | 81.2 | 94.9 | 86.9 |
CenterNet | 80.6 | 91.2 | 91.8 | 94.1 | 89.4 |
CenterNet++ | 76.6 | 92.3 | 92.7 | 94.2 | 89.0 |
Universal DA | 72.8 | 85.5 | 74.8 | 87.7 | 80.2 |
DA Faster R-CNN | 78.9 | 88.1 | 76.3 | 89.6 | 83.2 |
Ours (N=1) | 78.2 | 93.0 | 93.7 | 94.5 | 89.8 |
Ours (N=2) | 81.1 | 92.9 | 93 | 94.8 | 90.4 |
Ours (N=3) | 82.1 | 92.7 | 93.8 | 94.6 | 90.8 |
Ours (N=4) | 77.3 | 92.8 | 92.7 | 94.6 | 89.4 |
表3 多源数据集上各种算法的mAP (%)
Methods | NWPU VHR-10 | SSDD | HRSC | SAR-Ship-Dataset | Avg |
---|---|---|---|---|---|
Faster R-CNN | 70.2 | 84.1 | 85.2 | 94.9 | 83.6 |
Cascade R-CNN | 70.0 | 86.8 | 85.5 | 94.8 | 84.3 |
Libra R-CNN | 75.2 | 85.6 | 84.8 | 95.7 | 85.3 |
RetinaNet | 69.8 | 91.3 | 84.9 | 96.4 | 85.6 |
EFGRNet | 79.7 | 91.6 | 81.2 | 94.9 | 86.9 |
CenterNet | 80.6 | 91.2 | 91.8 | 94.1 | 89.4 |
CenterNet++ | 76.6 | 92.3 | 92.7 | 94.2 | 89.0 |
Universal DA | 72.8 | 85.5 | 74.8 | 87.7 | 80.2 |
DA Faster R-CNN | 78.9 | 88.1 | 76.3 | 89.6 | 83.2 |
Ours (N=1) | 78.2 | 93.0 | 93.7 | 94.5 | 89.8 |
Ours (N=2) | 81.1 | 92.9 | 93 | 94.8 | 90.4 |
Ours (N=3) | 82.1 | 92.7 | 93.8 | 94.6 | 90.8 |
Ours (N=4) | 77.3 | 92.8 | 92.7 | 94.6 | 89.4 |
1 | 种劲松, 朱敏慧. SAR图像舰船及其尾迹检测研究综述[J]. 电子学报, 2003, 31(9): 1356-1360. |
CHONG J S, ZHU M H. Survey of the study on ship and wake detection in SAR imagery[J]. Acta Electronica Sinica, 2003, 31(9): 1356-1360. (in Chinese) | |
2 | WANG C, BI F, ZHANG W, et al. An intensity-space domain CFAR method for ship detection in HR SAR images[J]. IEEE Geoscience and Remote Seneing Letters, 2017, 14(4): 529-533. |
3 | PAPPAS O, ACHIM A, BULL D. Superpixel-level CFAR detectors for ship detection in SAR images[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(9): 1397-1401. |
4 | HUO W, HUANG Y, PEI J, et al. Ship detection from ocean SAR image based on local contrast variance weighted information entropy[J]. Sensors, 2018, 18(4): 1196. |
5 | 王明春, 张嘉峰, 杨子渊, 等. Beta分布下基于白化滤波的极化SAR图像海面舰船目标CFAR检测方法[J]. 电子学报, 2019, 47(9): 77-84. |
WANG M C, ZHANG J F, YANG Z Y, et, al. A CFAR detection method of ship targets in polarimetric SAR imagery based on whitening filter under Beta distributed texture[J]. Acta Electronica Sinica, 2019, 47(9): 77-84. (in Chinese) | |
6 | NIE J, ANWER R, CHOLAKKAL H, et al. Enriched feature guided refinement network for object detection[C]//Proceedings of the IEEE International Conference on Computer Vision. Kerkyra: IEEE, 2019: 9537-9546. |
7 | ZHOU X, WANG D, KRAHENBUHL P. Objects as points[EB/OL]. (2019-04-16)[2021-07-05]. . |
8 | REN S, HE K, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(6): 1137-1149. |
9 | PANG J, CHEN K, SHI J, et al. Libra R-CNN: Towards balanced learning for object detection[C]//Proceedings of International Conference on Computer Vision and Pattern Recognition. Long Beach : IEEE, 2019: 821-830. |
10 | ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the IEEE International Conference on Computer Vision. Venice:IEEE, 2017: 2223-2232. |
11 | GIRSHICK R. Fast R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 1440-1448. |
12 | LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proceedings of International Conference on Computer Vision and Pattern Recognition. Venice: IEEE, 2017: 2117-2125. |
13 | JIANG B, LUO R, MAO J, et al. Acquisition of localization confidence for accurate object detection[C]//Proceedings of the European Conference on Computer Vision. Munich: Springer, 2018: 784-799. |
14 | LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//Proceedings of the IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2980-2988. |
15 | REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: Unified, real-time object detection [C]//Proceedings of International Conference on Computer Vision and Pattern Recognition. Los Vegas: IEEE, 2016: 779-788. |
16 | REDMON J, FARHADI A. YOLO9000: Better, faster, stronger[C]//Proceedings of International Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA: IEEE, 2017: 7263-7271. |
17 | REDMON J, FARHADI A. YOLOv3: An incremental improvement[EB/OL]. (2018-04-08)[2021-07-05]. . |
18 | BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: Optimal speed and accuracy of object detection[EB/OL]. (2020-04-23)[2021-07-05]. . |
19 | LAW H, DENG J. Cornernet: Detecting objects as paired keypoints[C]//Proceedings of the European Conference on Computer Vision. Munich: Springer, 2018: 734-750. |
20 | CUI Z, LI Q, CAO Z, et al. Dense attention pyramid networks for multi-scale ship detection in SAR images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(11): 8983-8997. |
21 | YANG X, LIU Q, YAN J, et al. R3 Det: Refined singlestage detector with feature refinement for rotating object[EB/OL]. (2019-08-15)[2021-07-05]. . |
22 | FU J, SUN X, WANG Z, et al. An anchor-free method based on feature balancing and refinement network for multiscale ship detection in SAR images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 59(2): 1331-1344. |
23 | CHEN Y, LI W, SAKARIDIS C, et al. Domain adaptive Faster R-CNN for object detection in the wild[C]//Proceedings of International Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 3339-3348. |
24 | SAITO K, USHIKU Y, HARADA T, et al. Strong-weak distribution alignment for adaptive object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 6956-6965. |
25 | WANG X, CAI Z, GAO D, et al. Towards universal object detection by domain attention[C]//Proceedings of International Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 7289-7298. |
26 | GATYS L A, ECKER A S, Bethge M. Image style transfer using convolutional neural networks[C]//Proceedings of International Conference on Computer Vision and Pattern Recognition. Bhubaneswar: IEEE, 2016: 2414-2423. |
27 | BERGER G, MEMISEVIC R. Incorporating long-range consistency in CNN-based texture generation[EB/OL]. (2016-01-03)[2021-07-05]. . |
28 | GU S, CHEN C, LIAO J, et al. Arbitrary style transfer with deep feature reshuffle[C]//Proceedings of International Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 8222-8231. |
29 | WANG X, OXHOLM G, ZHANG D, et al. Multimodal transfer: A hierarchical deep convolutional neural network for fast artistic style transfer[C]//Proceedings of International Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 5239-5247. |
30 | YU F, WANG D, SHELHAMER E, et al. Deep layer aggregation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 2403-2412. |
31 | HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7132-7141. |
32 | CHENG G, ZHOU P, HAN J. Learning rotation-invariant convolutional neural networks for object detection in VHR optical remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(12): 7405-7415. |
33 | LI J, QU C, SHAO J. Ship detection in SAR images based on an improved faster R-CNN[C]//SAR in Big Data Era: Models, Methods and Applications(BIGSARDATA). Beijing: IEEE, 2017: 1-6. |
34 | LIU Z, YUAN L, WENG L, et al. A high resolution optical satellite image dataset for ship recognition and some new baselines[C]//International Conference on Pattern Recognition Applications and Methods. Porto: ICPRAM, 2017: 324-331. |
35 | WANG Y, WANG C, ZHANG H, et al. A SAR dataset of ship detection for deep learning under complex backgrounds[J]. Remote Sensing, 2019, 11(7): 765. |
36 | CAI Z, VASCONCELOS N. Cascade R-CNN: Delving into high quality object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6154-6162. |
37 | GUO H, YANG X, WANG N, et al. A CenterNet++ model for ship detection in SAR images[J]. Pattern Recognition, 2021, 112: 107787. |
38 | MENON S, DAMIAN A, HU S, et al. PULSE: Self-supervised photo upsampling via latent space exploration of generative models[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 2437-2445. |
[1] | 张波, 陆云杰, 秦东明, 邹国建. 一种卷积自编码深度学习的空气污染多站点联合预测模型[J]. 电子学报, 2022, 50(6): 1410-1427. |
[2] | 杨伟超, 杜宇, 文伟, 侯舒维, 徐常志, 张建华. 基于多重分形谱智能分析的卫星信号调制识别研究[J]. 电子学报, 2022, 50(6): 1336-1343. |
[3] | 彭闯, 王伦文, 胡炜林. 融合深度特征的电磁频谱异常检测算法[J]. 电子学报, 2022, 50(6): 1359-1369. |
[4] | 李政伟, 李佳树, 尤著宏, 聂茹, 赵欢, 钟堂波. 基于异质图注意力网络的miRNA与疾病关联预测算法[J]. 电子学报, 2022, 50(6): 1428-1435. |
[5] | 冀振燕, 韩梦豪, 宋晓军, 冯其波. 面向激光光条图像修复的循环相似度映射网络[J]. 电子学报, 2022, 50(5): 1234-1242. |
[6] | 廖勇, 李玉杰. 一种轻量化低复杂度的FDD大规模MIMO系统CSI反馈方法[J]. 电子学报, 2022, 50(5): 1211-1217. |
[7] | 潘敏婷, 王韫博, 朱祥明, 高思宇, 龙明盛, 杨小康. 基于无标签视频数据的深度预测学习方法综述[J]. 电子学报, 2022, 50(4): 869-886. |
[8] | 崔亚奇, 何友, 唐田田, 熊伟. 一种深度学习航迹关联方法[J]. 电子学报, 2022, 50(3): 759-763. |
[9] | 张浩, 胡昌华, 杜党波, 裴洪, 张建勋. 多状态影响下基于Bi‑LSTM网络的锂电池剩余寿命预测方法[J]. 电子学报, 2022, 50(3): 619-624. |
[10] | 骈纬国, 吴映波, 陈蒙, 蔡俊鹏. 一种基于时空动态图注意力网络的共享出行需求预测方法[J]. 电子学报, 2022, 50(2): 432-439. |
[11] | 李居朋, 王颖慧, 李刚. 医学图像关键点检测深度学习方法研究与挑战[J]. 电子学报, 2022, 50(1): 226-237. |
[12] | 张聿远, 张立民, 闫文君. 基于深度多级残差网络的低信噪比下空频分组码识别方法[J]. 电子学报, 2022, 50(1): 79-88. |
[13] | 席亮, 刘涵, 樊好义, 张凤斌. 基于深度对抗学习潜在表示分布的异常检测模型[J]. 电子学报, 2021, 49(7): 1257-1265. |
[14] | 谢佳龙, 张波涛, 吕强. 一种基于双流融合3D卷积神经网络的动态头势识别方法[J]. 电子学报, 2021, 49(7): 1363-1369. |
[15] | 罗会兰, 袁璞, 童康. 基于深度学习的显著性目标检测方法综述[J]. 电子学报, 2021, 49(7): 1417-1427. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||