基于不变特征的多源遥感图像舰船目标检测算法

doi:10.12263/DZXB.20210842

电子学报 ›› 2022, Vol. 50 ›› Issue (4): 887-899.DOI: 10.12263/DZXB.20210842

• 机器学习交叉融合创新 • 上一篇下一篇

基于不变特征的多源遥感图像舰船目标检测算法

杨曦¹, 张鑫¹, 郭浩远¹, 王楠楠¹, 高新波¹^,²

^1.西安电子科技大学综合业务网理论及关键技术国家重点实验室，陕西西安 710071
^2.重庆邮电大学图像认知重庆市重点实验室，重庆 400065

收稿日期:2021-07-05 修回日期:2021-09-26 出版日期:2022-04-25 发布日期:2022-04-25
作者简介:杨曦女，1988年10月出生,陕西咸阳人.现为西安电子科技大学通信工程学院教授、博士生导师.研究方向为计算机视觉和机器学习.E-mail: yangx@xidian.edu.cn
张鑫男，1998年11月出生，陕西渭南人.现为西安电子科技大学通信工程学院硕士研究生.研究方向为计算机视觉和目标检测.E-mail: xzhang_98@stu.xidian.edu.cn
郭浩远男，1996年2月出生，河南郑州人.2021年获得西安电子科技大学通信工程学院硕士学位.研究方向计算机视觉和机器学习.E-mail: guohy@stu.xidian.edu.cn
王楠楠男，1986年11月出生，山东安丘人.现为西安电子科技大学通信工程学院教授、博士生导师.研究方向为计算机视觉和机器学习.E-mail: nnwang@xidian.edu.cn
高新波（通讯作者）男，1972年8月出生，山东莱芜人.现为西安电子科技大学电子工程学院教授、博士生导师，重庆邮电大学计算机学院教授、博士生导师.研究方向为计算机视觉和机器学习.
基金资助:
国家自然科学基金(61976166);陕西省重点研发计划(2021GY-030);陕西省创新人才推进计划(2020KJXX-027);中央高校基本科研业务费(JB210115)

Invariant Features Based Ship Detection Model for Multi-source Remote Sensing Images

YANG Xi¹, ZHANG Xin¹, GUO Hao-yuan¹, WANG Nan-nan¹, GAO Xin-bo¹^,²

^1.State Key Laboratory of Integrated Services Networks，Xidian University，Xi’an，Shaanxi 710071，China
^2.Chongqing Key Laboratory of Image Cognition，Chongqing University of Posts and Telecommunications，Chongqing 400065，China

Received:2021-07-05 Revised:2021-09-26 Online:2022-04-25 Published:2022-04-25

摘要/Abstract

摘要：

由于域偏移的存在，多源图像舰船目标检测任务面临着不同源传感器带来的图像风格差异难题.另外，为特定数据源训练特定的检测模型会消耗大量的计算资源，严重限制了其在军民用领域的工程应用.因此，设计一个通用网络以有效检测来自不同源遥感数据的舰船目标成了当下的研究热点.针对该需求，本文提出了一种基于不变特征的通用舰船目标检测方法，通过充分利用多源数据之间的共享知识实现通用遥感目标的网络检测.本方法由2部分组成：图像级的风格转换网络和特征级的域自适应网络.具体地，前者采用风格转换网络生成接近真实分布的伪多源图像，拉近多源数据之间的分布，在图像层面上学习多源数据的不变特征；为学习特征层面上多源数据的不变特征，后者通过适应网络对多源特征进行信息解耦，通过域注意力网络的自适应权重分配实现特征重组.本文在NWPU VHR-10，SSDD，HRSC和SAR-Ship-Dataset数据集上进行实验验证，结果表明：所提方法通过不变特征之间的信息互补，缓解了域偏移问题，可有效检测多源遥感数据.本文方法在上述多源数据集上的平均mAP为90.8%，相比现有主流舰船目标检测方法可以提高1.4%~10.6%.

关键词: 舰船检测, 遥感图像, 深度学习, 风格转换, 域自适应

Abstract:

Due to the domain shift, ship detection in multi-source data suffers from image variations caused by different source sensors. In addition, training a specific model for a particular data source consumes high computational cost, which severely limits its practical application in military and civilian fields. Therefore, designing a universal network to effectively detect ship objects from multi-source remote sensing images has become a research hotspot. To this end, the paper proposes a universal ship detection algorithm based on invariant features, which realizes a universal remote sensing object detection network by fully utilizing the shared knowledge among multi-source data. Our method mainly consists of two parts, i.e., an image-level style transfer network and a feature-level domain adaptive network. Specifically, the former employs style transfer network to generate pseudo-multi-source images that are close to the real distribution, narrow the distribution between multi-source data, and extract the invariant features of multi-source data at the image level; To extract invariant features at the feature level, the latter decouples the multi-source features through adaptive network, and realizes feature reorganization through adaptive weight allocation of domain attention network. We evaluate the proposed method using multiple datasets including NWPU VHR-10, SSDD, HRSC and SAR-Ship-Dataset. Experimental results show that the proposed method alleviates the problem of domain shift by complementing the information between invariant features, and can effectively detect multi-source remote sensing data. The average mAP of our method on the above-mentioned multi-source datasets is 90.8%, which exceeds 1.4%-10.6% compared with the existing mainstream ship object detection methods.

Key words: ship detection, remote sensing images, deep learning, style transfer, domain adaptation

中图分类号:

TN911.73

杨曦, 张鑫, 郭浩远, 王楠楠, 高新波. 基于不变特征的多源遥感图像舰船目标检测算法[J]. 电子学报, 2022, 50(4): 887-899.

YANG Xi, ZHANG Xin, GUO Hao-yuan, WANG Nan-nan, GAO Xin-bo. Invariant Features Based Ship Detection Model for Multi-source Remote Sensing Images[J]. Acta Electronica Sinica, 2022, 50(4): 887-899.

图/表 14

参考文献 38

1	种劲松, 朱敏慧. SAR图像舰船及其尾迹检测研究综述[J]. 电子学报, 2003, 31(9): 1356-1360.
	CHONG J S, ZHU M H. Survey of the study on ship and wake detection in SAR imagery[J]. Acta Electronica Sinica, 2003, 31(9): 1356-1360. (in Chinese)
2	WANG C, BI F, ZHANG W, et al. An intensity-space domain CFAR method for ship detection in HR SAR images[J]. IEEE Geoscience and Remote Seneing Letters, 2017, 14(4): 529-533.
3	PAPPAS O, ACHIM A, BULL D. Superpixel-level CFAR detectors for ship detection in SAR images[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(9): 1397-1401.
4	HUO W, HUANG Y, PEI J, et al. Ship detection from ocean SAR image based on local contrast variance weighted information entropy[J]. Sensors, 2018, 18(4): 1196.
5	王明春, 张嘉峰, 杨子渊, 等. Beta分布下基于白化滤波的极化SAR图像海面舰船目标CFAR检测方法[J]. 电子学报, 2019, 47(9): 77-84.
	WANG M C, ZHANG J F, YANG Z Y, et, al. A CFAR detection method of ship targets in polarimetric SAR imagery based on whitening filter under Beta distributed texture[J]. Acta Electronica Sinica, 2019, 47(9): 77-84. (in Chinese)
6	NIE J, ANWER R, CHOLAKKAL H, et al. Enriched feature guided refinement network for object detection[C]//Proceedings of the IEEE International Conference on Computer Vision. Kerkyra: IEEE, 2019: 9537-9546.
7	ZHOU X, WANG D, KRAHENBUHL P. Objects as points[EB/OL]. (2019-04-16)[2021-07-05]. .
8	REN S, HE K, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(6): 1137-1149.
9	PANG J, CHEN K, SHI J, et al. Libra R-CNN: Towards balanced learning for object detection[C]//Proceedings of International Conference on Computer Vision and Pattern Recognition. Long Beach : IEEE, 2019: 821-830.
10	ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the IEEE International Conference on Computer Vision. Venice:IEEE, 2017: 2223-2232.
11	GIRSHICK R. Fast R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 1440-1448.
12	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proceedings of International Conference on Computer Vision and Pattern Recognition. Venice: IEEE, 2017: 2117-2125.
13	JIANG B, LUO R, MAO J, et al. Acquisition of localization confidence for accurate object detection[C]//Proceedings of the European Conference on Computer Vision. Munich: Springer, 2018: 784-799.
14	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//Proceedings of the IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2980-2988.
15	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: Unified, real-time object detection [C]//Proceedings of International Conference on Computer Vision and Pattern Recognition. Los Vegas: IEEE, 2016: 779-788.
16	REDMON J, FARHADI A. YOLO9000: Better, faster, stronger[C]//Proceedings of International Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA: IEEE, 2017: 7263-7271.
17	REDMON J, FARHADI A. YOLOv3: An incremental improvement[EB/OL]. (2018-04-08)[2021-07-05]. .
18	BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: Optimal speed and accuracy of object detection[EB/OL]. (2020-04-23)[2021-07-05]. .
19	LAW H, DENG J. Cornernet: Detecting objects as paired keypoints[C]//Proceedings of the European Conference on Computer Vision. Munich: Springer, 2018: 734-750.
20	CUI Z, LI Q, CAO Z, et al. Dense attention pyramid networks for multi-scale ship detection in SAR images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(11): 8983-8997.
21	YANG X, LIU Q, YAN J, et al. R3 Det: Refined singlestage detector with feature refinement for rotating object[EB/OL]. (2019-08-15)[2021-07-05]. .
22	FU J, SUN X, WANG Z, et al. An anchor-free method based on feature balancing and refinement network for multiscale ship detection in SAR images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 59(2): 1331-1344.
23	CHEN Y, LI W, SAKARIDIS C, et al. Domain adaptive Faster R-CNN for object detection in the wild[C]//Proceedings of International Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 3339-3348.
24	SAITO K, USHIKU Y, HARADA T, et al. Strong-weak distribution alignment for adaptive object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 6956-6965.
25	WANG X, CAI Z, GAO D, et al. Towards universal object detection by domain attention[C]//Proceedings of International Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 7289-7298.
26	GATYS L A, ECKER A S, Bethge M. Image style transfer using convolutional neural networks[C]//Proceedings of International Conference on Computer Vision and Pattern Recognition. Bhubaneswar: IEEE, 2016: 2414-2423.
27	BERGER G, MEMISEVIC R. Incorporating long-range consistency in CNN-based texture generation[EB/OL]. (2016-01-03)[2021-07-05]. .
28	GU S, CHEN C, LIAO J, et al. Arbitrary style transfer with deep feature reshuffle[C]//Proceedings of International Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 8222-8231.
29	WANG X, OXHOLM G, ZHANG D, et al. Multimodal transfer: A hierarchical deep convolutional neural network for fast artistic style transfer[C]//Proceedings of International Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 5239-5247.
30	YU F, WANG D, SHELHAMER E, et al. Deep layer aggregation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 2403-2412.
31	HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7132-7141.
32	CHENG G, ZHOU P, HAN J. Learning rotation-invariant convolutional neural networks for object detection in VHR optical remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(12): 7405-7415.
33	LI J, QU C, SHAO J. Ship detection in SAR images based on an improved faster R-CNN[C]//SAR in Big Data Era: Models, Methods and Applications(BIGSARDATA). Beijing: IEEE, 2017: 1-6.
34	LIU Z, YUAN L, WENG L, et al. A high resolution optical satellite image dataset for ship recognition and some new baselines[C]//International Conference on Pattern Recognition Applications and Methods. Porto: ICPRAM, 2017: 324-331.
35	WANG Y, WANG C, ZHANG H, et al. A SAR dataset of ship detection for deep learning under complex backgrounds[J]. Remote Sensing, 2019, 11(7): 765.
36	CAI Z, VASCONCELOS N. Cascade R-CNN: Delving into high quality object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6154-6162.
37	GUO H, YANG X, WANG N, et al. A CenterNet++ model for ship detection in SAR images[J]. Pattern Recognition, 2021, 112: 107787.
38	MENON S, DAMIAN A, HU S, et al. PULSE: Self-supervised photo upsampling via latent space exploration of generative models[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 2437-2445.

图像级	特征级	NWPU VHR-10/%	SSDD/%	Avg/%
No	No	72.7	93.6	83.2
Yes	No	73.2	94.3	83.7
Yes	Yes	74.1	95.6	84.9

图像级	特征级	NWPU VHR-10/%	SSDD/%	Avg/%
No	No	72.7	93.6	83.2
Yes	No	73.2	94.3	83.7
Yes	Yes	74.1	95.6	84.9

Methods	NWPU VHR-10/%	SSDD/%	Avg/%	Runtime/s
Faster R-CNN	71.8	91.3	81.6	0.068
Cascade R-CNN	71.9	93.4	82.8	0.087
Libra R-CNN	72.3	93.9	83.1	0.071
RetinaNet	73.1	90.5	81.8	0.059
EFGRNet	72.2	92.4	82.3	0.033
CenterNet	72.7	93.6	83.2	0.031
CenterNet++	73.4	94.3	83.9	0.033
Universal DA	73.1	88.8	81.0	0.122
DA Faster R-CNN	74.1	88.1	81.1	0.069
Ours （N=1）	73.7	94.8	84.2	0.037
Ours （N=2）	73.9	95.0	84.5	0.037
Ours （N=3）	74.1	95.6	84.9	0.037
Ours （N=4）	73.8	94.6	84.2	0.038

Methods	NWPU VHR-10/%	SSDD/%	Avg/%	Runtime/s
Faster R-CNN	71.8	91.3	81.6	0.068
Cascade R-CNN	71.9	93.4	82.8	0.087
Libra R-CNN	72.3	93.9	83.1	0.071
RetinaNet	73.1	90.5	81.8	0.059
EFGRNet	72.2	92.4	82.3	0.033
CenterNet	72.7	93.6	83.2	0.031
CenterNet++	73.4	94.3	83.9	0.033
Universal DA	73.1	88.8	81.0	0.122
DA Faster R-CNN	74.1	88.1	81.1	0.069
Ours （N=1）	73.7	94.8	84.2	0.037
Ours （N=2）	73.9	95.0	84.5	0.037
Ours （N=3）	74.1	95.6	84.9	0.037
Ours （N=4）	73.8	94.6	84.2	0.038

Methods	NWPU VHR-10	SSDD	HRSC	SAR-Ship-Dataset	Avg
Faster R-CNN	70.2	84.1	85.2	94.9	83.6
Cascade R-CNN	70.0	86.8	85.5	94.8	84.3
Libra R-CNN	75.2	85.6	84.8	95.7	85.3
RetinaNet	69.8	91.3	84.9	96.4	85.6
EFGRNet	79.7	91.6	81.2	94.9	86.9
CenterNet	80.6	91.2	91.8	94.1	89.4
CenterNet++	76.6	92.3	92.7	94.2	89.0
Universal DA	72.8	85.5	74.8	87.7	80.2
DA Faster R-CNN	78.9	88.1	76.3	89.6	83.2
Ours （N=1）	78.2	93.0	93.7	94.5	89.8
Ours （N=2）	81.1	92.9	93	94.8	90.4
Ours （N=3）	82.1	92.7	93.8	94.6	90.8
Ours （N=4）	77.3	92.8	92.7	94.6	89.4

基于不变特征的多源遥感图像舰船目标检测算法

Invariant Features Based Ship Detection Model for Multi-source Remote Sensing Images

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献 38

相关文章 15

编辑推荐

Metrics

[1]	张波, 陆云杰, 秦东明, 邹国建. 一种卷积自编码深度学习的空气污染多站点联合预测模型[J]. 电子学报, 2022, 50(6): 1410-1427.
[2]	杨伟超, 杜宇, 文伟, 侯舒维, 徐常志, 张建华. 基于多重分形谱智能分析的卫星信号调制识别研究[J]. 电子学报, 2022, 50(6): 1336-1343.
[3]	彭闯, 王伦文, 胡炜林. 融合深度特征的电磁频谱异常检测算法[J]. 电子学报, 2022, 50(6): 1359-1369.
[4]	李政伟, 李佳树, 尤著宏, 聂茹, 赵欢, 钟堂波. 基于异质图注意力网络的miRNA与疾病关联预测算法[J]. 电子学报, 2022, 50(6): 1428-1435.
[5]	冀振燕, 韩梦豪, 宋晓军, 冯其波. 面向激光光条图像修复的循环相似度映射网络[J]. 电子学报, 2022, 50(5): 1234-1242.
[6]	廖勇, 李玉杰. 一种轻量化低复杂度的FDD大规模MIMO系统CSI反馈方法[J]. 电子学报, 2022, 50(5): 1211-1217.
[7]	潘敏婷, 王韫博, 朱祥明, 高思宇, 龙明盛, 杨小康. 基于无标签视频数据的深度预测学习方法综述[J]. 电子学报, 2022, 50(4): 869-886.
[8]	崔亚奇, 何友, 唐田田, 熊伟. 一种深度学习航迹关联方法[J]. 电子学报, 2022, 50(3): 759-763.
[9]	张浩, 胡昌华, 杜党波, 裴洪, 张建勋. 多状态影响下基于Bi‑LSTM网络的锂电池剩余寿命预测方法[J]. 电子学报, 2022, 50(3): 619-624.
[10]	骈纬国, 吴映波, 陈蒙, 蔡俊鹏. 一种基于时空动态图注意力网络的共享出行需求预测方法[J]. 电子学报, 2022, 50(2): 432-439.
[11]	李居朋, 王颖慧, 李刚. 医学图像关键点检测深度学习方法研究与挑战[J]. 电子学报, 2022, 50(1): 226-237.
[12]	张聿远, 张立民, 闫文君. 基于深度多级残差网络的低信噪比下空频分组码识别方法[J]. 电子学报, 2022, 50(1): 79-88.
[13]	席亮, 刘涵, 樊好义, 张凤斌. 基于深度对抗学习潜在表示分布的异常检测模型[J]. 电子学报, 2021, 49(7): 1257-1265.
[14]	谢佳龙, 张波涛, 吕强. 一种基于双流融合3D卷积神经网络的动态头势识别方法[J]. 电子学报, 2021, 49(7): 1363-1369.
[15]	罗会兰, 袁璞, 童康. 基于深度学习的显著性目标检测方法综述[J]. 电子学报, 2021, 49(7): 1417-1427.