基于改进CycleGAN的光学图像迁移生成水下小目标合成孔径声纳图像算法研究

doi:10.12263/DZXB.20200712

电子学报 ›› 2021, Vol. 49 ›› Issue (9): 1746-1753.DOI: 10.12263/DZXB.20200712

基于改进CycleGAN的光学图像迁移生成水下小目标合成孔径声纳图像算法研究

李宝奇¹^,², 黄海宁¹^,², 刘纪元¹^,², 李宇¹^,²

^1.中国科学院声学研究所，北京 100190
^2.中国科学院先进水下信息技术重点实验室，北京 100190

收稿日期:2020-07-14 修回日期:2021-05-17 出版日期:2021-10-21
- 作者简介:
- 李宝奇　男，1985年出生于天津，中国科学院声学研究所特别研究助理.主要从事水下目标探测、识别和跟踪等方面的研究.E⁃mail：libaoqi@mail.ioa.ac.cn
  黄海宁（通信作者）　男，1969年出生于河北，中国科学院声学研究所研究员，博士生导师，国务院政府津贴专家.目前担任中科院海洋信息技术创新研究院暨声学研究所科技委副主任，中科院先进水下信息技术重点实验室暨水声工程中心主任，中国声学学会理事.主要从事水声信号与信息处理、目标探测，水声通信与网络等方面的研究.E⁃mail:hhn@mail.ioa.ac.cn
  刘纪元　男，1963 年出生于辽宁，中国科学院声学研究所研究员，博士生导师.中国图形图像学会视觉与传感专业委员会会员，国家“863”计划“基于无人平台的合成孔径声纳系统研究”项目首席专家.主要研究领域包括水声信号处理、高分辨率水下成像技术等.E⁃mail:ljy@mail.ioa.ac.cn
  李　宇　男，1977年出生贵州，中国科学院声学研究所研究员.研究领域涉及水声信号处理、水声通信与网络、无人平台声纳技术、主动声纳技术、阵列信号处理等的多个方面. E⁃mail:ly@mail.ioa.ac.cn
- 基金资助:
- 国家自然科学基金 (11904386); 国家基础科研计划重大项目 (JCKY2016206A003); 中国科学院青年创新促进会

Optical Image-to-Underwater Small Target Synthetic Aperture Sonar Image Translation Algorithm Based on Improved CycleGAN

LI Bao-qi¹^,², HUANG Hai-ning¹^,², LIU Ji-yuan¹^,², LI Yu¹^,²

^1.Institute of Acoustics, Chinese Academy of Sciences, Beijing 100190, China
^2.Key Laboratory of Science and Technology on Advanced Underwater Acoustic Signal Processing, Chinese Academy of Sciences, Beijing 100190, China

Received:2020-07-14 Revised:2021-05-17 Online:2021-10-21 Published:2021-09-25
- Supported by:
- National Natural Science Foundation of China (11904386); Key Program of National Basic Research Project of China (JCKY2016206A003); Youth Innovation Promotion Association, CAS

摘要/Abstract

摘要：

针对循环生成对抗网络CycleGAN（Cycle Generative Adversarial Networks）在光学图像迁移生成水下小目标合成孔径声纳图像过程中存在质量差和速度慢的问题，本文提出一种新的特征提取单元SDK（Selective Dilated Kernel），并利用SDK设计了一个新的生成器网络SDKNet.与此同时，提出了一种新的循环一致损失函数MS-CCLF（Multiscale Cyclic Consistent Loss Function），MS-CCLF增加了图像多尺度结构相似性约束.在自建的图像迁移数据集OPT-SAS上，本文SM-CycleGAN（Selective and Multiscale Cycle Generative Adversarial Networks）比原始CycleGAN的图像迁移质量提升4.64%，生成器网络参数降低4.13MB, 运算时间减少0.143s.实验结果表明，SM-CycleGAN更适合水下小目标光学图像到合成孔径声纳图像的迁移任务.

关键词: 光学图像迁移生成合成孔径声纳图像, 生成对抗网络, 循环生成对抗网络, 可选择空洞核网络, 多尺度结构相似性

Abstract:

The original CycleGAN show poor quality and time consuming in optical image to underwater small target synthetic aperture sonar image translation task. To address those problems, a novel convolution building block, SDK (Selective Dilated Kernel), is proposed. By stacking SDK blocks, a generator SDKNet is created. At the same time, Multiscale Cycle Consistent Loss Function (MS-CCLF) is proposed, which add the Multiscale Structural Similarity Index (MS-SSIM) between input images and reconstructed images. On our image translation dataset (OPT-SAS), the classification accuracy of our SM-CycleGAN is 4.64% higher than that of original CycleGAN. The generator parameters of SM-CycleGAN is 4.13MB lower than that of CycleGAN, and the time consuming of SM-CycleGAN is 0.143s less than that of CycleGAN. The experimental results show that SM -CycleGAN is more suitable for the translation task of optical image to small underwater target synthetic aperture sonar image.

Key words: optical image-to-synthetic aperture sonar image translation, generative adversarial networks, cycle generative adversarial networks, selective dilated kernel networks, multiscale structural similarity index

中图分类号:

TP391

李宝奇, 黄海宁, 刘纪元, 李宇. 基于改进CycleGAN的光学图像迁移生成水下小目标合成孔径声纳图像算法研究[J]. 电子学报, 2021, 49(9): 1746-1753.

LI Bao-qi, HUANG Hai-ning, LIU Ji-yuan, LI Yu. Optical Image-to-Underwater Small Target Synthetic Aperture Sonar Image Translation Algorithm Based on Improved CycleGAN[J]. Acta Electronica Sinica, 2021, 49(9): 1746-1753.

图/表 7

参考文献 28

1	HayesM P, GoughP T. Synthetic aperture sonar: a review of current status[J]. IEEE Journal of Oceanic Engineering, 2009, 34(3): 207 - 224.
2	WangP, ChiC, ZhangY, et al. Fast imaging algorithm for downward-looking 3D synthetic aperture sonars[J]. IET Radar, Sonar and Navigation, 2020, 14(3): 459 - 467.
3	刘纪元,唐劲松,孙宝申,等. 基于回波信号的一种合成孔径声纳运动补偿方法[J]. 电子学报, 2003, 31(1): 131 - 134.
	LIUJ Y, TANGJ S, SUNB S, et al. A receiving-data-based motion compensation method of synthetic aperture sonar[J]. Acta Electronica Sinica, 2003, 31(1): 131 - 134. (in Chinese)
4	SunS B, ChenY C, QinL H, et al. Inverse synthetic aperture sonar imaging of underwater vehicles utilizing 3-D rotations[J]. IEEE Journal of Oceanic Engineering, 2020, 45(2): 563 - 576.
5	LiY, TangS, ZhangR, et al. Asymmetric GAN for unpaired image-to-image translation[J]. IEEE Transactions on Image Processing, 2019, 28(12): 5881 - 5896.
6	LinJ X, XiaY C, QinT, et al. Conditional image-to-image translation[A]. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition[C]. Salt Lake City, USA: Institute of Electrical and Electronics Engineering, 2018. 5524 - 5532.
7	HintonG. Where do features come from? [J]. Cognitive Science, 2014, 38(6):1078 - 1101.
8	LeCunY, BengioY, HintonG. Deep learning[J]. Nature, 2016, 521(7553): 436 - 444.
9	SchmidhuberJ. Deep learning in neural networks: an overview[J]. Neural Networks, 2015, 61:85 - 117.
10	贺昱曜, 李宝奇. 一种组合型的深度学习模型学习率策略[J]. 自动化学报, 2016, 42(6): 953 - 958.
	HeY Y, LiB Q. A combinatory form learning rate scheduling for deep learning model[J]. Acta Automatica Sinica, 2016, 42(6): 953 - 958.(in Chinese)
11	GoodfellowI J, Pouget-AbadieJ, MirzaM, et al. Generative adversarial networks[J]. Advances in Neural Information Processing Systems, 2014, 3: 2672 - 2680.
12	RadfordA, MetzL. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks[EB/OL]. , 2020-07-14.
13	ArjovskyM, ChintalaS, BottouL. Wasserstein GAN[EB/OL]. , 2020-07-14
14	ChenX, DuanY, HouthooftR, et al. InfoGAN: interpretable representation learning by information maximizing generative adversarial nets[A]. Proceedings of the 2016 Advances in Neural Information Processing Systems [C]. Barcelona, Spain: Neural Information Processing Systems, 2016. 2172 - 2180.
15	XuQ T, HuangG, YuanY, et al. An Empirical Study on Evaluation Metrics of Generative Adversarial Networks[EB/OL]. , 2020-07-14.
16	IsolaP, ZhuJ Y, ZhouT, et al. Image⁃to⁃image translation with conditional adversarial networks[A]. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition[C]. Honolulu, HI, USA: Institute of Electrical and Electronics Engineering, 2017. 5967 - 5976.
17	ZhuJ Y, ParkT, IsolaP, et al. Unpaired image⁃to⁃image translation using cycle-consistent adversarial networks[A]. Proceedings of the 2017 IEEE International Conference on Computer Vision[C]. Venice, Italy: IEEE, 2017. 2242 - 2251.
18	HeK M, ZhangX Y, RenS Q, et al. Deep residual learning for image recognition[A]. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition[C]. Las Vegas, NV, USA: IEEE, 2016. 770 - 778.
19	XieS N, GirshickR, DollárP, et al. Aggregated residual transformations for deep neural networks[A]. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition[C]. Honolulu, HI, USA: IEEE,2017. 5987 - 5995.
20	SzegedyC, IoffeS, VanhouckeV, et al. Inception-v4, inception-ResNet and the impact of residual connections on learning[A]. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence[C]. San Francisco California USA: AAAI Press,2016. 4278 - 4284
21	LiX, WangW H, HuX L, et al. Selective kernel networks[A]. Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition[C]. Long Beach, CA, USA: IEEE, 2019. 510 - 519.
22	HuangX J, WenL W, DingJ S. SAR and optical image registration method based on improved CycleGAN[A]. Proceedings of the 2019 Asia-Pacific Conference on Synthetic Aperture Radar[C]. Xiamen, China: IEEE, 2019. 1 - 6.
23	李宝奇, 贺昱曜, 强伟, 等. 基于并行附加特征提取网络的SSD地面小目标检测模型[J]. 电子学报, 2020, 48(1): 84 - 91.
	LiB Q, HeY Y, QiangW, et al. SSD with parallel additional feature extraction network for ground small target detection[J]. Acta Electronica Sinica, 2020, 48(1) 84 - 91. (in Chinese)
24	HowardA G, ZhuM L, ChenB, et al. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications [EB/OL]. , 2020-07-14.
25	ChenL C, PapandreouG, KokkinosI, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834 - 848.
26	WangP Q, ChenP F, YuanY, et al. Understanding convolution for semantic segmentation[A]. Proceedings of the 2018 IEEE Conference on Applications of Computer Vision [C]. Lake Tahoe, NV: IEEE, 2018. 1451 - 1460.
27	WangZ, LiQ. Information content weighting for perceptual image quality assessment[J]. IEEE Transactions on Image Processing, 2011, 20(5): 1185 - 1198.
28	ChoiY, ChoiM, KimM, et al. StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-image Translation[EB/OL]. , 2020-07-14.

	分类准确率（%）	参数大小（MB）	运算时间（s）
CycleGAN	89.83	10.97	0.154
SS-CycleGAN	89.63	10.97	0.154
SK-CycleGAN	92.54	18.21	0.026
SM-CycleGAN	94.47	6.84	0.011

	分类准确率（%）	参数大小（MB）	运算时间（s）
CycleGAN	89.83	10.97	0.154
SS-CycleGAN	89.63	10.97	0.154
SK-CycleGAN	92.54	18.21	0.026
SM-CycleGAN	94.47	6.84	0.011

	生成器网络	结构相似性	分类准确率（%）
CycleGAN	RENet	None	89.83
CycleGAN 变体	SKNet	None	92.54
	SDKNet	None	90.64
	RENet	SSIM	89.63
	SKNet	SSIM	91.46
	SDKNet	SSIM	90.65
	RENet	MS-CCLF	91.94
	SKNet	MS-CCLF	94.71
SM-CycleGAN	SDKNet	MS-CCLF	94.47

	生成器网络	结构相似性	分类准确率（%）
CycleGAN	RENet	None	89.83
CycleGAN 变体	SKNet	None	92.54
	SDKNet	None	90.64
	RENet	SSIM	89.63
	SKNet	SSIM	91.46
	SDKNet	SSIM	90.65
	RENet	MS-CCLF	91.94
	SKNet	MS-CCLF	94.71
SM-CycleGAN	SDKNet	MS-CCLF	94.47

[1]	吴靖, 叶晓晶, 黄峰, 陈丽琼, 王志锋, 刘文犀. 基于深度学习的单帧图像超分辨率重建综述[J]. 电子学报, 2022, 50(9): 2265-2294.
[2]	裴炤, 邱文涛, 王淼, 马苗, 张艳宁. 基于Transformer动态场景信息生成对抗网络的行人轨迹预测方法[J]. 电子学报, 2022, 50(7): 1537-1547.
[3]	刘文杰, 赵胶胶, 张颖, 葛业波. 一种量子条件生成对抗网络算法[J]. 电子学报, 2022, 50(7): 1586-1593.
[4]	孙晨峰, 吕卫民, 戴洪德, 张浩晨. 一种基于TimeGAN和OCSVM的多元退化设备小子样数据增广方法[J]. 电子学报, 2022, 50(11): 2678-2687.
[5]	席亮, 刘涵, 樊好义, 张凤斌. 基于深度对抗学习潜在表示分布的异常检测模型[J]. 电子学报, 2021, 49(7): 1257-1265.
[6]	陈星宇, 叶锋, 黄添强, 翁彬, 陈家祯, 林文忠. 融合小型深度生成模型的显著性检测[J]. 电子学报, 2021, 49(4): 768-774.
[7]	吴文, 万毅. 基于低尺度细节恢复的单幅图像阴影去除方法[J]. 电子学报, 2020, 48(7): 1293-1302.
[8]	王格格, 郭涛, 余游, 苏菡. 基于生成对抗网络的无监督域适应分类模型[J]. 电子学报, 2020, 48(6): 1190-1197.
[9]	朱克凡, 王杰贵, 刘有军. 小样本条件下基于数据增强和WACGAN的雷达目标识别算法[J]. 电子学报, 2020, 48(6): 1124-1131.
[10]	江泽涛, 覃露露. 一种基于U-Net生成对抗网络的低照度图像增强方法[J]. 电子学报, 2020, 48(2): 258-264.
[11]	方晨, 郭渊博, 王娜, 甄帅辉, 唐国栋. 基于生成对抗网络的差分隐私数据发布方法[J]. 电子学报, 2020, 48(10): 1983-1992.

基于改进CycleGAN的光学图像迁移生成水下小目标合成孔径声纳图像算法研究

Optical Image-to-Underwater Small Target Synthetic Aperture Sonar Image Translation Algorithm Based on Improved CycleGAN

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 7

参考文献 28

相关文章 11

编辑推荐

Metrics

	光学图像	合成孔径声纳图像
合计	148	16
圆形目标	76	8
圆柱形目标	72	8

	光学图像	合成孔径声纳图像
合计	148	16
圆形目标	76	8
圆柱形目标	72	8