面向医学图像生成的鲁棒条件生成对抗网络

刘少鹏; 赵慧民; 洪佳明; 吴晓航; 许发宝; 欧阳佳; 梁鹏; 熊建斌

doi:10.12263/DZXB.20210051

您当前的位置：

首页 >

文章列表页 >

面向医学图像生成的鲁棒条件生成对抗网络

学术论文 | 更新时间：2025-12-08

- 面向医学图像生成的鲁棒条件生成对抗网络
- Medical Image Synthesis Using Robust Conditional GAN
- 电子学报 2023年51卷第2期页码：427-437
- 作者机构：
  
  1.广东技术师范大学计算机科学学院，广东广州 510665
  2.广州大学管理学院，广东广州 510006
  3.广东省大数据分析与处理重点实验室，广东广州 510006
  4.广州中医药大学医学信息工程学院，广东广州 510006
  5.中山大学中山眼科中心眼科学国家重点实验室，广东广州 510060
  6.山东大学齐鲁医院眼科，山东济南 250012
  7.广东技术师范大学网络空间安全学院，广东广州 510665
  8.广东技术师范大学自动化学院，广东广州 510665
- 作者简介：
  
  [ "刘少鹏男，1984年生，广东人.2013获得中山大学工学博士学位.现为广大技术师范大学计算机科学学院讲师、研究生导师.主要研究方向为人工智能和医学图像处理.E-mail: 149265005@qq.com" ]
  [ "赵慧民（通讯作者）男，1966年生，陕西人.博士（后），教授.广东省优势重点一级学科控制科学与工程学科带头人，中国电子学会高级会员，广东省计算机学会常务理事，广东省图像图形学会常任理事，广东省网络安全协会副主任，英国University of Strathclyde多媒体信息处理合作博士生导师.从事数字信号处理与信息安全技术研究与教学工作.近5年主持国家自然基金面上项目3项，主持省自然基金重点项目以及省重大专项项目10余项，在高质量期刊发表有关的研究论文60余篇（其中40篇被SCI/EI检索），授权发明专利13项.获得2015年、2018年和2021年广东省科学技术奖." ]
- 基金信息：
  
  国家自然科学基金(62072122;62072123;62073090);广东省自然科学基金(2019A1515012048;2019A1515010700);广东省省级科技计划项目(2021A0505030074);广东省大数据分析与处理重点实验室开放基金(202001;202202)
- DOI：10.12263/DZXB.20210051
  中图分类号： TN957;
- 收稿：2020-12-31，
  
  修回：2022-02-20，
  
  纸质出版：2023-02-25
- 稿件说明：
移动端阅览
刘少鹏,赵慧民,洪佳明等.面向医学图像生成的鲁棒条件生成对抗网络[J].电子学报,2023,51(02):427-437.

LIU Shao-peng,ZHAO Hui-min,HONG Jia-ming,et al.Medical Image Synthesis Using Robust Conditional GAN[J].ACTA ELECTRONICA SINICA,2023,51(02):427-437.
刘少鹏,赵慧民,洪佳明等.面向医学图像生成的鲁棒条件生成对抗网络[J].电子学报,2023,51(02):427-437. DOI： 10.12263/DZXB.20210051.

LIU Shao-peng,ZHAO Hui-min,HONG Jia-ming,et al.Medical Image Synthesis Using Robust Conditional GAN[J].ACTA ELECTRONICA SINICA,2023,51(02):427-437. DOI： 10.12263/DZXB.20210051.

摘要

医学图像生成是计算机辅助诊断技术的关键组成，具有广泛的应用场景.当前基于生成对抗网络的端对端学习模型，依靠生成器和判别器两者对抗训练，获取真实数据的概率分布，从而指导图像生成.但标注有限的医学图像及其高分辨率特点，加大了模型训练难度，影响图像生成质量；同时，模型未纳入数据扰动因素，鲁棒性有限，容易被恶意攻击.为此，本文提出一个基于鲁棒条件生成对抗网络的医学图像生成模型——MiSrc-GAN.该模型包括精度渐进生成器、多尺度判别器以及对抗样本配对构造模块，有效融合GAN框架和对抗样本，改善判别器鲁棒性，有利于学习原始图像与待生成图像的联合概率分布.在真实数据集CSC和REFUGE上的实验表明，MiSrc-GAN生成的图像质量优于现有模型.

Abstract

Medical image synthesis is a key technology in computer aided diagnosis due to its wide applications. Recently

the end-to-end models based on generative adversarial networks (GAN) learn the true data distribution to guide the image generation via the competing generator and discriminator. However

the high resolution of the medical images and the lack of labeled samples make these models hard to train. Furthermore

they are not robust against data perturbations and vulnerable to malicious attacks. In this paper

we propose a novel image synthesis model using robust conditional GAN

namely MiSrc-GAN. The MiSrc-GAN contains a progressive resolution generator

a multi-scale discriminator and a paired adversarial examples generating module. Through effective integrating GAN framework and adversarial examples

the MiSrc-GAN is able to simultaneously improve the robustness of the multi-scale discriminator and the quality of synthetic images under the joint probability distribution of the original medical images and their translating versions. The extensive experiments show that the proposed method achieves state-of-the-art image synthesis results on both CSC and REFUGE datasets.

关键词

Keywords

references

郑光远 , 刘峡壁 , 韩光辉 . 医学影像计算机辅助检测与诊断系统综述 [J]. 软件学报 , 2018 , 29 ( 5 ): 1471 - 1514 .

ZHENG G Y , LIU X B , HAN G H . Survey on medical image computer aided detection and diagnosis systems [J]. Journal of Software , 2018 , 29 ( 5 ): 1471 - 1514 . (in Chinese)

YI X , WALIA E , BABYN P . Generative adversarial network in medical imaging: A review [J]. Medical Image Analysis , 2019 , 58 : 101552 .

GOODFELLOW I J , POUGET-ABADIE J , MIRZA M , et al . Generative adversarial nets [C]// NIPS'14: Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2 . Montreal : ACM , 2014 : 2672 - 2680 .

HUANG H , YU P S , WANG C . An introduction to image synthesis with generative adversarial nets [EB/OL]. ( 2018 )[2021]. https://arxiv.org/abs/1803.04469 https://arxiv.org/abs/1803.04469 .

田娟秀 , 刘国才 , 谷珊珊 , 等 . 医学图像分析深度学习方法研究与挑战 [J]. 自动化学报 , 2018 , 44 ( 3 ): 401 - 424 .

TIAN J X , LIU G C , GU S S , et al . Deep learning in medical image analysis and its challenges [J]. Acta Automatica Sinica , 2018 , 44 ( 3 ): 401 - 424 . (in Chinese)

YU B , WANG Y , WANG L , et al . Medical image synthesis via deep learning [J]. Advances in Experimental Medicine and Biology , 2020 , 1213 : 23 - 44 .

KAZEMINIA S , BAUR C , KUIJPER A , et al . GANs for medical image analysis [J]. Artificial Intelligence in Medicine , 2020 , 109 : 101938 .

RADFORD A , METZ L , CHINTALA S . Unsupervised representation learning with deep convolutional generative adversarial networks [EB/OL]. ( 2015 )[2021]. https://arxiv.org/abs/1511.06434 https://arxiv.org/abs/1511.06434 .

DENTON E , CHINTALA S , SZLAM A , et al . Deep generative image models using a Laplacian pyramid of adversarial networks [C]// Advances in Neural Information Processing Systems(NIPS) . Montreal : MIT Press , 2015 : 1486 - 1494 .

MIRZA M , OSINDERO S . Conditional generative adversarial nets [EB/OL]. ( 2014 )[2021]. https://arxiv.org/abs/1411.1784 https://arxiv.org/abs/1411.1784 .

ODENA A , OLAH C , SHLENS J . Conditional image synthesis with auxiliary classifier gans [EB/OL]. ( 2017 )[2021]. https://arxiv.org/abs/1610.09585 https://arxiv.org/abs/1610.09585 .

KARRAS T , AILA T , LAINE S . Progressive growing of gans for improved quality, stability, and variation [EB/OL]. ( 2018 )[2021]. https://arxiv.org/abs/1710.10196 https://arxiv.org/abs/1710.10196 .

WANG T C , LIU M Y , ZHU J Y , et al . High-resolution image synthesis and semantic manipulation with conditional GANs [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE , 2018 : 8798 - 8807 .

SZEGEDY C , ZAREMBA W , SUTSKEVER I , et al . Intriguing properties of neural networks [EB/OL]. ( 2014 )[2021]. https://arxiv.org/abs/1312.6199 https://arxiv.org/abs/1312.6199 .

XIAO C , LI B , ZHU J Y , et al . Generating adversarial examples with adversarial networks [EB/OL]. ( 2018 )[2021]. https://arxiv.org/abs/1801.02610 https://arxiv.org/abs/1801.02610 .

JANDIAL S , MANGLA P , VARSHNEY S , et al . AdvGAN: harnessing latent layers for adversary generation [C]// 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) . Seoul : IEEE , 2019 : 2045 - 2048 .

LIU X Q , HSIEH C J . Rob-GAN: Generator, discriminator, and adversarial attacker [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Long Beach : IEEE , 2019 : 11226 - 11235 .

MADRY A , MAKELOV A , SCHMIDT L , et al . Towards deep learning models resistant to adversarial attacks [EB/OL]. ( 2018 )[2021]. https://arxiv.org/abs/1706.06083 https://arxiv.org/abs/1706.06083 .

JOHNSON J , ALAHI A , LI F F . Perceptual losses for real-time style transfer and super-resolution [C]// Computer Vision - ECCV 2016 . Cham : Springer International Publishing , 2016 : 694 - 711 .

DURUGKAR I , GEMP I M , MAHADEVAN S . Generative multi-adversarial networks [C]// Proceedings of the International Conference on Learning Representations(ICLR) . New York : ACM , 2017 : 1 - 14 .

周沛 , 陈后金 , 于泽宽 , 等 . 跨模态医学图像预测综述 [J]. 电子学报 , 2019 , 47 ( 1 ): 220 - 226 .

ZHOU P , CHEN H J , YU Z K , et al . Review of cross-modality medical image prediction [J]. Acta Electronica Sinica , 2019 , 47 ( 1 ): 220 - 226 . (in Chinese)

BEN-COHEN A , KLANG E , RASKIN S , et al . Cross-modality synthesis from CT to PET using FCN and GAN networks for improved automated lesion detection [EB/OL]. ( 2019 )[2021]. https://arxiv.org/abs/1802.07846 https://arxiv.org/abs/1802.07846 .

SHELHAMER E , LONG J , DARRELL T . Fully convolutional networks for semantic segmentation [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2017 , 39 ( 4 ): 640 - 651 .

PAN Y S , LIU M X , LIAN C F , et al . Synthesizing missing PET from MRI with cycle-consistent generative adversarial networks for alzheimer\u2019s disease diagnosis [C]// Medical Image Computing and Computer Assisted Intervention - MICCAI 2018 . Cham : Springer International Publishing , 2018 : 455 - 463 .

NIE D , TRULLO R , LIAN J , et al . Medical image synthesis with deep convolutional adversarial networks [J]. IEEE Transactions on Biomedical Engineering , 2018 , 65 ( 12 ): 2720 - 2730 .

WOLTERINK J M , DINKLA A M , SAVENIJE M H F , et al . Deep MR to CT synthesis using unpaired data [M]// Simulation and Synthesis in Medical Imaging . Cham : Springer International Publishing , 2017 : 14 - 23 .

ZHU J Y , PARK T , ISOLA P , et al . Unpaired image-to-image translation using cycle-consistent adversarial networks [C]// 2017 IEEE International Conference on Computer Vision . Venice : IEEE , 2017 : 2242 - 2251 .

YANG H R , SUN J , CARASS A , et al . Unsupervised MR-to-CT synthesis using structure-constrained CycleGAN [J]. IEEE Transactions on Medical Imaging , 2020 , 39 ( 12 ): 4249 - 4261 .

DAR S U , YURT M , KARACAN L , et al . Image synthesis in multi-contrast MRI with conditional generative adversarial networks [J]. IEEE Transactions on Medical Imaging , 2019 , 38 ( 10 ): 2375 - 2388 .

ZHOU T , FU H Z , CHEN G , et al . Hi-net: Hybrid-fusion network for multi-modal MR image synthesis [J]. IEEE Transactions on Medical Imaging , 2020 , 39 ( 9 ): 2772 - 2781 .

YOU Q J , WAN C , SUN J , et al . Fundus image enhancement method based on CycleGAN [C]// 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society . Berlin : IEEE , 2019 : 4500 - 4503 .

LI W Y , LIU G X , HE Y , et al . Quality improvement of adaptive optics retinal images using conditional adversarial networks [J]. Biomedical Optics Express , 2020 , 11 ( 2 ): 831 - 849 .

TOM F , SHEET D . Simulating patho-realistic ultrasound images using deep generative networks with adversarial learning [C]// 2018 IEEE 15th International Symposium on Biomedical Imaging . Washington : IEEE , 2018 : 1174 - 1177 .

ZHANG H , XU T , LI H S , et al . StackGAN: text to photo-realistic image synthesis with stacked generative adversarial networks [C]// 2017 IEEE International Conference on Computer Vision . Venice : IEEE , 2017 : 5908 - 5916 .

MA Y H , LIU Y H , CHENG J , et al . Cycle structure and illumination constrained GAN for medical image enhancement [C]// Medical Image Computing and Computer Assisted Intervention - MICCAI 2020 . Cham : Springer International Publishing , 2020 : 667 - 677 .

LIU Y , YANG J , ZHOU Y , et al . Prediction of OCT images of short-term response to anti-VEGF treatment for neovascular age-related macular degeneration using generative adversarial network [J]. British Journal of Ophthalmology , 2020 , 104 ( 12 ): 1735 - 1740 .

BAILO O , HAM D , SHIN Y M . Red blood cell image generation for data augmentation using conditional generative adversarial networks [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . Long Beach : IEEE , 2019 : 1039 - 1048 .

RUAN Y N , LI D W , MARSHALL H , et al . MB-FSGAN: Joint segmentation and quantification of kidney tumor on CT by the multi-branch feature sharing generative adversarial network [J]. Medical Image Analysis , 2020 , 64 : 101721 .

XUE Y , XU T , ZHANG H , et al . SegAN: Adversarial network with multi-scale L1 loss for medical image segmentation [J]. Neuroinformatics , 2018 , 16 ( 3 ): 383 - 392 .

LIU S P , HONG J M , LU X , et al . Joint optic disc and cup segmentation using semi-supervised conditional GANs [J]. Computers in Biology and Medicine , 2019 , 115 : 103485 .

刘少鹏 , 洪佳明 , 梁杰鹏 , 等 . 面向医学图像分割的半监督条件生成对抗网络 [J]. 软件学报 , 2020 , 31 ( 8 ): 2588 - 2602 .

LIU S P , HONG J M , LIANG J P , et al . Medical image segmentation using semi-supervised conditional generative adversarial nets [J]. Journal of Software , 2020 , 31 ( 8 ): 2588 - 2602 . (in Chinese)

WOLTERINK J M , LEINER T , VIERGEVER M A , et al . Generative adversarial networks for noise reduction in low-dose CT [J]. IEEE Transactions on Medical Imaging , 2017 , 36 ( 12 ): 2536 - 2545 .

ZHOU L , SCHAEFFERKOETTER J D , THAM I W K , et al . Supervised learning with cyclegan for low-dose FDG PET image denoising [J]. Medical Image Analysis , 2020 , 65 : 101770 .

ARJOVSKY M , CHINTALA S , BOTTOU L . Wasserstein Gan [EB/OL]. ( 2017 )[2021]. https://arxiv.org/abs/1701.07875 https://arxiv.org/abs/1701.07875 .

YI X , BABYN P . Sharpness-aware low-dose CT denoising using conditional generative adversarial network [J]. Journal of Digital Imaging , 2018 , 31 ( 5 ): 655 - 669 .

袁子晗 , 蒋明峰 , 李杨 , 等 . 基于DAWGAN-GP的磁共振图像重构方法研究 [J]. 电子学报 , 2020 , 48 ( 10 ): 1883 - 1890 .

YUAN Z H , JIANG M F , LI Y , et al . Research of MRI reconstruction method by using de-aliasing Wasserstein generative adversarial networks with gradient penalty [J]. Acta Electronica Sinica , 2020 , 48 ( 10 ): 1883 - 1890 . (in Chinese)

LIU Z C , BICER T , KETTIMUTHU R , et al . TomoGAN: Low-dose synchrotron X-ray tomography with generative adversarial networks: Discussion [J]. Journal of the Optical Society of America A , 2020 , 37 ( 3 ): 422 - 434 .

MA X J , NIU Y H , GU L , et al . Understanding adversarial attacks on deep learning based medical image analysis systems [J]. Pattern Recognition , 2021 , 110 : 107332 .

PAUL R , SCHABATH M , GILLIES R , et al . Mitigating adversarial attacks on medical image understanding systems [C]// 2020 IEEE 17th International Symposium on Biomedical Imaging . Iowa City : IEEE , 2020 : 1517 - 1521 .

HIRANO H , KOGA K , TAKEMOTO K . Vulnerability of deep neural networks for detecting covid-19 cases from chest X-ray images to universal adversarial attacks [EB/OL]. ( 2020 )[2021]. https://arxiv.org/abs/2005.11061 https://arxiv.org/abs/2005.11061 .

ZHAO Z , DUA D , SINGH S . Generating natural adversarial examples [C]// Proceedings of the International Conference on Learning Representations (ICLR) . New York : ACM , 2018 : 1 - 15 .

CISSE M , BOJANOWSKI P , GRAVE E , et al . Parseval networks: Improving robustness to adversarial examples [C]// ICML'17: Proceedings of the 34th International Conference on Machine Learning-Volume 70 . New York : ACM , 2017 : 854 - 863 .

SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition [C]// Proceedings of the International Conference on Learning Representations . New York : ACM , 2015 : 1 - 14 .

罗会兰 , 敖阳 , 袁璞 . 一种生成对抗网络用于图像修复的方法 [J]. 电子学报 , 2020 , 48 ( 10 ): 1891 - 1898 .

LUO H L , AO Y , YUAN P . Image inpainting using generative adversarial networks [J]. Acta Electronica Sinica , 2020 , 48 ( 10 ): 1891 - 1898 . (in Chinese)

ZHANG R , ISOLA P , EFROS A A , et al . The unreasonable effectiveness of deep features as a perceptual metric [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE , 2018 : 586 - 595 .

FU H Z , CHENG J , XU Y W , et al . Joint optic disc and cup segmentation based on multi-label deep network and polar transformation [J]. IEEE Transactions on Medical Imaging , 2018 , 37 ( 7 ): 1597 - 1605 .

SEVASTOPOLSKY A . Optic disc and cup segmentation methods for glaucoma detection with modification of U-Net convolutional neural network [J]. Pattern Recognition and Image Analysis , 2017 , 27 ( 3 ): 618 - 624 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于Diffusion-Mamba和尺度不变损失的渐进式图像生成方法

基于鲁棒对抗防御边界的语音伪造方法识别

SwinT-Unet：基于双通道自注意力机制的超声图像分割方法

针对图像分类的鲁棒物理域对抗伪装