• •
刘金平, 吴娟娟, 张荣, 徐鹏飞
收稿日期:
2022-04-07
修回日期:
2022-11-22
出版日期:
2023-01-28
通讯作者:
作者简介:
基金资助:
LIU Jin-ping, WU Juan-juan, ZHANG Rong, XU Peng-fei
Received:
2022-04-07
Revised:
2022-11-22
Online:
2023-01-28
Corresponding author:
Supported by:
摘要:
基于深度学习模型的胸部CT(Computed Tomography)图像自动分割有助于辅助医生诊疗.但随着网络宽度与深度的加深,网络训练困难且推理减慢.为提高隐藏层的学习能力,深度监督机制被用于网络训练.但以往的深度监督方法没有考虑模型中多尺度特征图的分层表示以及上采样对参与损失计算的特征图质量的影响.为加强隐藏层学习过程的直接性同时加快网络推理,本文提出一种结构重参数化与多尺度深度监督分割网络(Structural Reparameterization and Multi-Scale Deep Supervision Network,SR&MDS-Net),以实现COVID-19(COrona VIrus Disease 2019)胸部CT图像的高效准确分割.首先构建一种结构重参数化特征变异(Structure Reparameterized FeatureVariation,SRFV)模块将网络的训练与推理进行解耦,在提高模型表达能力的同时加快推理速度;然后,提出一种新颖的多尺度深度监督机制,以加强网络监督效果,提高网络性能.在公开的COVID-19胸部CT图像数据集上进行实验,SR&MDS-Net的灵敏度、特异性、准确率、Dice分别达到了91.5%、99.5%、72.8%、80.1%,与同类其他方法比较,具有更优的性能.
中图分类号:
刘金平, 吴娟娟, 张荣, 徐鹏飞. 基于结构重参数化与多尺度深度监督的COVID-19胸部CT图像自动分割[J]. 电子学报, DOI: 10.12263/DZXB.20220368.
LIU Jin-ping, WU Juan-juan, ZHANG Rong, XU Peng-fei. Toward Automated Segmentation of COVID-19 Chest CT Images Based on Structural Reparameterization and Multi-Scale Deep Supervision[J]. Acta Electronica Sinica, DOI: 10.12263/DZXB.20220368.
数据集 | CT切片数量/张 | 病例数/个 | 图像大小/px |
---|---|---|---|
COVID-19 Dataset | 100 | 50 | 512🞩512 |
MosMedData | 785 | ~60 | 512🞩512 |
Jun数据集 | 1844 | 20 | 512🞩512 630🞩401 630🞩630 |
Our | 2792 | ~130 | 512🞩512 |
表1 数据集描述
数据集 | CT切片数量/张 | 病例数/个 | 图像大小/px |
---|---|---|---|
COVID-19 Dataset | 100 | 50 | 512🞩512 |
MosMedData | 785 | ~60 | 512🞩512 |
Jun数据集 | 1844 | 20 | 512🞩512 630🞩401 630🞩630 |
Our | 2792 | ~130 | 512🞩512 |
网络结构 | 参数/M | 速度/fps |
---|---|---|
训练网络 | 19.79 | 10.69 |
推理网络 | 17.81 | 18.29 |
表2 网络图像处理效率对比
网络结构 | 参数/M | 速度/fps |
---|---|---|
训练网络 | 19.79 | 10.69 |
推理网络 | 17.81 | 18.29 |
注意力机制 | 灵敏度 | 特异性 | 准确率 | DICE |
---|---|---|---|---|
SENet[ | 92.2 | 99.5 | 71.4 | 79.9 |
SGE[ | 92.4 | 99.5 | 71.2 | 79.8 |
CBAM[ | 91.4 | 99.5 | 71.7 | 79.5 |
PSA[ | 93.2 | 99.4 | 70.5 | 79.0 |
SR&MDS-Net | 91.5 | 99.5 | 72.8 | 80.1 |
表3 不同注意力机制对比结果 %
注意力机制 | 灵敏度 | 特异性 | 准确率 | DICE |
---|---|---|---|---|
SENet[ | 92.2 | 99.5 | 71.4 | 79.9 |
SGE[ | 92.4 | 99.5 | 71.2 | 79.8 |
CBAM[ | 91.4 | 99.5 | 71.7 | 79.5 |
PSA[ | 93.2 | 99.4 | 70.5 | 79.0 |
SR&MDS-Net | 91.5 | 99.5 | 72.8 | 80.1 |
上采样方法 | 灵敏度 | 特异性 | 准确率 | DICE |
---|---|---|---|---|
最近邻插值 | 81.2 | 99.5 | 72.4 | 74.6 |
双线性插值 | 92.6 | 99.4 | 71.8 | 79.9 |
转置卷积 | 88.1 | 99.5 | 71.3 | 77.2 |
反池化 | 91.5 | 99.4 | 68.3 | 76.5 |
SR&MDS-Net | 91.5 | 99.5 | 72.8 | 80.1 |
表4 上采样方式对比结果 %
上采样方法 | 灵敏度 | 特异性 | 准确率 | DICE |
---|---|---|---|---|
最近邻插值 | 81.2 | 99.5 | 72.4 | 74.6 |
双线性插值 | 92.6 | 99.4 | 71.8 | 79.9 |
转置卷积 | 88.1 | 99.5 | 71.3 | 77.2 |
反池化 | 91.5 | 99.4 | 68.3 | 76.5 |
SR&MDS-Net | 91.5 | 99.5 | 72.8 | 80.1 |
组件 | 灵敏度 | 特异性 | 准确率 | DICE |
---|---|---|---|---|
UNet | 90.8 | 99.2 | 64.5 | 74.0 |
Unet+SRFV-CA | 89.2 | 99.4 | 67.7 | 75.8 |
Unet+SRFV | 90.6 | 99.3 | 67.7 | 76.0 |
SR&MDS-Net | 91.5 | 99.5 | 72.8 | 80.1 |
表5 消融实验 %
组件 | 灵敏度 | 特异性 | 准确率 | DICE |
---|---|---|---|---|
UNet | 90.8 | 99.2 | 64.5 | 74.0 |
Unet+SRFV-CA | 89.2 | 99.4 | 67.7 | 75.8 |
Unet+SRFV | 90.6 | 99.3 | 67.7 | 76.0 |
SR&MDS-Net | 91.5 | 99.5 | 72.8 | 80.1 |
BN | 灵敏度 | 特异性 | 准确率 | DICE |
---|---|---|---|---|
Inner BN +CE | 89.7 | 99.4 | 72.1 | 77.8 |
InnerBN+CE+Dice | 88.4 | 99.5 | 73.3 | 78.0 |
InnerBN+CE+0.5Dice | 91.5 | 99.5 | 72.8 | 80.1 |
ExternalBN+CE | 90.8 | 99.4 | 70.2 | 77.8 |
ExternalBN+CE+Dice | 89.7 | 99.4 | 68.5 | 76.5 |
ExternalBN+CE+0.5Dice | 88.1 | 99.4 | 72.2 | 76.9 |
表6 BN层位置消融对比 %
BN | 灵敏度 | 特异性 | 准确率 | DICE |
---|---|---|---|---|
Inner BN +CE | 89.7 | 99.4 | 72.1 | 77.8 |
InnerBN+CE+Dice | 88.4 | 99.5 | 73.3 | 78.0 |
InnerBN+CE+0.5Dice | 91.5 | 99.5 | 72.8 | 80.1 |
ExternalBN+CE | 90.8 | 99.4 | 70.2 | 77.8 |
ExternalBN+CE+Dice | 89.7 | 99.4 | 68.5 | 76.5 |
ExternalBN+CE+0.5Dice | 88.1 | 99.4 | 72.2 | 76.9 |
Method | 灵敏度 | 特异性 | 准确率 | DICE |
---|---|---|---|---|
Miniseg[ | 91.0 | 99.5 | 72.8 | 79.6 |
SAUNet++[ | 80.8 | 99.3 | 66.9 | 71.3 |
D2A U-Net[ | 90.0 | 99.3 | 68.9 | 75.7 |
TFCNs[ | 73.0 | 99.5 | 70.1 | 68.6 |
SR&MDS-Net | 91.5 | 99.5 | 72.8 | 80.1 |
表7 SR&MDS-Net与先进COVID-19分割模型对比 %
Method | 灵敏度 | 特异性 | 准确率 | DICE |
---|---|---|---|---|
Miniseg[ | 91.0 | 99.5 | 72.8 | 79.6 |
SAUNet++[ | 80.8 | 99.3 | 66.9 | 71.3 |
D2A U-Net[ | 90.0 | 99.3 | 68.9 | 75.7 |
TFCNs[ | 73.0 | 99.5 | 70.1 | 68.6 |
SR&MDS-Net | 91.5 | 99.5 | 72.8 | 80.1 |
Method | 灵敏度/% | 特异性/% | 准确率/% | DICE/% | FLOPs/G | 参数量/M |
---|---|---|---|---|---|---|
Segnet[ | 86.7 | 99.0 | 57.6 | 67.7 | 160.56 | 29.44 |
PSPNet[ | 87.7 | 99.3 | 66.5 | 73.2 | 159.48 | 27.50 |
Attention U-Net[ | 88.5 | 99.3 | 65.1 | 74.2 | 333.53 | 40.11 |
R2U-Net[ | 83.7 | 98.4 | 48.6 | 59.5 | 611.69 | 39.09 |
Unet3+[ | 90.6 | 99.2 | 63.7 | 73.2 | 799.72 | 26.97 |
Unet3+_DeepSupvise[ | 92.7 | 99.2 | 64.1 | 74.6 | 800.26 | 27.01 |
SR&MDS-Net | 91.5 | 99.5 | 72.8 | 80.1 | 157.98 | 17.81 |
表8 SR&MDS-Net与传统模型对比
Method | 灵敏度/% | 特异性/% | 准确率/% | DICE/% | FLOPs/G | 参数量/M |
---|---|---|---|---|---|---|
Segnet[ | 86.7 | 99.0 | 57.6 | 67.7 | 160.56 | 29.44 |
PSPNet[ | 87.7 | 99.3 | 66.5 | 73.2 | 159.48 | 27.50 |
Attention U-Net[ | 88.5 | 99.3 | 65.1 | 74.2 | 333.53 | 40.11 |
R2U-Net[ | 83.7 | 98.4 | 48.6 | 59.5 | 611.69 | 39.09 |
Unet3+[ | 90.6 | 99.2 | 63.7 | 73.2 | 799.72 | 26.97 |
Unet3+_DeepSupvise[ | 92.7 | 99.2 | 64.1 | 74.6 | 800.26 | 27.01 |
SR&MDS-Net | 91.5 | 99.5 | 72.8 | 80.1 | 157.98 | 17.81 |
1 | 孙捷, 张永梅. 基于动静态特征双输入神经网络的咳嗽声诊断新冠肺炎算法[J/OL]. 电子学报, 2022: 1-11. DOI: 10.12263/DZXB.20211630 . |
SUN Jie, ZHANG Yong-mei. A dynamic-static dual input deep neural network algorithm for diagnosing COVID-19 by cough[J/OL]. Acta Electronica Sinica, 2022:1-11 DOI: 10.12263/DZXB.20211630. (in Chinese) | |
2 | SHI F, WANG J, SHI J, et al. Review of artificial intelligence techniques in imaging data acquisition, segmentation, and diagnosis for COVID-19[J]. IEEE Reviews in Biomedical Engineering, 2021, 14: 4-15. |
3 | RONNEBERGER O, FISCHER P, BROX T. U-Net: Convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham: Springer, 2015: 234-241. |
4 | FAN D P, ZHOU T, JI G P, et al. Inf-Net: Automatic COVID-19 lung infection segmentation from CT images[J]. IEEE Transactions on Medical Imaging, 2020, 39(8): 2626-2637. |
5 | PALURU N, DAYAL A, JENSSEN H B, et al. Anam-Net: Anamorphicdepth embedding-based lightweight CNN for segmentation of anomalies in COVID-19 chest CT images[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(3): 932-946. |
6 | GAO K, SU J, JIANG Z, et al. Dual-branch combination network(DCN): Towards accurate diagnosis and lesion segmentation of COVID-19 using CT images[J]. Medical Image Analysis, 2021, 67: 101836. |
7 | WANG X, YUAN Y, GUO D, et al. SSA-Net: Spatial self-attention network for COVID-19 pneumonia infection segmentation with semi-supervised few-shot learning[J]. Medical Image Analysis, 2022, 79: 102459. |
8 | LIU J, DONG B, WANG S, et al. COVID-19 lung infection segmentation with a novel two-stage cross-domain transfer learning framework[J]. Medical Image Analysis, 2021, 74: 102205. |
9 | ZHU Q, DU B, TURKBEY B, et al. Deeply-supervised CNN for prostate segmentation[C]//2017 International Joint Conference on Neural Networks(IJCNN). Anchorage:IEEE, 2017: 178-184. |
10 | 陆倩杰, 柏正尧, 樊圣澜,等. COVID-19肺部CT 图像多尺度编解码分割[J]. 中国图象图形学报, 2022, 27(3): 827-837. |
LU Qian-jie, BAI Zheng-yao, FANG Sheng-lan, et al. Multiscalecodec network based CT image segmentation for human lung disease derived of COVID-19[J]. Journal of Image and Graphics, 2022, 27(3): 827-837. (in Chinese) | |
11 | MAJI D, SIGEDAR P, SINGH M. Attention Res-UNet with guided decoder for semantic segmentation of brain tumors[J]. Biomedical Signal Processing and Control, 2022, 71: 103077. |
12 | SHELHAMER E, LONG J, DARRELL T. Fully convolutionalnetworks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651. |
13 | DING X, ZHANG X, MA N, et al. RepVGG: Making VGG-style convnets great again[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Nashville: IEEE, 2021: 13733-13742. |
14 | HOU Q, ZHOU D, FENG J. Coordinate attention for efficient mobile network design[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Nashville: IEEE, 2021: 13713-13722. |
15 | CAO J, LIY, SUN M, et al. DO-Conv: Depthwise over-parameterized convolutional layer[J]. IEEE Transactions on Image Processing, 2022, 31: 3726-3736. |
16 | DING X, GUO Y, DING G, et al. ACNET: Strengthening the kernel skeletons for powerful cnn via asymmetric convolution blocks[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 1911-1920. |
17 | MOROZOV S, ANDREYCHENKO A, PAVLOV N, et al. Mosmeddata: Chest CT scans with COVID-19 related findings dataset[EB/OL]. (2020-05-13)[2022-04-07]. . |
18 | JUN M, CHENG G, YIXIN W, et al. COVID-19 CT lung andinfection segmentation dataset(Verson 1.0)[DS/OL]. (2020-04-20)[2022-04-07]. . |
19 | HUJ, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011-2023. |
20 | LI X, HU X, YANG J. Spatial group-wise enhance: Improving semantic feature learning in convolutional networks[EB/OL]. (2019-05-25)[2022-04-07]. . |
21 | WOO S, PARK J, LEE J, et al. CBAM: Convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision(ECCV). Cham: Springer, 2018: 3-19. |
22 | LIU H, LIU F, FAN X, et al. Polarized self-attention: Towards high-quality pixel-wise regression[EB/OL]. (2021-07-08)[2022-04-07]. . |
23 | QIU Y, LIU Y, LI S, et al.MiniSeg: An Extremely Minimum Network for Efficient COVID-19 Segmentation[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(6): 4846-4854. |
24 | XIAO H, RAN Z, MABU S, et al. SAUNet++: An automatic segmentation model of COVID-19 lesion from CT slices[J/OL]. The Visual Computer, 2022: 1-14. : . |
25 | ZHAO X, ZHANG P, SONG F, et al. D2A U-Net: Automatic segmentation of COVID-19 CT slices based on dual attention and hybrid dilated convolution[J]. Computers in Biology and Medicine, 2021, 135: 104526. |
26 | LI Z, LI D, XU C, et al. TFCNs: A CNN-Transformer hybrid network for medical image segmentation[C]//Artificial Neural Networks and Machine Learning-ICANN 2022. Cham: Springer, 2022: 781-792. |
27 | BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet:A deep convolutional encoder-decoder architecture for image segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495. |
28 | ZHAO H, SHI J, QI X, et al. Pyramid scene parsing network[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Honolulu: IEEE, 2017: 6230-6239. |
29 | OKTAY O, SCHLEMPER J, FOLGOC L, et al. Attention U-Net: Learning where to look for the pancreas[EB/OL]. (2018-05-20)[2022-04-07]. . |
30 | ALOM M Z, HASAN M, YAKOPCIC C, et al. Recurrent residual convolutional neural network based on U-Net (R2U-Net) for medical image segmentation[EB/OL]. (2018-05-29)[2022-04-07]. . |
31 | HUANG H, LIN L, TONG R, et al. Unet 3+: A full-scale connected unet for medical image segmentation[C]//ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP).Barcelona: IEEE, 2020: 1055-1059. |
[1] | 吴靖, 叶晓晶, 黄峰, 陈丽琼, 王志锋, 刘文犀. 基于深度学习的单帧图像超分辨率重建综述[J]. 电子学报, 2022, 50(9): 2265-2294. |
[2] | 李雪莹, 王田路, 梁鹏, 王翀. 基于系统模型的用户评论中非功能需求的自动分类[J]. 电子学报, 2022, 50(9): 2079-2089. |
[3] | 琚长瑞, 秦晓燕, 袁广林, 李豪, 朱虹. 尺度敏感损失与特征融合的快速小目标检测方法[J]. 电子学报, 2022, 50(9): 2119-2126. |
[4] | 张志昌, 于沛霖, 庞雅丽, 朱林, 曾扬扬. SMGN:用于对话状态跟踪的状态记忆图网络[J]. 电子学报, 2022, 50(8): 1851-1858. |
[5] | 张亚洲, 俞洋, 朱少林, 陈锐, 戎璐, 梁辉. 一种量子概率启发的对话讽刺识别网络模型[J]. 电子学报, 2022, 50(8): 1885-1893. |
[6] | 王飞扬, 冀鹏欣, 孙笠, 危倩, 李根, 张忠宝. 一种基于深度学习的动态社交网络用户对齐方法[J]. 电子学报, 2022, 50(8): 1925-1936. |
[7] | 徐兴荣, 刘聪, 李婷, 郭娜, 任崇广, 曾庆田. 基于双向准循环神经网络和注意力机制的业务流程剩余时间预测方法[J]. 电子学报, 2022, 50(8): 1975-1984. |
[8] | 欧阳与点, 谢鲲, 谢高岗, 文吉刚. 面向大规模网络测量的数据恢复算法:基于关联学习的张量填充[J]. 电子学报, 2022, 50(7): 1653-1663. |
[9] | 裴炤, 邱文涛, 王淼, 马苗, 张艳宁. 基于Transformer动态场景信息生成对抗网络的行人轨迹预测方法[J]. 电子学报, 2022, 50(7): 1537-1547. |
[10] | 彭闯, 王伦文, 胡炜林. 融合深度特征的电磁频谱异常检测算法[J]. 电子学报, 2022, 50(6): 1359-1369. |
[11] | 杨伟超, 杜宇, 文伟, 侯舒维, 徐常志, 张建华. 基于多重分形谱智能分析的卫星信号调制识别研究[J]. 电子学报, 2022, 50(6): 1336-1343. |
[12] | 李政伟, 李佳树, 尤著宏, 聂茹, 赵欢, 钟堂波. 基于异质图注意力网络的miRNA与疾病关联预测算法[J]. 电子学报, 2022, 50(6): 1428-1435. |
[13] | 张波, 陆云杰, 秦东明, 邹国建. 一种卷积自编码深度学习的空气污染多站点联合预测模型[J]. 电子学报, 2022, 50(6): 1410-1427. |
[14] | 廖勇, 李玉杰. 一种轻量化低复杂度的FDD大规模MIMO系统CSI反馈方法[J]. 电子学报, 2022, 50(5): 1211-1217. |
[15] | 冀振燕, 韩梦豪, 宋晓军, 冯其波. 面向激光光条图像修复的循环相似度映射网络[J]. 电子学报, 2022, 50(5): 1234-1242. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||