基于结构重参数化与多尺度深度监督的COVID-19胸部CT图像自动分割

doi:10.12263/DZXB.20220368

电子学报

• •

基于结构重参数化与多尺度深度监督的COVID-19胸部CT图像自动分割

刘金平, 吴娟娟, 张荣, 徐鹏飞

湖南师范大学信息科学与工程学院，湖南长沙410006

收稿日期:2022-04-07 修回日期:2022-11-22 出版日期:2023-01-28
- 通讯作者:
- 徐鹏飞
- 作者简介:
- 刘金平　男，1983年9月出生于湖南省邵阳市.现为湖南师范大学信息科学与工程学院教授、博士生导师，主要从事智能信息处理、过程检测、故障诊断等相关研究.E-mail: ljp202518@163.com
  吴娟娟　女，1996年5月出生于湖南省株洲市.目前在湖南师范大学信息科学与工程学院攻读硕士学位.研究方向为生物医学图像处理. E-mail: Ajuan0527@hunnu.edu.cn
  张荣　女，1999年12月出生于湖南省长沙市.目前在湖南师范大学信息科学与工程学院攻读硕士学位.研究方向为生物医学图像处理. E-mail: cpstzr@163.com
  徐鹏飞（通讯作者）　男，1977年9月出生于湖南省岳阳市.现为湖南师范大学信息科学与工程学院副教授，主要从事智能信息处理方面的研究. Email: xupf@hunnu.edu.cn
- 基金资助:
- 国家自然科学基金 (61971188)

Toward Automated Segmentation of COVID-19 Chest CT Images Based on Structural Reparameterization and Multi-Scale Deep Supervision

LIU Jin-ping, WU Juan-juan, ZHANG Rong, XU Peng-fei

College of Information Science and Engineering，Hunan Normal University，Changsha，Hunan 410081，China

Received:2022-04-07 Revised:2022-11-22 Online:2023-01-28
- Corresponding author:
- XU Peng-fei
- Supported by:
- National Natural Science Foundation of China (61971188)

摘要/Abstract

摘要：

基于深度学习模型的胸部CT（Computed Tomography）图像自动分割有助于辅助医生诊疗.但随着网络宽度与深度的加深，网络训练困难且推理减慢.为提高隐藏层的学习能力，深度监督机制被用于网络训练.但以往的深度监督方法没有考虑模型中多尺度特征图的分层表示以及上采样对参与损失计算的特征图质量的影响.为加强隐藏层学习过程的直接性同时加快网络推理，本文提出一种结构重参数化与多尺度深度监督分割网络（Structural Reparameterization and Multi-Scale Deep Supervision Network，SR&MDS-Net），以实现COVID-19（COrona VIrus Disease 2019）胸部CT图像的高效准确分割.首先构建一种结构重参数化特征变异（Structure Reparameterized FeatureVariation，SRFV）模块将网络的训练与推理进行解耦，在提高模型表达能力的同时加快推理速度；然后，提出一种新颖的多尺度深度监督机制，以加强网络监督效果，提高网络性能.在公开的COVID-19胸部CT图像数据集上进行实验，SR&MDS-Net的灵敏度、特异性、准确率、Dice分别达到了91.5%、99.5%、72.8%、80.1%，与同类其他方法比较，具有更优的性能.

关键词: COVID-19, 医学图像分割, 深度学习, U-Net, 结构重参数化, 深度监督学习

Abstract:

Automatic segmentation of chest CT(Computed Tomography) images based on deep learning models is helpful to assist doctors in diagnosis and treatment. However, with the deepening of the network width and depth, the network training is difficult and the inference slows down. In order to improve the learning ability of the hidden layer, the deep supervision mechanism is used in network training. However, previous deep supervision methods did not considerthe layered representation of multi-scale feature maps in the segmentation model and the influence of up-sampling on the quality of feature maps involved in loss calculation. In order to improve the directness of the hidden layer learning process and speed up the reasoning. This article proposes a structural reparameterization and multi-scale deep supervision network(SR&MDS-Net) for the purpose of achieving the efficient and accurate segmentation results of lesions in COVID-19(COrona VIrus Disease 2019) pneumonia chest CT images. Structure reparameterized featurevariation(SRFV) module was constructed to decoupage the network training and reasoning, which improved the model expression ability and accelerated the reasoning speed. In order to strengthen the effect of network supervision and improve the performance of network, a novel multi-scale deep supervision mechanism is proposed. The experiment was carried out on the public COVID-19 pneumonia chest CT image dataset. The sensitivity, specificity, accuracy, and Dice of SR&MDS-Net reached 91.5%, 99.5%, 72.8%, and 80.1%, respectively, which had better performance compared with other similar methods.

Key words: COVID-19, medicalimage segmentation, deep learning, U-Net, structure reparameterization, supervised learning

中图分类号:

TP391.41

刘金平, 吴娟娟, 张荣, 徐鹏飞. 基于结构重参数化与多尺度深度监督的COVID-19胸部CT图像自动分割[J]. 电子学报, DOI: 10.12263/DZXB.20220368.

LIU Jin-ping, WU Juan-juan, ZHANG Rong, XU Peng-fei. Toward Automated Segmentation of COVID-19 Chest CT Images Based on Structural Reparameterization and Multi-Scale Deep Supervision[J]. Acta Electronica Sinica, DOI: 10.12263/DZXB.20220368.

图/表 14

参考文献 31

1	孙捷, 张永梅. 基于动静态特征双输入神经网络的咳嗽声诊断新冠肺炎算法[J/OL]. 电子学报, 2022: 1-11. DOI: 10.12263/DZXB.20211630 .
	SUN Jie, ZHANG Yong-mei. A dynamic-static dual input deep neural network algorithm for diagnosing COVID-19 by cough[J/OL]. Acta Electronica Sinica, 2022:1-11 DOI: 10.12263/DZXB.20211630. (in Chinese)
2	SHI F, WANG J, SHI J, et al. Review of artificial intelligence techniques in imaging data acquisition, segmentation, and diagnosis for COVID-19[J]. IEEE Reviews in Biomedical Engineering, 2021, 14: 4-15.
3	RONNEBERGER O, FISCHER P, BROX T. U-Net: Convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham: Springer, 2015: 234-241.
4	FAN D P, ZHOU T, JI G P, et al. Inf-Net: Automatic COVID-19 lung infection segmentation from CT images[J]. IEEE Transactions on Medical Imaging, 2020, 39(8): 2626-2637.
5	PALURU N, DAYAL A, JENSSEN H B, et al. Anam-Net: Anamorphicdepth embedding-based lightweight CNN for segmentation of anomalies in COVID-19 chest CT images[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(3): 932-946.
6	GAO K, SU J, JIANG Z, et al. Dual-branch combination network(DCN): Towards accurate diagnosis and lesion segmentation of COVID-19 using CT images[J]. Medical Image Analysis, 2021, 67: 101836.
7	WANG X, YUAN Y, GUO D, et al. SSA-Net: Spatial self-attention network for COVID-19 pneumonia infection segmentation with semi-supervised few-shot learning[J]. Medical Image Analysis, 2022, 79: 102459.
8	LIU J, DONG B, WANG S, et al. COVID-19 lung infection segmentation with a novel two-stage cross-domain transfer learning framework[J]. Medical Image Analysis, 2021, 74: 102205.
9	ZHU Q, DU B, TURKBEY B, et al. Deeply-supervised CNN for prostate segmentation[C]//2017 International Joint Conference on Neural Networks(IJCNN). Anchorage:IEEE, 2017: 178-184.
10	陆倩杰, 柏正尧, 樊圣澜,等. COVID-19肺部CT 图像多尺度编解码分割[J]. 中国图象图形学报, 2022, 27(3): 827-837.
	LU Qian-jie, BAI Zheng-yao, FANG Sheng-lan, et al. Multiscalecodec network based CT image segmentation for human lung disease derived of COVID-19[J]. Journal of Image and Graphics, 2022, 27(3): 827-837. (in Chinese)
11	MAJI D, SIGEDAR P, SINGH M. Attention Res-UNet with guided decoder for semantic segmentation of brain tumors[J]. Biomedical Signal Processing and Control, 2022, 71: 103077.
12	SHELHAMER E, LONG J, DARRELL T. Fully convolutionalnetworks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651.
13	DING X, ZHANG X, MA N, et al. RepVGG: Making VGG-style convnets great again[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Nashville: IEEE, 2021: 13733-13742.
14	HOU Q, ZHOU D, FENG J. Coordinate attention for efficient mobile network design[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Nashville: IEEE, 2021: 13713-13722.
15	CAO J, LIY, SUN M, et al. DO-Conv: Depthwise over-parameterized convolutional layer[J]. IEEE Transactions on Image Processing, 2022, 31: 3726-3736.
16	DING X, GUO Y, DING G, et al. ACNET: Strengthening the kernel skeletons for powerful cnn via asymmetric convolution blocks[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 1911-1920.
17	MOROZOV S, ANDREYCHENKO A, PAVLOV N, et al. Mosmeddata: Chest CT scans with COVID-19 related findings dataset[EB/OL]. (2020-05-13)[2022-04-07]. .
18	JUN M, CHENG G, YIXIN W, et al. COVID-19 CT lung andinfection segmentation dataset(Verson 1.0)[DS/OL]. (2020-04-20)[2022-04-07]. .
19	HUJ, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011-2023.
20	LI X, HU X, YANG J. Spatial group-wise enhance: Improving semantic feature learning in convolutional networks[EB/OL]. (2019-05-25)[2022-04-07]. .
21	WOO S, PARK J, LEE J, et al. CBAM: Convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision(ECCV). Cham: Springer, 2018: 3-19.
22	LIU H, LIU F, FAN X, et al. Polarized self-attention: Towards high-quality pixel-wise regression[EB/OL]. (2021-07-08)[2022-04-07]. .
23	QIU Y, LIU Y, LI S, et al.MiniSeg: An Extremely Minimum Network for Efficient COVID-19 Segmentation[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(6): 4846-4854.
24	XIAO H, RAN Z, MABU S, et al. SAUNet++: An automatic segmentation model of COVID-19 lesion from CT slices[J/OL]. The Visual Computer, 2022: 1-14. : .
25	ZHAO X, ZHANG P, SONG F, et al. D2A U-Net: Automatic segmentation of COVID-19 CT slices based on dual attention and hybrid dilated convolution[J]. Computers in Biology and Medicine, 2021, 135: 104526.
26	LI Z, LI D, XU C, et al. TFCNs: A CNN-Transformer hybrid network for medical image segmentation[C]//Artificial Neural Networks and Machine Learning-ICANN 2022. Cham: Springer, 2022: 781-792.
27	BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet:A deep convolutional encoder-decoder architecture for image segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495.
28	ZHAO H, SHI J, QI X, et al. Pyramid scene parsing network[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Honolulu: IEEE, 2017: 6230-6239.
29	OKTAY O, SCHLEMPER J, FOLGOC L, et al. Attention U-Net: Learning where to look for the pancreas[EB/OL]. (2018-05-20)[2022-04-07]. .
30	ALOM M Z, HASAN M, YAKOPCIC C, et al. Recurrent residual convolutional neural network based on U-Net (R2U-Net) for medical image segmentation[EB/OL]. (2018-05-29)[2022-04-07]. .
31	HUANG H, LIN L, TONG R, et al. Unet 3+: A full-scale connected unet for medical image segmentation[C]//ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP).Barcelona: IEEE, 2020: 1055-1059.

数据集	CT切片数量/张	病例数/个	图像大小/px
COVID-19 Dataset	100	50	512🞩512
MosMedData	785	~60	512🞩512
Jun数据集	1844	20	512🞩512 630🞩401 630🞩630
Our	2792	~130	512🞩512

数据集	CT切片数量/张	病例数/个	图像大小/px
COVID-19 Dataset	100	50	512🞩512
MosMedData	785	~60	512🞩512
Jun数据集	1844	20	512🞩512 630🞩401 630🞩630
Our	2792	~130	512🞩512

网络结构	参数/M	速度/fps
训练网络	19.79	10.69
推理网络	17.81	18.29

网络结构	参数/M	速度/fps
训练网络	19.79	10.69
推理网络	17.81	18.29

注意力机制	灵敏度	特异性	准确率	DICE
SENet^[19]	92.2	99.5	71.4	79.9
SGE^[20]	92.4	99.5	71.2	79.8
CBAM^[21]	91.4	99.5	71.7	79.5
PSA^[22]	93.2	99.4	70.5	79.0
SR&MDS-Net	91.5	99.5	72.8	80.1

基于结构重参数化与多尺度深度监督的COVID-19胸部CT图像自动分割

Toward Automated Segmentation of COVID-19 Chest CT Images Based on Structural Reparameterization and Multi-Scale Deep Supervision

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献 31

相关文章 15

编辑推荐

Metrics

上采样方法	灵敏度	特异性	准确率	DICE
最近邻插值	81.2	99.5	72.4	74.6
双线性插值	92.6	99.4	71.8	79.9
转置卷积	88.1	99.5	71.3	77.2
反池化	91.5	99.4	68.3	76.5
SR&MDS-Net	91.5	99.5	72.8	80.1

组件	灵敏度	特异性	准确率	DICE
UNet	90.8	99.2	64.5	74.0
Unet+SRFV-CA	89.2	99.4	67.7	75.8
Unet+SRFV	90.6	99.3	67.7	76.0
SR&MDS-Net	91.5	99.5	72.8	80.1

BN	灵敏度	特异性	准确率	DICE
Inner BN +CE	89.7	99.4	72.1	77.8
InnerBN+CE+Dice	88.4	99.5	73.3	78.0
InnerBN+CE+0.5Dice	91.5	99.5	72.8	80.1
ExternalBN+CE	90.8	99.4	70.2	77.8
ExternalBN+CE+Dice	89.7	99.4	68.5	76.5
ExternalBN+CE+0.5Dice	88.1	99.4	72.2	76.9

Method	灵敏度	特异性	准确率	DICE
Miniseg^[23]	91.0	99.5	72.8	79.6
SAUNet++^[24]	80.8	99.3	66.9	71.3
D2A U-Net^[25]	90.0	99.3	68.9	75.7
TFCNs^[26]	73.0	99.5	70.1	68.6
SR&MDS-Net	91.5	99.5	72.8	80.1

Method	灵敏度/%	特异性/%	准确率/%	DICE/%	FLOPs/G	参数量/M
Segnet^[27]	86.7	99.0	57.6	67.7	160.56	29.44
PSPNet^[28]	87.7	99.3	66.5	73.2	159.48	27.50
Attention U-Net^[29]	88.5	99.3	65.1	74.2	333.53	40.11
R2U-Net^[30]	83.7	98.4	48.6	59.5	611.69	39.09
Unet3+^[31]	90.6	99.2	63.7	73.2	799.72	26.97
Unet3+_DeepSupvise^[31]	92.7	99.2	64.1	74.6	800.26	27.01
SR&MDS-Net	91.5	99.5	72.8	80.1	157.98	17.81

[1]	吴靖, 叶晓晶, 黄峰, 陈丽琼, 王志锋, 刘文犀. 基于深度学习的单帧图像超分辨率重建综述[J]. 电子学报, 2022, 50(9): 2265-2294.
[2]	李雪莹, 王田路, 梁鹏, 王翀. 基于系统模型的用户评论中非功能需求的自动分类[J]. 电子学报, 2022, 50(9): 2079-2089.
[3]	琚长瑞, 秦晓燕, 袁广林, 李豪, 朱虹. 尺度敏感损失与特征融合的快速小目标检测方法[J]. 电子学报, 2022, 50(9): 2119-2126.
[4]	张志昌, 于沛霖, 庞雅丽, 朱林, 曾扬扬. SMGN：用于对话状态跟踪的状态记忆图网络[J]. 电子学报, 2022, 50(8): 1851-1858.
[5]	张亚洲, 俞洋, 朱少林, 陈锐, 戎璐, 梁辉. 一种量子概率启发的对话讽刺识别网络模型[J]. 电子学报, 2022, 50(8): 1885-1893.
[6]	王飞扬, 冀鹏欣, 孙笠, 危倩, 李根, 张忠宝. 一种基于深度学习的动态社交网络用户对齐方法[J]. 电子学报, 2022, 50(8): 1925-1936.
[7]	徐兴荣, 刘聪, 李婷, 郭娜, 任崇广, 曾庆田. 基于双向准循环神经网络和注意力机制的业务流程剩余时间预测方法[J]. 电子学报, 2022, 50(8): 1975-1984.
[8]	欧阳与点, 谢鲲, 谢高岗, 文吉刚. 面向大规模网络测量的数据恢复算法：基于关联学习的张量填充[J]. 电子学报, 2022, 50(7): 1653-1663.
[9]	裴炤, 邱文涛, 王淼, 马苗, 张艳宁. 基于Transformer动态场景信息生成对抗网络的行人轨迹预测方法[J]. 电子学报, 2022, 50(7): 1537-1547.
[10]	彭闯, 王伦文, 胡炜林. 融合深度特征的电磁频谱异常检测算法[J]. 电子学报, 2022, 50(6): 1359-1369.
[11]	杨伟超, 杜宇, 文伟, 侯舒维, 徐常志, 张建华. 基于多重分形谱智能分析的卫星信号调制识别研究[J]. 电子学报, 2022, 50(6): 1336-1343.
[12]	李政伟, 李佳树, 尤著宏, 聂茹, 赵欢, 钟堂波. 基于异质图注意力网络的miRNA与疾病关联预测算法[J]. 电子学报, 2022, 50(6): 1428-1435.
[13]	张波, 陆云杰, 秦东明, 邹国建. 一种卷积自编码深度学习的空气污染多站点联合预测模型[J]. 电子学报, 2022, 50(6): 1410-1427.
[14]	廖勇, 李玉杰. 一种轻量化低复杂度的FDD大规模MIMO系统CSI反馈方法[J]. 电子学报, 2022, 50(5): 1211-1217.
[15]	冀振燕, 韩梦豪, 宋晓军, 冯其波. 面向激光光条图像修复的循环相似度映射网络[J]. 电子学报, 2022, 50(5): 1234-1242.