基于混洗特征编码与门控解码的医学图像分割网络

雷涛; 张峻铭; 杜晓刚; 闵重丹; 杨子瑶

doi:10.12263/DZXB.20231011

您当前的位置：

首页 >

文章列表页 >

基于混洗特征编码与门控解码的医学图像分割网络

学术论文 | 更新时间：2025-12-11

- 基于混洗特征编码与门控解码的医学图像分割网络
- Medical Image Segmentation Network Based on Shuffled Feature Encoding and Gated Decoding
- 电子学报 2024年52卷第12期页码：4142-4152
- 作者机构：
  
  1.陕西科技大学电子信息与人工智能学院，陕西西安 710021
  2.陕西省人工智能联合实验室（陕西科技大学），陕西西安 710021
- 作者简介：
  
  [ "雷涛男，1981年11月出生，陕西大荔人.2011年在西北工业大学获得博士学位，现为陕西科技大学教授，博士生导师.主要研究方向为图像处理、模式识别和计算机视觉等. E-mail: leitao@sust.edu.cn" ]
  [ "张峻铭男，1998年9月出生，江苏无锡人.陕西科技大学硕士.主要研究方向为计算机视觉. E-mail: zhangjm922@outlook.com" ]
  [ "杜晓刚男，1985年出生，陕西宝鸡人.现为陕西科技大学电子信息与人工智能学院副教授.主要研究方向为机器学习、计算机视觉、医学图像处理.E-mail: du423@sina.com" ]
- 基金信息：
  
  国家自然科学基金(62271296;62201334);陕西省杰出青年科学基金(2021JC-47);陕西省重点研发计划(2022GY-436;2021ZDLGY08-07);陕西省创新能力支撑计划(2020SS-03);陕西省教育厅科学研究计划项目(23JP014;23JP022)
- DOI：10.12263/DZXB.20231011
  中图分类号： TP391;
- 收稿：2023-10-27，
  
  修回：2024-08-07，
  
  纸质出版：2024-12-25
- 稿件说明：
移动端阅览
雷涛, 张峻铭, 杜晓刚, 等. 基于混洗特征编码与门控解码的医学图像分割网络[J]. 电子学报, 2024, 52(12): 4142-4152.

LEI Tao, ZHANG Jun-ming, DU Xiao-gang, et al. Medical Image Segmentation Network Based on Shuffled Feature Encoding and Gated Decoding[J]. Acta Electronica Sinica, 2024, 52(12): 4142-4152.
雷涛, 张峻铭, 杜晓刚, 等. 基于混洗特征编码与门控解码的医学图像分割网络[J]. 电子学报, 2024, 52(12): 4142-4152. DOI：10.12263/DZXB.20231011

LEI Tao, ZHANG Jun-ming, DU Xiao-gang, et al. Medical Image Segmentation Network Based on Shuffled Feature Encoding and Gated Decoding[J]. Acta Electronica Sinica, 2024, 52(12): 4142-4152. DOI：10.12263/DZXB.20231011

摘要

针对医学图像分割领域长期存在的多目标尺度变化大和边界模糊以致分割困难的问题，提出了一种新型的基于混洗特征编码和门控解码的双分支混合网络框架用于多器官精准分割.为了充分利用卷积神经网络（Convolutional Neural Network，CNN）在局部信息提取方面和Transformer在长程依赖关系建模方面的优势，采用U-Net和Swin-Unet构建双分支网络.该方法的创新之处在于对不同网络分支的多个阶段学习到的高维特征进行混洗操作，通过双支路通道交叉融合的方式实现局部信息与全局信息的高效融合，加强了双分支网络在不同阶段间的信息交互，从而解决了图像目标轮廓模糊引起的分割精度受限的问题.此外，为了解决多器官尺度变化大的问题，进一步提出了一种全新的基于多尺度特征图的门控解码器（Gated Decoder based on Multi-scale Feature，GDMF）.该解码器能够学习网络不同阶段的多尺度高维特征并进行自适应特征增强，采用注意力机制和特征映射来辅助获取精准目标信息.实验结果表明，与现有主流医学图像分割方法相比，所提方法在ACDC（Automated Cardiac Diagnosis Challenge）和FLARE21（Fast and Low GPU memory Abdominal oRgan sEgmentation challenge 2021）数据集上均表现出更优的性能，有效解决了医学图像中多目标尺度变化大和边界模糊问题.

Abstract

To solve the long-standing problems of the great scale variation in target sizes and blurred boundaries that make segmentation difficult in medical image segmentation

we propose a novel dual-branch hybrid network framework based on feature encoding and gated decoder based on multi-scale feature for accurate multi-organ segmentation. In order to fully exploit the strengths of convolutional neural network (CNN) in local information extraction and transformers in modeling long-range dependency

we employ U-Net and Swin-Unet to construct the dual-branch network. The innovation of this method lies in the shuffling operation of high-dimensional features extracted at multiple stages from different branches of the network. It efficiently integrates local and global information by means of a dual-branch channel cross-fusion

enhancing information interaction between the dual-branch network at different stages. This addresses the limitation in segmentation accuracy caused by the blurring of object contours in images. Additionally

to address the challenge of great scale variation among multiple organs

we introduce a new gated decoder based on multi-scale feature (GDMF) to extract multi-scale high-dimensional features at different stages of the network and perform adaptive feature enhancement

and adopts the attention mechanisms and feature mappings to assist in acquiring accurate target information. The experimental results on automated cardiac diagnosis challenge (ACDC) and fast and low GPU memory abdominal organ segmentation challenge 2021 (FLARE21) datasets demonstrate that our proposed method outperforms existing mainstream medical image segmentation methods and effectively solves the problems of the great scale variation in target sizes and blurred boundary in medical images.

关键词

Keywords

references

RONNEBERGER O , FISCHER P , BROX T . U-net: Convolutional networks for biomedical image segmentation [M ] // Lecture Notes in Computer Science . Cham : Springer International Publishing , 2015 : 234 - 241 .

MILLETARI F , NAVAB N , AHMADI S A . V-net: Fully convolutional neural networks for volumetric medical image segmentation [C ] // 2016 Fourth International Conference on 3D Vision (3DV) . Piscataway : IEEE , 2016 : 565 - 571 .

LI R , ZHENG S Y , DUAN C X , et al . Multistage attention ResU-net for semantic segmentation of fine-resolution remote sensing images [J ] . IEEE Geoscience and Remote Sensing Letters , 2021 , 19 : 8009205 .

BI R R , JI C L , YANG Z P , et al . Residual based attention-Unet combing DAC and RMP modules for automatic liver tumor segmentation in CT [J ] . Mathematical Biosciences and Engineering , 2022 , 19 ( 5 ): 4703 - 4718 .

CHENG Z M , QU A P , HE X F . Contour-aware semantic segmentation network with spatial attention mechanism for medical image [J ] . The Visual Computer , 2022 , 38 ( 3 ): 749 - 762 .

OKTAY O , SCHLEMPER J , LE FOLGOC L , et al . Attention U-net: Learning where to look for the pancreas [EB/OL ] . ( 2018-05-20 )[ 2023-10-27 ] . https://arxiv.org/abs/1804.03999v3 https://arxiv.org/abs/1804.03999v3 .

SCHLEMPER J , OKTAY O , SCHAAP M , et al . Attention gated networks: Learning to leverage salient regions in medical images [J ] . Medical Image Analysis , 2019 , 53 : 197 - 207 .

HE H Y , CAI J F , LIU J , et al . Pruning self-attentions into convolutional layers in single path [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2024 , 46 ( 5 ): 3910 - 3922 .

ZHAO H S , SHI J P , QI X J , et al . Pyramid scene parsing network [C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2017 : 6230 - 6239 .

LIN T Y , DOLLÁR P , GIRSHICK R , et al . Feature pyramid networks for object detection [C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2017 : 936 - 944 .

CHEN L C , PAPANDREOU G , SCHROFF F , et al . Rethinking atrous convolution for semantic image segmentation [EB/OL ] . ( 2017-12-05 )[ 2023-10-27 ] . https://arxiv.org/abs/1706.05587v3 https://arxiv.org/abs/1706.05587v3 .

ZHOU Z X , HE Z S , JIA Y Y . AFPNet: A 3D fully convolutional neural network with atrous-convolution feature pyramid for brain tumor segmentation via MRI images [J ] . Neurocomputing , 2020 , 402 : 235 - 244 .

YU F , KOLTUN V , FUNKHOUSER T . Dilated residual networks [C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2017 : 636 - 644 .

XIE F , HUANG Z , SHI Z J , et al . DUDA-Net: A double U-shaped dilated attention network for automatic infection area segmentation in COVID-19 lung CT images [J ] . International Journal of Computer Assisted Radiology and Surgery , 2021 , 16 ( 9 ): 1425 - 1434 .

GU Z W , CHENG J , FU H Z , et al . CE-net: Context encoder network for 2D medical image segmentation [J ] . IEEE Transactions on Medical Imaging , 2019 , 38 ( 10 ): 2281 - 2292 .

HOWARD A G , ZHU M L , CHEN B , et al . MobileNets: Efficient convolutional neural networks for mobile vision applications [EB/OL ] . ( 2017-04-17 )[ 2023-10-27 ] . https://arxiv.org/abs/1704.04861v1 https://arxiv.org/abs/1704.04861v1 .

LEI T , SUN R , DU X G , et al . SGU-net: Shape-guided ultralight network for abdominal image segmentation [J ] . IEEE Journal of Biomedical and Health Informatics , 2023 , 27 ( 3 ): 1431 - 1442 .

YANG B , BENDER G , LE Q V , et al . CondConv: Conditionally parameterized convolutions for efficient inference [EB/OL ] . ( 2020-09-04 )[ 2023-10-27 ] . https://arxiv.org/abs/1904.04971v3 https://arxiv.org/abs/1904.04971v3 .

LEI T , ZHANG D , DU X G , et al . Semi-supervised medical image segmentation using adversarial consistency learning and dynamic convolution network [J ] . IEEE Transactions on Medical Imaging , 2023 , 42 ( 5 ): 1265 - 1277 .

DAI J F , QI H Z , XIONG Y W , et al . Deformable convolutional networks [C ] // 2017 IEEE International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2017 : 764 - 773 .

YANG X , LI Z Q , GUO Y Q , et al . DCU-net: A deformable convolutional neural network based on cascade U-net for retinal vessel segmentation [J ] . Multimedia Tools and Applications , 2022 , 81 ( 11 ): 15593 - 15607 .

LEI T , WANG R S , ZHANG Y X , et al . DefED-net: Deformable encoder-decoder network for liver and liver tumor segmentation [J ] . IEEE Transactions on Radiation and Plasma Medical Sciences , 2022 , 6 ( 1 ): 68 - 78 .

ZHOU Z W , SIDDIQUEE M M R , TAJBAKHSH N , et al . UNet++: Redesigning skip connections to exploit multiscale features in image segmentation [J ] . IEEE Transactions on Medical Imaging , 2020 , 39 ( 6 ): 1856 - 1867 .

CAI S J , TIAN Y X , LUI H , et al . Dense-UNet: A novel multiphoton in vivo cellular image segmentation model based on a convolutional neural network [J ] . Quantitative Imaging in Medicine and Surgery , 2020 , 10 ( 6 ): 1275 - 1285 .

CAI Z T , XIN J M , SHI P W , et al . DSTUNet: UNet with efficient dense SWIN transformer pathway for medical image segmentation [C ] // 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI) . Piscataway : IEEE , 2022 : 1 - 5 .

WANG H N , CAO P , WANG J Q , et al . UCTransNet: Rethinking the skip connections in U-net from a channel-wise perspective with transformer [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2022 , 36 ( 3 ): 2441 - 2449 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [EB/OL ] . ( 2017-06-12 )[ 2023-10-27 ] . https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762 .

DOSOVITSKIY A , BEYER L , KOLESNIKOV A , et al . An image is worth 16 x 16 words: Transformers for image recognition at scale [EB/OL ] . ( 2021-06-03 )[ 2023-10-27 ] . https://arxiv.org/abs/2010.11929v2 https://arxiv.org/abs/2010.11929v2 .

CHEN J N , LU Y Y , YU Q H , et al . TransUNet: Transformers make strong encoders for medical image segmentation [EB/OL ] . ( 2021-02-08 )[ 2023-10-27 ] . https://arxiv.org/abs/2102.04306v1 https://arxiv.org/abs/2102.04306v1 .

CAO H , WANG Y Y , CHEN J , et al . Swin-Unet: Unet-like pure transformer for medical image segmentation [M ] // Lecture Notes in Computer Science . Cham : Springer Nature Switzerland , 2023 : 205 - 218 .

LIU Z , LIN Y T , CAO Y , et al . Swin transformer: Hierarchical vision transformer using shifted windows [C ] // 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2021 : 9992 - 10002 .

HUANG H M , LIN L F , TONG R F , et al . UNet 3 +: A full-scale connected UNet for medical image segmentation[C ] // ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . Piscataway : IEEE , 2020 : 1055 - 1059 .

GUO C L , SZEMENYEI M , HU Y T , et al . Channel attention residual U-net for retinal vessel segmentation [C ] // ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . Piscataway : IEEE , 2021 : 1185 - 1189 .

WOO S , PARK J , LEE J Y , et al . CBAM: Convolutional block attention module [M ] // Lecture Notes in Computer Science . Cham : Springer International Publishing , 2018 : 3 - 19 .

PEIRIS H , HAYAT M , CHEN Z L , et al . A robust volumetric transformer for accurate 3D tumor segmentation [M ] // Lecture Notes in Computer Science . Cham : Springer Nature Switzerland , 2022 : 162 - 172 .

TRAGAKIS A , KAUL C , MURRAY-SMITH R , et al . The fully convolutional transformer for medical image segmentation [C ] // 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) . Piscataway : IEEE , 2023 : 3649 - 3658 .

HE Z Q , UNBERATH M , KE J , et al . TransNuSeg: A lightweight multi-task transformer for nuclei segmentation [M ] // Lecture Notes in Computer Science . Cham : Springer Nature Switzerland , 2023 : 206 - 215 .

LIU Z , MAO H Z , WU C Y , et al . A ConvNet for the 2020 s[C ] // 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2022 : 11966 - 11976 .

LEE H H , BAO S X , HUO Y K , et al . 3D UX-Net: A large kernel volumetric ConvNet modernizing hierarchical transformer for medical image segmentation [EB/OL ] . ( 2023-03-02 )[ 2023-10-27 ] . https://arxiv.org/abs/2209.15076v4 https://arxiv.org/abs/2209.15076v4 .

GAO Y H , ZHOU M , METAXAS D N . UTNet: A hybrid transformer architecture for medical image segmentation [M ] // Lecture Notes in Computer Science . Cham : Springer International Publishing , 2021 : 61 - 71 .

LEI T , SUN R , WANG X , et al . CiT-net: Convolutional neural networks hand in hand with vision transformers for medical image segmentation [C ] // Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence . California : International Joint Conferences on Artificial Intelligence Organization , 2023 : 1017 - 1025 .

XU G P , ZHANG X , HE X W , et al . LeViT-UNet: Make faster encoders with transformer for medical image segmentation [M ] // Lecture Notes in Computer Science . Singapore : Springer Nature Singapore , 2023 : 42 - 53 .

GONG Z D , FRENCH A P , QIU G P , et al . CTranS: A multi-resolution convolution-transformer network for medical image segmentation [C ] // 2024 IEEE International Symposium on Biomedical Imaging (ISBI) . Piscataway : IEEE , 2024 : 1 - 5 .

HATAMIZADEH A , TANG Y C , NATH V , et al . UNETR: Transformers for 3D medical image segmentation [C ] // 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) . Piscataway : IEEE , 2022 : 1748 - 1758 .

HATAMIZADEH A , NATH V , TANG Y C , et al . Swin UNETR: Swin transformers for semantic segmentation of brain tumors in MRI images [M ] // Lecture Notes in Computer Science . Cham : Springer International Publishing , 2022 : 272 - 284 .

QIN X Y , LI N , WENG C , et al . Simple attention module based speaker verification with iterative noisy label detection [C ] // ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . Piscataway : IEEE , 2022 : 6722 - 6726 .

BERNARD O , LALANDE A , ZOTTI C , et al . Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: Is the problem solved? [J ] . IEEE Transactions on Medical Imaging , 2018 , 37 ( 11 ): 2514 - 2525 .

MA J , ZHANG Y , GU S , et al . Fast and low-GPU-memory abdomen CT organ segmentation: The FLARE challenge [J ] . Medical Image Analysis , 2022 , 82 : 102616 .

HUANG X H , DENG Z F , LI D D , et al . MISSFormer: An effective medical image segmentation transformer [EB/OL ] . ( 2021-12-19 )[ 2023-10-27 ] . https://arxiv.org/abs/2109.07162v2 https://arxiv.org/abs/2109.07162v2 .

ISENSEE F , JAEGER P F , KOHL S A A , et al . nnU-Net: A self-configuring method for deep learning-based biomedical image segmentation [J ] . Nature Methods , 2021 , 18 ( 2 ): 203 - 211 .

ZHOU H Y , GUO J S , ZHANG Y H , et al . nnFormer: Volumetric medical image segmentation via a 3D transformer [J ] . IEEE Transactions on Image Processing , 2023 , 32 : 4036 - 4045 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于交叉视觉状态空间与多分支交互注意力的医学图像分割

基于可疑像素相互修正的半监督医学图像分割

基于多尺度卷积调制的医学图像分割

SAU-Net：基于U-Net和自注意力机制的医学图像分割方法