SM<sup>3</sup>RNet：一种基于空间Mamba的模态缺失鲁棒病灶分割网络

贾熹滨; 杨川旭; 范超; 郑依鸣; 杨正汉; 杨大为; 徐辉

doi:10.12263/DZXB.20250677

您当前的位置：

首页 >

文章列表页 >

SM³RNet：一种基于空间Mamba的模态缺失鲁棒病灶分割网络

学术论文 | 更新时间：2026-02-10

- SM³RNet：一种基于空间Mamba的模态缺失鲁棒病灶分割网络
- SM³RNet: Spatial Mamba Based Missing Modality Robust Lesion Segmentation Network
- 电子学报 2025年53卷第11期页码：3943-3955
- 作者机构：
  
  1.北京工业大学计算机学院，北京 100124
  2.首都医科大学附属北京友谊医院，北京 100050
- 作者简介：
  
  [ "贾熹滨女，1969年11月出生于山西省太原市.现为北京工业大学计算机学院教授.主要研究方向为计算机视觉、多模态深度学习、智能医学影像、情感计算、行为识别.E-mail: jiaxibin@bjut.edu.cn" ]
  [ "杨川旭男，2001年8月出生于黑龙江省哈尔滨市.现为北京工业大学硕士研究生.主要研究方向为计算机视觉和多模态医学图像分割. E-mail: ycx_shiroi@emails.bjut.edu.cn" ]
  [ "范超男，1994年2月出生于安徽省安庆市.现为北京工业大学博士研究生.主要研究方向为医学图像分析.E-mail: chao.fancripac@gmail.com" ]
  [ "郑依鸣男，2001年6月出生于湖北省黄冈市.毕业于北京工业大学.主要研究方向为半监督医学图像分割.E-mail: 1746257500@qq.com" ]
  [ "杨正汉男，1968年9月出生于浙江省衢州市.现为北京友谊医院放射科主任.主要研究方向为腹部疾病的影像诊断、肝细胞癌及癌前病变的早期影像诊断以及新MRI技术的开发与应用.E-mail: zhenghanyang@263.net" ]
  [ "杨大为男，1983年4月出生于湖南省岳阳市.现为北京友谊医院放射科主任助理.主要研究方向为肝脏疾病的影像诊断及相关研究.E-mail: dawei-yang@vip.163.com" ]
  [ "徐辉男，1978年12月出生于山东省德州市.现为北京友谊医院放射科主任医师.主要研究方向为深度学习在肝脏疾病诊断中的应用.E-mail: mr_xuhui@163.com" ]
- 基金信息：
  
  国家自然科学基金(62171298;82372043;82071876;82371904;62476015)
- DOI：10.12263/DZXB.20250677
  中图分类号： TP391;
- 收稿：2025-08-04，
  
  录用：2025-11-19，
  
  纸质出版：2025-11-25
- 稿件说明：
移动端阅览
贾熹滨, 杨川旭, 范超, 等. SM³RNet：一种基于空间Mamba的模态缺失鲁棒病灶分割网络[J]. 电子学报, 2025, 53(11): 3943-3955.

JIA Xi-bin, YANG Chuan-xu, FAN Chao, et al. SM³RNet: Spatial Mamba Based Missing Modality Robust Lesion Segmentation Network[J]. Acta Electronica Sinica, 2025, 53(11): 3943-3955.
贾熹滨, 杨川旭, 范超, 等. SM³RNet：一种基于空间Mamba的模态缺失鲁棒病灶分割网络[J]. 电子学报, 2025, 53(11): 3943-3955. DOI：10.12263/DZXB.20250677

JIA Xi-bin, YANG Chuan-xu, FAN Chao, et al. SM³RNet: Spatial Mamba Based Missing Modality Robust Lesion Segmentation Network[J]. Acta Electronica Sinica, 2025, 53(11): 3943-3955. DOI：10.12263/DZXB.20250677

摘要

多模态医学图像分割能够充分利用MRI（Magnetic Resonance Imaging）不同序列（如T1、T1ce、T2、FLAIR）之间丰富的信息互补特性，在脑肿瘤等复杂病灶分割任务中获得远超单模态的精度与鲁棒性.然而，现有绝大多数方法均建立在“所有模态推理时完整可用”的强假设之上，而在真实临床场景中，由于患者运动、扫描协议差异、设备限制或历史数据缺失等原因，经常出现一个或多个模态不可用的情况，导致这些方法在模态缺失时性能大幅衰减，极大限制了其在实际诊疗中的可落地性.为此，本文提出了一种基于空间Mamba的全新模态缺失鲁棒病灶分割网络（Spatial Mamba based Missing Modality Robust lesion segmentation Network，SM

RNet），从编码、融合到解码阶段系统性地解决了任意模态组合下性能稳定的核心难题.SM

RNet设计了基于Mamba的多分支空间特征编码器（Mamba-based Multi-branch Spatial Feature Encoder，SME），实现了线性计算复杂度下对三维医学数据的全局上下文建模；进一步，为确保部分模态缺失时病灶分割性能的稳定性，提取并利用模态共有的可判别特征，SM

RNet提出了一种基于多视角注意力机制引导

的跨模态特征融合方法（Multi-view Attention-guided Cross-modal feature Fusion，MACF），通过多视角的交互与注意力机制，动态增强模态间共有的语义特征贡献、自适应地协同不同模态组合，提升融合的鲁棒性，有效缓解模态缺失导致的性能衰减；此外，SM

RNet在跳跃连接中集成了并行的空间和通道双流注意力机制（Dual Stream attention Decoder，DSD）从两个维度协同优化融合特征表示，增加病灶辨识度，强化边缘细节恢复能力，从而在最终分割图上获得更高的准确率和完整性.在国际权威的BraTS2020和BraTS2018数据集上进行的大量对比与消融实验充分验证了所提方法的优越性：在所有模态完备时，SM

RNet的Dice等指标优于现有方法，在随机缺失模态的环境下，仍能超越当前专门处理模态缺失的先进方法，展现出较强的鲁棒性和临床部署潜力，为临床实用的多模态医学图像分割提供了高效、可靠的新范式.

Abstract

Multi-modal medical image segmentation can fully exploit the rich complementary information across different magnetic resonance imaging (MRI) sequences (e.g.

T1ce

FLAIR)

achieving markedly superior accuracy and robustness compared to single-modality approaches in complex lesion segmentation tasks such as brain tumors. However

the vast majority of existing methods rely on the strong assumption that all modalities are fully available during inference

whereas in real-world clinical practice

one or several modalities are frequently missing due to patient motion

heterogeneous scanning protocols

equipment constraints

or absent historical data

causing drastic performance degradation and severely limiting practical deployability. To address this critical challenge

we propose a novel spatial Mamba-based missing-modality-robust lesion segmentation network

termed spatial Mamba-based missing modality robust lesion segmentation network (SM

RNet)

which systematically ensures stable performance across arbitrary modality combinations from encoding to decoding stages. SM

RNet first introduces a Mamba-based multi-branch spatial feature encoder (SME) that assigns an independent Mamba branch to each modality and performs efficient bidirectional long-range dependency modeling along the

and

axes

realizing global contextual modeling of 3D medical volumes with only linear computational and memory complexity

far surpa

ssing the quadratic burden of Transformer-based alternatives. To maintain segmentation stability when partial modalities are absent

RNet extracts and leverages discriminative features shared across modalities through a multi-view attention-guided cross-modal feature fusion module (MACF); by simultaneously operating from channel

spatial

and inter-modality perspectives

MACF dynamically amplifies the contribution of shared semantic features

adaptively coordinates varying modality combinations via attention mechanisms

and effectively alleviates performance drops caused by missing modalities. Furthermore

a parallel dual-stream attention decoder (DSD) is integrated into skip connections to synergistically refine multi-scale fused features from both spatial and channel dimensions

significantly enhancing lesion discriminability and boundary detail recovery

thereby yielding superior accuracy and completeness in the final segmentation maps. Extensive comparative and ablation experiments conducted on the internationally authoritative BraTS2020 and BraTS2018 datasets comprehensively validate the superiority of the proposed method: When all modalities are complete

RNet outperforms existing methods in metrics such as Dice. In environments with randomly missing modalities

it still surpasses state-of-the-art methods specifically designed to handle missing modalities. This demonstrates strong robustness and significant potential for clinical deployment

providing an efficient and reliable novel paradigm for clinically practical multi-modal medical image segmentation.

关键词

Keywords

references

LITJENS G , KOOI T , BEJNORDI B E , et al . A survey on deep learning in medical image analysis [J ] . Medical Image Analysis , 2017 , 42 : 60 - 88 .

SHEN D G , WU G R , SUK H I . Deep learning in medical image analysis [J ] . Annual Review of Biomedical Engineering , 2017 , 19 : 221 - 248 .

XING Z H , YU L Q , WAN L , et al . NestedFormer: Nested modality-aware transformer for Brain tumor segmentation [C ] // Medical Image Computing and Computer Assisted Intervention - MICCAI 2022 . Cham : Springer , 2022 : 140 - 150 .

周新民 , 熊智谋 , 史长发 , 等 . 基于多尺度卷积调制的医学图像分割 [J ] . 电子学报 , 2024 , 52 ( 9 ): 3159 - 3171 .

ZHOU X M , XIONG Z M , SHI C F , et al . Medical image segmentation based on multi-scale convolution modulation [J ] . Acta Electronica Sinica , 2024 , 52 ( 9 ): 3159 - 3171 . (in Chinese)

雷涛 , 张峻铭 , 杜晓刚 , 等 . 基于混洗特征编码与门控解码的医学图像分割网络 [J ] . 电子学报 , 2024 , 52 ( 12 ): 4142 - 4152 .

LEI T , ZHANG J M , DU X G , et al . Medical image segmentation network based on shuffled feature encoding and gated decoding [J ] . Acta Electronica Sinica , 2024 , 52 ( 12 ): 4142 - 4152 . (in Chinese)

MENZE B H , JAKAB A , BAUER S , et al . The multimodal brain tumor image segmentation benchmark (BRATS) [J ] . IEEE Transactions on Medical Imaging , 2015 , 34 ( 10 ): 1993 - 2024 .

DING Y H , YU X , YANG Y . RFNet: Region-aware fusion network for incomplete multi-modal brain tumor segmentation [C ] // 2021 IEEE/CVF International Conference on Computer Vision . Piscataway : IEEE , 2022 : 3955 - 3964 .

DORENT R , JOUTARD S , MODAT M , et al . Hetero-modal variational encoder-decoder for joint modality completion and segmentation [C ] // Medical Image Computing and Computer Assisted Intervention - MICCAI 2019 . Cham : Springer , 2019 : 74 - 82 .

HAVAEI M , GUIZARD N , CHAPADOS N , et al . HeMIS: Hetero-modal image segmentation [C ] // Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016 . Cham : Springer , 2016 : 469 - 477 .

ZHANG Y , HE N J , YANG J W , et al . mmFormer: Multimodal medical transformer for incomplete multimodal learning of brain tumor segmentation [C ] // Medical Image Computing and Computer Assisted Intervention - MICCAI 2022 . Cham : Springer , 2022 : 107 - 117 .

LIU H , WEI D , LU D H , et al . M3AE: Multimodal representation learning for brain tumor segmentation with missing modalities [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2023 , 37 ( 2 ): 1657 - 1665 .

RONNEBERGER O , FISCHER P , BROX T . U-Net: Convolutional networks for biomedical image segmentation [C ] // Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015 . Cham : Springer , 2015 : 234 - 241 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [EB/OL ] . ( 2023-08-02 )[ 2025-10-10 ] . https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762 .

GU A , DAO T . Mamba: Linear-time sequence modeling with selective state spaces [EB/OL ] . ( 2024-05-31 )[ 2025-10-10 ] . https://arXiv.org/abs/2312.00752 https://arXiv.org/abs/2312.00752 .

LIU J R , YANG H , ZHOU H Y , et al . Swin-UMamba: Mamba-based UNet with ImageNet-based pretraining [C ] // Medical Image Computing and Computer Assisted Intervention - MICCAI 2024 . New York : ACM , 2024 : 615 - 625 .

MA J , LI F F , WANG B . U-mamba: Enhancing long-range dependency for biomedical image segmentation [EB/OL ] . ( 2024-01-09 )[ 2025-11-10 ] . https://arXiv.org/abs/2401.04722 https://arXiv.org/abs/2401.04722 .

ZHU L H , LIAO B C , ZHANG Q , et al . Vision Mamba: Efficient visual representation learning with bidirectional state space model [EB/OL ] . ( 2024-11-14 )[ 2025-10-10 ] . https://arXiv.org/abs/2401.09417 https://arXiv.org/abs/2401.09417 .

RUAN J , XIANG S . VM-UNet: Vision Mamba UNet for medical image segmentation [EB/OL ] . ( 2024-05-28 )[ 2025-11-10 ] . https://arXiv.org/abs/2402.15604 https://arXiv.org/abs/2402.15604 .

LIAO W B , ZHU Y H , WANG X Y , et al . LightM-UNet: Mamba assists in lightweight UNet for medical image segmentation [EB/OL ] . ( 2024-03-11 )[ 2025-11-10 ] . https://arXiv.org/abs/2403.05246 https://arXiv.org/abs/2403.05246 .

FAN C , YU H Y , HUANG Y , et al . SliceMamba with neural architecture search for medical image segmentation [J ] . IEEE Journal of Biomedical and Health Informatics , 2025 , 29 ( 10 ): 7446 - 7458 .

ZHOU Z W , RAHMAN SIDDIQUEE M M , TAJBAKHSH N , et al . UNet++: A nested U-Net architecture for medical image segmentation [C ] // Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support . Cham : Springer , 2018 : 3 - 11 .

OKTAY O , SCHLEMPER J , LE FOLGOC L , et al . Attention U-Net: Learning where to look for the pancreas [EB/OL ] . ( 2018-05-20 )[ 2025-11-10 ] . https://arXiv.org/abs/1804.03999 https://arXiv.org/abs/1804.03999 .

ÇIÇEK Ö , ABDULKADIR A , LIENKAMP S S , et al . 3D U-Net: Learning dense volumetric segmentation from sparse annotation [C ] // Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016 . Cham : Springer , 2016 : 424 - 432 .

MILLETARI F , NAVAB N , AHMADI S A . V-net: Fully convolutional neural networks for volumetric medical image segmentation [C ] // 2016 Fourth International Conference on 3D Vision . Piscataway : IEEE , 2016 : 565 - 571 .

ALOM M Z , YAKOPCIC C , HASAN M , et al . Recurrent residual U-Net for medical image segmentation [J ] . Journal of Medical Imaging , 2019 , 6 ( 1 ): 014006 .

ISENSEE F , PETERSEN J , KLEIN A , et al . Abstract: NnU-Net: Self-adapting framework for U-net-based medical image segmentation [M ] // Bildverarbeitung für die Medizin 2019 . Wiesbaden : Springer Fachmedien Wiesbaden , 2019 : 22 .

WANG W X , CHEN C , DING M , et al . TransBTS: Multimodal brain tumor segmentation using transformer [C ] // Medical Image Computing and Computer Assisted Intervention - MICCAI 2021 . Cham : Springer , 2021 : 109 - 119 .

HATAMIZADEH A , TANG Y C , NATH V , et al . UNETR: Transformers for 3D medical image segmentation [C ] // 2022 IEEE/CVF Winter Conference on Applications of Computer Vision . Piscataway : IEEE , 2022 : 1748 - 1758 .

XING Z H , YE T , YANG Y J , et al . SegMamba: Long-range sequential modeling Mamba for 3D medical image segmentation [C ] // Medical Image Computing and Computer Assisted Intervention - MICCAI 2024 . Cham : Springer , 2024 : 578 - 588 .

WOO S , PARK J , LEE J Y , et al . CBAM: Convolutional block attention module [C ] // Computer Vision - ECCV 2018 . Cham : Springer , 2018 : 3 - 19 .

VALANARASU J M J , PATEL V M . UNeXt: MLP-based rapid medical image segmentation network [C ] // Medical Image Computing and Computer Assisted Intervention - MICCAI 2022 . Cham : Springer , 2022 : 23 - 33 .

LIU F , ZHANG Y D , LU T , et al . Hierarchical in-out fusion for incomplete multimodal brain tumor segmentation [J ] . Scientific Reports , 2025 , 15 : 23017 .

XIONG Y Z , DU H W , ZHANG D K . A guided-learning network for brain tumor segmentation with missing modalities [C ] // Proceedings of the 2025 2nd International Conference on Generative Artificial Intelligence and Information Security . New York : ACM , 2025 : 219 - 225 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于视觉与深度学习的无人机自主着陆场景感知方法研究进展

基于跨视图上下文感知的高分辨率遥感图像半监督语义分割方法

DynKANet：KA表示定理启发的图像语义分割动态融合网络模型

面向时序异常检测的可变视距多向扫描方法

Ho³⁺和Er³⁺在YLiF₄晶体中的光谱强度和Ω_λ参数

SM3RNet：一种基于空间Mamba的模态缺失鲁棒病灶分割网络

SM3RNet: Spatial Mamba Based Missing Modality Robust Lesion Segmentation Network

DOI：10.12263/DZXB.20250677

摘要

Abstract

关键词

Keywords

references

SM³RNet：一种基于空间Mamba的模态缺失鲁棒病灶分割网络

SM³RNet: Spatial Mamba Based Missing Modality Robust Lesion Segmentation Network