SM<sup>3</sup>RNet: Spatial Mamba Based Missing Modality Robust Lesion Segmentation Network

JIA Xi-bin; YANG Chuan-xu; FAN Chao; ZHENG Yi-ming; YANG Zheng-han; YANG Da-wei; XU Hui

doi:10.12263/DZXB.20250677

您当前的位置：

首页 >

文章列表页 >

SM³RNet: Spatial Mamba Based Missing Modality Robust Lesion Segmentation Network

PAPERS | 更新时间：2026-02-10

- SM³RNet: Spatial Mamba Based Missing Modality Robust Lesion Segmentation Network
- ACTA ELECTRONICA SINICA Vol. 53, Issue 11, Pages: 3943-3955(2025)
- 作者机构：
  
  1.北京工业大学计算机学院，北京 100124
  2.首都医科大学附属北京友谊医院，北京 100050
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China(62171298;82372043;82071876;82371904;62476015)
- DOI：10.12263/DZXB.20250677
  CLC： TP391;
- Received：04 August 2025，
  
  Accepted：19 November 2025，
  
  Published：25 November 2025
- 稿件说明：
移动端阅览
贾熹滨, 杨川旭, 范超, 等. SM³RNet：一种基于空间Mamba的模态缺失鲁棒病灶分割网络[J]. 电子学报, 2025, 53(11): 3943-3955.

JIA Xi-bin, YANG Chuan-xu, FAN Chao, et al. SM³RNet: Spatial Mamba Based Missing Modality Robust Lesion Segmentation Network[J]. Acta Electronica Sinica, 2025, 53(11): 3943-3955.
贾熹滨, 杨川旭, 范超, 等. SM³RNet：一种基于空间Mamba的模态缺失鲁棒病灶分割网络[J]. 电子学报, 2025, 53(11): 3943-3955. DOI：10.12263/DZXB.20250677

JIA Xi-bin, YANG Chuan-xu, FAN Chao, et al. SM³RNet: Spatial Mamba Based Missing Modality Robust Lesion Segmentation Network[J]. Acta Electronica Sinica, 2025, 53(11): 3943-3955. DOI：10.12263/DZXB.20250677

摘要

多模态医学图像分割能够充分利用MRI（Magnetic Resonance Imaging）不同序列（如T1、T1ce、T2、FLAIR）之间丰富的信息互补特性，在脑肿瘤等复杂病灶分割任务中获得远超单模态的精度与鲁棒性.然而，现有绝大多数方法均建立在“所有模态推理时完整可用”的强假设之上，而在真实临床场景中，由于患者运动、扫描协议差异、设备限制或历史数据缺失等原因，经常出现一个或多个模态不可用的情况，导致这些方法在模态缺失时性能大幅衰减，极大限制了其在实际诊疗中的可落地性.为此，本文提出了一种基于空间Mamba的全新模态缺失鲁棒病灶分割网络（Spatial Mamba based Missing Modality Robust lesion segmentation Network，SM

RNet），从编码、融合到解码阶段系统性地解决了任意模态组合下性能稳定的核心难题.SM

RNet设计了基于Mamba的多分支空间特征编码器（Mamba-based Multi-branch Spatial Feature Encoder，SME），实现了线性计算复杂度下对三维医学数据的全局上下文建模；进一步，为确保部分模态缺失时病灶分割性能的稳定性，提取并利用模态共有的可判别特征，SM

RNet提出了一种基于多视角注意力机制引导

的跨模态特征融合方法（Multi-view Attention-guided Cross-modal feature Fusion，MACF），通过多视角的交互与注意力机制，动态增强模态间共有的语义特征贡献、自适应地协同不同模态组合，提升融合的鲁棒性，有效缓解模态缺失导致的性能衰减；此外，SM

RNet在跳跃连接中集成了并行的空间和通道双流注意力机制（Dual Stream attention Decoder，DSD）从两个维度协同优化融合特征表示，增加病灶辨识度，强化边缘细节恢复能力，从而在最终分割图上获得更高的准确率和完整性.在国际权威的BraTS2020和BraTS2018数据集上进行的大量对比与消融实验充分验证了所提方法的优越性：在所有模态完备时，SM

RNet的Dice等指标优于现有方法，在随机缺失模态的环境下，仍能超越当前专门处理模态缺失的先进方法，展现出较强的鲁棒性和临床部署潜力，为临床实用的多模态医学图像分割提供了高效、可靠的新范式.

Abstract

Multi-modal medical image segmentation can fully exploit the rich complementary information across different magnetic resonance imaging (MRI) sequences (e.g.

T1ce

FLAIR)

achieving markedly superior accuracy and robustness compared to single-modality approaches in complex lesion segmentation tasks such as brain tumors. However

the vast majority of existing methods rely on the strong assumption that all modalities are fully available during inference

whereas in real-world clinical practice

one or several modalities are frequently missing due to patient motion

heterogeneous scanning protocols

equipment constraints

or absent historical data

causing drastic performance degradation and severely limiting practical deployability. To address this critical challenge

we propose a novel spatial Mamba-based missing-modality-robust lesion segmentation network

termed spatial Mamba-based missing modality robust lesion segmentation network (SM

RNet)

which systematically ensures stable performance across arbitrary modality combinations from encoding to decoding stages. SM

RNet first introduces a Mamba-based multi-branch spatial feature encoder (SME) that assigns an independent Mamba branch to each modality and performs efficient bidirectional long-range dependency modeling along the

and

axes

realizing global contextual modeling of 3D medical volumes with only linear computational and memory complexity

far surpa

ssing the quadratic burden of Transformer-based alternatives. To maintain segmentation stability when partial modalities are absent

RNet extracts and leverages discriminative features shared across modalities through a multi-view attention-guided cross-modal feature fusion module (MACF); by simultaneously operating from channel

spatial

and inter-modality perspectives

MACF dynamically amplifies the contribution of shared semantic features

adaptively coordinates varying modality combinations via attention mechanisms

and effectively alleviates performance drops caused by missing modalities. Furthermore

a parallel dual-stream attention decoder (DSD) is integrated into skip connections to synergistically refine multi-scale fused features from both spatial and channel dimensions

significantly enhancing lesion discriminability and boundary detail recovery

thereby yielding superior accuracy and completeness in the final segmentation maps. Extensive comparative and ablation experiments conducted on the internationally authoritative BraTS2020 and BraTS2018 datasets comprehensively validate the superiority of the proposed method: When all modalities are complete

RNet outperforms existing methods in metrics such as Dice. In environments with randomly missing modalities

it still surpasses state-of-the-art methods specifically designed to handle missing modalities. This demonstrates strong robustness and significant potential for clinical deployment

providing an efficient and reliable novel paradigm for clinically practical multi-modal medical image segmentation.

关键词

Keywords

references

LITJENS G , KOOI T , BEJNORDI B E , et al . A survey on deep learning in medical image analysis [J ] . Medical Image Analysis , 2017 , 42 : 60 - 88 .

SHEN D G , WU G R , SUK H I . Deep learning in medical image analysis [J ] . Annual Review of Biomedical Engineering , 2017 , 19 : 221 - 248 .

XING Z H , YU L Q , WAN L , et al . NestedFormer: Nested modality-aware transformer for Brain tumor segmentation [C ] // Medical Image Computing and Computer Assisted Intervention - MICCAI 2022 . Cham : Springer , 2022 : 140 - 150 .

周新民 , 熊智谋 , 史长发 , 等 . 基于多尺度卷积调制的医学图像分割 [J ] . 电子学报 , 2024 , 52 ( 9 ): 3159 - 3171 .

ZHOU X M , XIONG Z M , SHI C F , et al . Medical image segmentation based on multi-scale convolution modulation [J ] . Acta Electronica Sinica , 2024 , 52 ( 9 ): 3159 - 3171 . (in Chinese)

雷涛 , 张峻铭 , 杜晓刚 , 等 . 基于混洗特征编码与门控解码的医学图像分割网络 [J ] . 电子学报 , 2024 , 52 ( 12 ): 4142 - 4152 .

LEI T , ZHANG J M , DU X G , et al . Medical image segmentation network based on shuffled feature encoding and gated decoding [J ] . Acta Electronica Sinica , 2024 , 52 ( 12 ): 4142 - 4152 . (in Chinese)

MENZE B H , JAKAB A , BAUER S , et al . The multimodal brain tumor image segmentation benchmark (BRATS) [J ] . IEEE Transactions on Medical Imaging , 2015 , 34 ( 10 ): 1993 - 2024 .

DING Y H , YU X , YANG Y . RFNet: Region-aware fusion network for incomplete multi-modal brain tumor segmentation [C ] // 2021 IEEE/CVF International Conference on Computer Vision . Piscataway : IEEE , 2022 : 3955 - 3964 .

DORENT R , JOUTARD S , MODAT M , et al . Hetero-modal variational encoder-decoder for joint modality completion and segmentation [C ] // Medical Image Computing and Computer Assisted Intervention - MICCAI 2019 . Cham : Springer , 2019 : 74 - 82 .

HAVAEI M , GUIZARD N , CHAPADOS N , et al . HeMIS: Hetero-modal image segmentation [C ] // Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016 . Cham : Springer , 2016 : 469 - 477 .

ZHANG Y , HE N J , YANG J W , et al . mmFormer: Multimodal medical transformer for incomplete multimodal learning of brain tumor segmentation [C ] // Medical Image Computing and Computer Assisted Intervention - MICCAI 2022 . Cham : Springer , 2022 : 107 - 117 .

LIU H , WEI D , LU D H , et al . M3AE: Multimodal representation learning for brain tumor segmentation with missing modalities [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2023 , 37 ( 2 ): 1657 - 1665 .

RONNEBERGER O , FISCHER P , BROX T . U-Net: Convolutional networks for biomedical image segmentation [C ] // Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015 . Cham : Springer , 2015 : 234 - 241 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [EB/OL ] . ( 2023-08-02 )[ 2025-10-10 ] . https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762 .

GU A , DAO T . Mamba: Linear-time sequence modeling with selective state spaces [EB/OL ] . ( 2024-05-31 )[ 2025-10-10 ] . https://arXiv.org/abs/2312.00752 https://arXiv.org/abs/2312.00752 .

LIU J R , YANG H , ZHOU H Y , et al . Swin-UMamba: Mamba-based UNet with ImageNet-based pretraining [C ] // Medical Image Computing and Computer Assisted Intervention - MICCAI 2024 . New York : ACM , 2024 : 615 - 625 .

MA J , LI F F , WANG B . U-mamba: Enhancing long-range dependency for biomedical image segmentation [EB/OL ] . ( 2024-01-09 )[ 2025-11-10 ] . https://arXiv.org/abs/2401.04722 https://arXiv.org/abs/2401.04722 .

ZHU L H , LIAO B C , ZHANG Q , et al . Vision Mamba: Efficient visual representation learning with bidirectional state space model [EB/OL ] . ( 2024-11-14 )[ 2025-10-10 ] . https://arXiv.org/abs/2401.09417 https://arXiv.org/abs/2401.09417 .

RUAN J , XIANG S . VM-UNet: Vision Mamba UNet for medical image segmentation [EB/OL ] . ( 2024-05-28 )[ 2025-11-10 ] . https://arXiv.org/abs/2402.15604 https://arXiv.org/abs/2402.15604 .

LIAO W B , ZHU Y H , WANG X Y , et al . LightM-UNet: Mamba assists in lightweight UNet for medical image segmentation [EB/OL ] . ( 2024-03-11 )[ 2025-11-10 ] . https://arXiv.org/abs/2403.05246 https://arXiv.org/abs/2403.05246 .

FAN C , YU H Y , HUANG Y , et al . SliceMamba with neural architecture search for medical image segmentation [J ] . IEEE Journal of Biomedical and Health Informatics , 2025 , 29 ( 10 ): 7446 - 7458 .

ZHOU Z W , RAHMAN SIDDIQUEE M M , TAJBAKHSH N , et al . UNet++: A nested U-Net architecture for medical image segmentation [C ] // Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support . Cham : Springer , 2018 : 3 - 11 .

OKTAY O , SCHLEMPER J , LE FOLGOC L , et al . Attention U-Net: Learning where to look for the pancreas [EB/OL ] . ( 2018-05-20 )[ 2025-11-10 ] . https://arXiv.org/abs/1804.03999 https://arXiv.org/abs/1804.03999 .

ÇIÇEK Ö , ABDULKADIR A , LIENKAMP S S , et al . 3D U-Net: Learning dense volumetric segmentation from sparse annotation [C ] // Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016 . Cham : Springer , 2016 : 424 - 432 .

MILLETARI F , NAVAB N , AHMADI S A . V-net: Fully convolutional neural networks for volumetric medical image segmentation [C ] // 2016 Fourth International Conference on 3D Vision . Piscataway : IEEE , 2016 : 565 - 571 .

ALOM M Z , YAKOPCIC C , HASAN M , et al . Recurrent residual U-Net for medical image segmentation [J ] . Journal of Medical Imaging , 2019 , 6 ( 1 ): 014006 .

ISENSEE F , PETERSEN J , KLEIN A , et al . Abstract: NnU-Net: Self-adapting framework for U-net-based medical image segmentation [M ] // Bildverarbeitung für die Medizin 2019 . Wiesbaden : Springer Fachmedien Wiesbaden , 2019 : 22 .

WANG W X , CHEN C , DING M , et al . TransBTS: Multimodal brain tumor segmentation using transformer [C ] // Medical Image Computing and Computer Assisted Intervention - MICCAI 2021 . Cham : Springer , 2021 : 109 - 119 .

HATAMIZADEH A , TANG Y C , NATH V , et al . UNETR: Transformers for 3D medical image segmentation [C ] // 2022 IEEE/CVF Winter Conference on Applications of Computer Vision . Piscataway : IEEE , 2022 : 1748 - 1758 .

XING Z H , YE T , YANG Y J , et al . SegMamba: Long-range sequential modeling Mamba for 3D medical image segmentation [C ] // Medical Image Computing and Computer Assisted Intervention - MICCAI 2024 . Cham : Springer , 2024 : 578 - 588 .

WOO S , PARK J , LEE J Y , et al . CBAM: Convolutional block attention module [C ] // Computer Vision - ECCV 2018 . Cham : Springer , 2018 : 3 - 19 .

VALANARASU J M J , PATEL V M . UNeXt: MLP-based rapid medical image segmentation network [C ] // Medical Image Computing and Computer Assisted Intervention - MICCAI 2022 . Cham : Springer , 2022 : 23 - 33 .

LIU F , ZHANG Y D , LU T , et al . Hierarchical in-out fusion for incomplete multimodal brain tumor segmentation [J ] . Scientific Reports , 2025 , 15 : 23017 .

XIONG Y Z , DU H W , ZHANG D K . A guided-learning network for brain tumor segmentation with missing modalities [C ] // Proceedings of the 2025 2nd International Conference on Generative Artificial Intelligence and Information Security . New York : ACM , 2025 : 219 - 225 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Research Progress of UAV Autonomous Landing Scene Perception Methods Based on Vision and Deep Learning

Cross-View Context-Aware Semi-Supervised Semantic Segmentation for High-Resolution Remote Sensing Images

DynKANet: Dynamic Fusion Network for Real-Time Semantic Segmentation Based on Kolmogorov-Arnold Representation Theorem

Variable Horizon Multi-Directional Scanning Method for Time Series Anomaly Detection

PHOTOLUMINESCENCE MEASUREMENT SYSTEM BASED ON AN IBM-PC MICROCOMPUTER

Related Author

WANG Zhong-tian

WU Yi-quan

LÜ Liang

LAN Jie

LAN Meng

LU Xian-kai

ZHANG Le-fei

LIANG Xin

Related Institution

College of Electronic and Information Engineering, Nanjing University of Aeronautics and Astronautics

School of Software, Shandong University

Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology

School of Computer Science, Wuhan University

School of Advanced Interdisciplinary, Hunan University of Technology and Business

⁰

SM3RNet: Spatial Mamba Based Missing Modality Robust Lesion Segmentation Network

DOI：10.12263/DZXB.20250677

摘要

Abstract

关键词

Keywords

references

SM³RNet: Spatial Mamba Based Missing Modality Robust Lesion Segmentation Network