基于双模态知识协同驱动的教师-学生模型无造影剂CT肝脏肿瘤分割

刘婕; 徐晨初; 孙怡宁; 熊焰

doi:10.12263/DZXB.20240782

您当前的位置：

首页 >

文章列表页 >

基于双模态知识协同驱动的教师-学生模型无造影剂CT肝脏肿瘤分割

学术论文 | 更新时间：2025-08-18

- 基于双模态知识协同驱动的教师-学生模型无造影剂CT肝脏肿瘤分割
- Bimodal Knowledge Collaborative Driven Teacher-Student Model for Non-Contrast Agent CT Liver Tumor Segmentation
- 电子学报 2025年53卷第5期页码：1596-1606
- 作者机构：
  
  1.中国科学技术大学，安徽合肥 230026
  2.中国科学院合肥物质科学研究院，安徽合肥 230031
  3.安徽大学，安徽合肥 230601
- 作者简介：
  
  [ "刘婕女，1994年1月生，安徽亳州人.博士，安徽省立医院南区影像中心主治医师.主要研究方向为医学人工智能. E-mail: 528579064@qq.com" ]
  [ "徐晨初男，1987年6月生，安徽合肥人.安徽大学计算机科学与技术学院副教授、硕士生导师.主要研究方向为医疗人工智能、机器学习、医学图像分析." ]
  [ "孙怡宁男，1962年12月生，安徽太湖人.中科院合肥物质科学研究院研究员、博士生导师，合肥智能所首席科学家，运动与健康信息技术学科带头人.主要研究方向为光机电一体化技术、机器人技术、传感技术. E-mail: ynsun@iim.ac.cn" ]
  [ "熊焰男，1960年8月生，安徽合肥人.中国科学技术大学计算机科学与技术学院教授、博士生导师.主要研究方向为计算机网络与信息安全、移动计算与移动网络、分布式处理等. E-mail: yxiong@ustc.edu.cn" ]
- 基金信息：
  
  国家自然科学基金项目(62476001)
- DOI：10.12263/DZXB.20240782
  中图分类号： TP391.4;
- 收稿：2024-08-28，
  
  修回：2025-01-16，
  
  纸质出版：2025-05-25
- 稿件说明：
移动端阅览
刘婕, 徐晨初, 孙怡宁, 等. 基于双模态知识协同驱动的教师-学生模型无造影剂CT肝脏肿瘤分割[J]. 电子学报, 2025, 53(05): 1596-1606.

LIU Jie, XU Chen-chu, SUN Yi-ning, et al. Bimodal Knowledge Collaborative Driven Teacher-Student Model for Non-Contrast Agent CT Liver Tumor Segmentation[J]. Acta Electronica Sinica, 2025, 53(05): 1596-1606.
刘婕, 徐晨初, 孙怡宁, 等. 基于双模态知识协同驱动的教师-学生模型无造影剂CT肝脏肿瘤分割[J]. 电子学报, 2025, 53(05): 1596-1606. DOI：10.12263/DZXB.20240782

LIU Jie, XU Chen-chu, SUN Yi-ning, et al. Bimodal Knowledge Collaborative Driven Teacher-Student Model for Non-Contrast Agent CT Liver Tumor Segmentation[J]. Acta Electronica Sinica, 2025, 53(05): 1596-1606. DOI：10.12263/DZXB.20240782

摘要

无造影剂分割的CT（Computed Tomography）肝脏肿瘤图像在推进结直肠癌伴肝转移瘤筛查方面显示出巨大潜力，能够直接从无造影剂的CT图像中提供可靠的肝脏肿瘤分割结果，避免了造影剂的毒性、CT扫描的辐射和高昂的造影剂费用.本文提出了一种创新的“双模态知识协同驱动的教师-学生模型（Bimodal Knowledge Collaborative Driven Teacher-Student Model， BKC-TS）”，用于精准分割无造影剂的肝脏CT图像中的肝脏肿瘤，显著提升诊疗的安全性、准确性和时效性.BKC-TS利用教师网络学习显性肝脏肿瘤知识，指导学生网络从无造影剂图像中识别几乎不可见的肿瘤.它还协同临床检查文本数据和医学影像数据构建肝脏肿瘤知识，文本作为先验信息指引CT图像中肿瘤的学习过程，提升知识的精准性及指导的准确性.首先，文本-影像协同学习的教师学生框架通过引入文本知识，改进CT图像分辨率低的问题，提高无造影剂图像的肿瘤分割准确性.其次，双模知识融合传递模块通过知识提炼、知识融合和知识传递，深度整合影像和临床数据，有效支持学生网络在无造影环境中的肿瘤定位和识别.最后，高斯分布约束的学生自主学习策略采用学生网络异步迭代计算分割分布的方法，评估并筛选出有益知识，提高学生网络的自主学习能力、泛化能力和鲁棒性.所有实验均在一个泛化的数据集上进行，该数据集包含各1 140张增强前后的CT肝脏图像.实验结果表明，BKC-TS在肝脏肿瘤分割任务上获得了最佳性能（IOU提升至少2.17个百分点），证明了其在无造影剂肝脏肿瘤分割技术发展中的重要作用.

Abstract

Non-contrast CT (Computed Tomography) liver tumor imaging shows great potential in advancing the screening of colorectal cancer with liver metastasis. It provides reliable liver tumor segmentation from non-enhanced CT images

avoiding contrast agent toxicity

radiation

and costs. In this paper

we propose an innovative “teacher-student model driven by dual-modal knowledge collaboration (BKC-TS)”for accurately segmenting liver tumors in non-contrast CT images

significantly improving the safety

accuracy

and efficiency of liver tumor diagnosis and treatment. BKC-TS employs a teacher network to acquire explicit liver tumor knowledge and guide a student network in recognizing nearly invisible tumors from non-contrast images. It integrates clinical examination text data with medical imaging data. Text data

as prior information

guides tumor learning in CT images

enhancing precision and accuracy. The text-image collaborative learning teacher-student framework improves liver tumor segmentation accuracy in non-contrast images by integrating text knowledge and addressing CT image resolution issues. The dual-modal knowledge fusion and transmission module combines imaging and clinical data through knowledge extraction

fusion

and transmission

effectively supporting tumor localization and recognition in non-contrast images. The gaussian distribution-constrained student self-learning strategy boosts the student network’s independent learning

generalization

and robustness by iterating segmentation distribution and selecting beneficial knowledge. All experiments were conducted on a generalized dataset containing 1 140 CT liver images before and after enhancement. Experimental results show that BKC-TS achieved optimal liver tumor segmentation (at least a 2.17 percentage points IOU improvement)

demonstrating its importance in non-contrast technology development.

关键词

Keywords

references

HUANG Q , SUN J F , DING H , et al . Robust liver vessel extraction using 3D U-Net with variant dice loss function [J ] . Computers in Biology and Medicine , 2018 , 101 : 153 - 162 .

VORONTSOV E , TANG A , PAL C , et al . Liver lesion segmentation informed by joint liver segmentation [C ] // 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) . Piscataway : IEEE , 2018 : 1332 - 1335 .

ALMOTAIRI S , KAREEM G , AOUF M , et al . Liver tumor segmentation in CT scans using modified SegNet [J ] . Sensors , 2020 , 20 ( 5 ): 1516 .

KAVUR A E , GEZER N S , BARIS M , et al . CHAOS challenge-combined (CT-MR) healthy abdominal organ segmentation [J ] . Medical Image Analysis , 2021 , 69 : 101950 .

KÜSTNER T , HEPP T , FISCHER M , et al . Fully automated and standardized segmentation of adipose tissue compartments via deep learning in 3D whole-body MRI of epidemiologic cohort studies [J ] . Artificial Intelligence , 2020 , 2 ( 6 ): e200010 .

LI B N , CHUI C K , CHANG S , et al . A new unified level set method for semi-automatic liver tumor segmentation on contrast-enhanced CT images [J ] . Expert Systems with Applications , 2012 , 39 ( 10 ): 9661 - 9668 .

SUN C J , GUO S X , ZHANG H M , et al . Automatic segmentation of liver tumors from multiphase contrast-enhanced CT images based on FCNs [J ] . Artificial Intelligence in Medicine , 2017 , 83 : 58 - 66 .

NASEEM R , KHAN Z A , SATPUTE N , et al . Cross-modality guided contrast enhancement for improved liver tumor image segmentation [J ] . IEEE Access , 2021 , 9 : 118154 - 118167 .

ZHOU J Y , WONG D W K , DING F , et al . Liver tumour segmentation using contrast-enhanced multi-detector CT data: Performance benchmarking of three semiautomated methods [J ] . European Radiology , 2010 , 20 ( 7 ): 1738 - 1748 .

ZHANG D , CHEN B , CHONG J , et al . Weakly-Supervised teacher-Student network for liver tumor segmentation from non-enhanced images [J ] . Medical Image Analysis , 2021 , 70 : 102005 .

LYU P J , WANG C , FANG W , et al . Registration-propagated liver tumor segmentation for non-enhanced CT-bBased interventions [M ] // Image and Graphics . Cham : Springer Nature Switzerland , 2023 : 124 - 136 .

ZHANG J N , LUO S C , QIANG Y , et al . Edge constraint and location mapping for liver tumor segmentation from nonenhanced images [J ] . Computational and Mathematical Methods in Medicine , 2022 , 2022 : 1248311 .

XIAO X J , HU Q V , WANG G H . Edge-Aware Multi-Task Network for Integrating Quantification Segmentation and Uncertainty Prediction of Liver Tumor on Multi-Modality Non-Contrast MRI [M ] // Medical Image Computing and Computer Assisted Intervention-MICCAI 2023 . Cham : Springer Nature Switzerland , 2023 : 652 - 661 .

WANG W Z , SONG Q Y , ZHOU J R , et al . Dual-level selective transfer learning for intrahepatic cholangiocarcinoma segmentation in non-enhanced abdominal CT [M ] // Medical Image Computing and Computer Assisted Intervention-MICCAI 2020 . Cham : Springer International Publishing , 2020 : 64 - 73 .

GJESTANG H L , HICKS S A , THAMBAWITA V , et al . A self-learning teacher-student framework for gastrointestinal image classification [C ] // 2021 IEEE 34th International Symposium on Computer-Based Medical Systems (CBMS) . Piscataway : IEEE , 2021 : 539 - 544 .

WANG L , YOON K J . Knowledge distillation and student-teacher learning for visual intelligence: A review and new outlooks [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022 , 44 ( 6 ): 3048 - 3068 .

LIU Z , LIN Y T , CAO Y , et al . Swin transformer: Hierarchical vision transformer using shifted windows [C ] // 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2021 : 9992 - 10002 .

WANG T , YUAN L , ZHANG X P , et al . Distilling object detectors with fine-grained feature imitation [C ] // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2019 : 4933 - 4942 .

YANG X Y , LI Q J , ZHANG C , et al . Knowledge distillation from multiple foundation models for end-to-end speech recognition [EB/OL ] . ( 2023-03-20 )[ 2025-3-11 ] . https://arxiv.org/abs/2303.10917v1 https://arxiv.org/abs/2303.10917v1 .

ANAND D , GUPTA V , PARUCHURI P , et al . An enhanced advising model in teacher-student framework using state categorization [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2021 , 35 ( 8 ): 6653 - 6660 .

CHEN H T , WANG Y H , XU C , et al . Learning student networks via feature embedding [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2021 , 32 ( 1 ): 25 - 35 .

GOU J P , YU B S , MAYBANK S J , et al . Knowledge distillation: A survey [J ] . International Journal of Computer Vision , 2021 , 129 ( 6 ): 1789 - 1819 .

RADOSAVOVIC I , KOSARAJU R P , GIRSHICK R , et al . Designing network design spaces [C ] // 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2020 : 10428 - 10436 .

LANG C B , CHENG G , TU B F , et al . Learning what not to segment: A new perspective on few-shot segmentation [C ] // 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2022 : 8047 - 8057 .

YU L Q , WANG S J , LI X M , et al . Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation [M ] // Medical Image Computing and Computer Assisted Intervention-MICCAI 2019 . Cham : Springer International Publishing , 2019 : 605 - 613 .

AHMAD USMANI U , WATADA J , JAAFAR J , et al . A reinforced active learning algorithm for semantic segmentation in complex imaging [J ] . IEEE Access , 2021 , 9 : 168415 - 168432 .

RAKESH V , JAIN S . Efficacy of Bayesian neural networks in active learning [C ] // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . Piscataway : IEEE , 2021 : 2601 - 2609 .

FENG D H , CHEN X , WANG X Y , et al . Penalized Entropy: A novel loss function for uncertainty estimation and optimization in medical image classification [C ] // 2022 IEEE 35th International Symposium on Computer-Based Medical Systems (CBMS) . Piscataway : IEEE , 2022 : 306 - 310 .

LIU X B , SONG L P , LIU S , et al . A review of deep-learning-based medical image segmentation methods [J ] . Sustainability , 2021 , 13 ( 3 ): 1224 .

ZHANG G S , ZHAO K H , HONG Y F , et al . SHA-MTL: Soft and hard attention multi-task learning for automated breast cancer ultrasound image segmentation and classification [J ] . International Journal of Computer Assisted Radiology and Surgery , 2021 , 16 ( 10 ): 1719 - 1725 .

LOU A G , GUAN S Y , KO H , et al . CaraNet: Context axial reverse attention network for segmentation of small medical objects [C ] // Medical Imaging 2022: Image Processing . California : SPIE , 2022 : 81 - 92 .

ZHOU T , ZHOU Y , HE K L , et al . Cross-level feature aggregation network for polyp segmentation [J ] . Pattern Recognition , 2023 , 140 : 109555 .

FAN D P , JI G P , ZHOU T , et al . PraNet: Parallel reverse attention network for polyp segmentation [M ] // Medical Image Computing and Computer Assisted Intervention-MICCAI 2020 . Cham : Springer International Publishing , 2020 : 263 - 273 .

Kim T , LEE H , KIM D . Uacanet: Uncertainty augmented context attention for polyp segmentation [EB/OL ] . ( 2021-07-22 )[ 2025-03-11 ] . https://arxiv.org/abs/2107.02368v3 https://arxiv.org/abs/2107.02368v3 .

XU C C , WU X , WANG B Y , et al . Accurate segmentation of liver tumor from multi-modality non-contrast images using a dual-stream multi-level fusion framework [J ] . Computerized Medical Imaging and Graphics , 2024 , 116 : 102414 .

ZHANG R F , LI G B , LI Z , et al . Adaptive context selection for polyp segmentation [M ] // Medical Image Computing and Computer Assisted Intervention-MICCAI 2020 . Cham : Springer International Publishing , 2020 , 1 : 253 - 262 .

HU C F , XIA T Y , CUI Y , et al . Trustworthy multi-phase liver tumor segmentation via evidence-based uncertainty [J ] . Engineering Applications of Artificial Intelligence , 2024 , 133 : 108289 .

LIU H X , YANG J , JIANG C , et al . S2DA-Net: Spatial and spectral-learning double-branch aggregation network for liver tumor segmentation in CT images [J ] . Computers in Biology and Medicine , 2024 , 174 : 108400 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

暂无数据