面向肝细胞癌微血管侵犯评估的高效多模态贡献度感知网络研究

贾熹滨; 于高远; 王珞; 邓玉辉; 杨大为; 杨正汉

doi:10.12263/DZXB.20220919

您当前的位置：

首页 >

文章列表页 >

面向肝细胞癌微血管侵犯评估的高效多模态贡献度感知网络研究

学术论文 | 更新时间：2025-12-11

- 面向肝细胞癌微血管侵犯评估的高效多模态贡献度感知网络研究
- Efficient Multimodal Contribution Aware Network for Assessment of Microvascular Invasion in Hepatocellular Carcinoma
- 电子学报 2024年52卷第6期页码：2053-2066
- 作者机构：
  
  1.北京工业大学信息学部,北京 100124
  2.多媒体与智能软件技术北京市重点实验室(北京工业大学),北京 100124
  3.北京人工智能研究院(北京工业大学),北京 100124
  4.首都医科大学附属北京友谊医院放射科,北京 100050
- 作者简介：
  
  [ "贾熹滨女，1969年出生，山西太原人.现为北京工业大学信息学部计算机学院正教授.主要研究方向为视觉信息认知与计算、智能医学图像分析诊断、情感计算. E-mail: jiaxibin@bjut.edu.cn" ]
  [ "于高远男，1997年出生，河北石家庄人.现为北京工业大学信息学部计算机学院在读研究生.主要研究方向为机器学习、医学图像智能诊断、小样本学习. E-mail: yugaoyuan@emails.bjut.edu.cn" ]
  [ "王珞男，1990年出生，北京人.现为北京工业大学信息学部计算机学院讲师.主要研究方向为计算机视觉、深度学习、跨模态语义提取.E-mail: wangluo@bjut.edu.cn" ]
  [ "邓玉辉女，1985年出生，湖南郴州人.现为黑龙江省医院医学影像部副主任医师.主要研究方向为肝细胞癌微血管侵犯的深度学习.E-mail: yuhuida@163.com" ]
  [ "杨大为男，1983年出生，湖南岳阳人.现为首都医科大学附属北京友谊医院放射科副主任医师、副教授.主要研究方向为腹部疾病影像诊断与智能医学影像.E-mail: yangdawei@ccmu.edu.cn" ]
  [ "杨正汉男，1968年出生，浙江江山人.现为首都医科大学附属北京友谊医院放射科主任、正教授.主要研究方向为临床MRI检查技术及图像判读规范、消化系统疾病影像诊断与智能医学影像.E-mail: yangzhenghan@vip.163.com" ]
- 基金信息：
  
  国家自然科学基金(82071876;62171298;61871276);北京市医管中心青苗人才项目(QML20200108)
- DOI：10.12263/DZXB.20220919
  中图分类号： TP18;
- 收稿：2022-08-05，
  
  修回：2022-12-08，
  
  纸质出版：2024-06-25
- 稿件说明：
移动端阅览
贾熹滨, 于高远, 王珞, 等. 面向肝细胞癌微血管侵犯评估的高效多模态贡献度感知网络研究[J]. 电子学报, 2024, 52(06): 2053-2066.

JIA Xi-bin, YU Gao-yuan, WANG Luo, et al. Efficient Multimodal Contribution Aware Network for Assessment of Microvascular Invasion in Hepatocellular Carcinoma[J]. Acta Electronica Sinica, 2024, 52(06): 2053-2066.
贾熹滨, 于高远, 王珞, 等. 面向肝细胞癌微血管侵犯评估的高效多模态贡献度感知网络研究[J]. 电子学报, 2024, 52(06): 2053-2066. DOI：10.12263/DZXB.20220919

JIA Xi-bin, YU Gao-yuan, WANG Luo, et al. Efficient Multimodal Contribution Aware Network for Assessment of Microvascular Invasion in Hepatocellular Carcinoma[J]. Acta Electronica Sinica, 2024, 52(06): 2053-2066. DOI：10.12263/DZXB.20220919

摘要

微血管侵犯（MicroVascular Invasion，MVI）是肝细胞癌（HepatoCellular Carcinoma，HCC）切除或移植患者出现早期复发和长期预后不良的重要影响因素，因此在HCC患者术前评估是否存在MVI具有非常重要的临床价值.近年来，深度学习为MVI影像诊断评估提供了有价值的解决方法，但受数据标注收集困难等因素的影响，目前研究多独立利用电子计算机断层扫描（Computed Tomography，CT）或核磁共振成像（Magnetic Resonance Imaging，MRI）手段采集影像中的单模态序列，缺乏对各成像手段中多模态序列的综合应用.在小样本场景下，为有效利用多模态序列数据，提高诊断效能，本文提出一种高效多模态贡献度感知网络.该网络可以利用模态分组卷积和高效多模态自适应加权模块，在极少计算开销的引入下，学习CT或MRI的各模态影像信息在复杂多样的MVI表象下的诊断贡献.本文在三甲医院收集的临床数据集上进行实验，结果表明该网络模型可以在少量有标注数据的支持下，取得优于多种基于注意力机制的神经网络模型的MVI诊断性能，为专业医师的诊断分析提供了有效参考.

Abstract

Microvascular invasion (MVI) is an important factor for early recurrence and poor long-term prognosis in patients with hepatocellular carcinoma (HCC) after resection or transplantation. Therefore

it is of great clinical value to evaluate whether MVI exists in patients with HCC before operation. In recent years

deep learning has provided a valuable solution for MVI image diagnosis and evaluation. Nevertheless

due to the difficulties of data annotation and collection

the current researches mostly use computed tomography (CT) or magnetic resonance imaging (MRI) methods to collect single modal sequences in images independently

which lacks the comprehensive application of multimodal sequences in various imaging methods. In order to make more effective use of multimodal data of CT and MRI images and improve diagnosis efficiency under few-shot scenarios

an efficient multimodal montribution aware network is proposed in this paper. The modality grouping convolution and efficient multimodal adaptive weighting module in this network are used to to learn the diagnostic contribution of each modal information of CT or MRI under complex and diverse MVI representation with little computational cost introduced. The experiment is carried out on the clinical dataset collected by the third-class hospital. Result show that with the support of a small amount of labeled data

our method can achieve better MVI diagnostic performance than many deep neural networks based on attention mechanism

which provides an effective reference for professional doctors’ diagnostic analysis.

关键词

Keywords

references

GOLABI P , FAZEL S , OTGONSUREN M , et al . Mortality assessment of patients with hepatocellular carcinoma according to underlying disease and treatment modalities [J ] . Medicine , 2017 , 96 ( 9 ): e5904 .

EL-SERAG H B . Epidemiology of viral hepatitis and hepatocellular carcinoma [J ] . Gastroenterology , 2012 , 142 ( 6 ): 1264 - 1273 .

IGUCHI T , SHIRABE K , AISHIMA S , et al . New pathologic stratification of microvascular invasion in hepatocellular carcinoma: Predicting prognosis after living-donor liver transplantation [J ] . Transplantation , 2015 , 99 ( 6 ): 1236 - 1242 .

IMAMURA H , MATSUYAMA Y , TANAKA E , et al . Risk factors contributing to early and late phase intrahepatic recurrence of hepatocellular carcinoma after hepatectomy [J ] . Journal of Hepatology , 2003 , 38 ( 2 ): 200 - 207 .

CUCCHETTI A , PISCAGLIA F , CATURELLI E , et al . Comparison of recurrence of hepatocellular carcinoma after resection in patients with cirrhosis to its occurrence in a surveilled cirrhotic population [J ] . Annals of Surgical Oncology , 2009 , 16 ( 2 ): 413 - 422 .

SUMIE S , KUROMATSU R , OKUDA K , et al . Microvascular invasion in patients with hepatocellular carcinoma and its predictable clinicopathological factors [J ] . Annals of Surgical Oncology , 2008 , 15 ( 5 ): 1375 - 1382 .

OISHI K , ITAMOTO T , AMANO H , et al . Clinicopathologic features of poorly differentiated hepatocellular carcinoma [J ] . Journal of Surgical Oncology , 2007 , 95 ( 4 ): 311 - 316 .

MIN J H , LEE M W , PARK H S , et al . Interobserver variability and diagnostic performance of gadoxetic acid-enhanced MRI for predicting microvascular invasion in hepatocellular carcinoma [J ] . Radiology , 2020 , 297 ( 3 ): 573 - 581 .

WEI J W , JIANG H Y , ZENG M S , et al . Prediction of microvascular invasion in hepatocellular carcinoma via deep learning: A multi-center and prospective validation study‍ [J ] . Cancers , 2021 , 13 ( 10 ): 2368 .

ZHOU W , JIAN W W , CEN X P , et al . Prediction of microvascular invasion of hepatocellular carcinoma based on contrast-enhanced MR and 3D convolutional neural networks [J ] . Frontiers in Oncology , 2021 , 11 : 588010 .

ZHANG Y X , LV X F , QIU J L , et al . Deep learning with 3D convolutional neural network for noninvasive prediction of microvascular invasion in hepatocellular carcinoma [J ] . Journal of Magnetic Resonance Imaging , 2021 , 54 ( 1 ): 134 - 143 .

HU H T , ZHENG Q , HUANG Y , et al . A non-smooth tumor margin on preoperative imaging assesses microvascular invasion of hepatocellular carcinoma: A systematic review and meta-analysis [J ] . Scientific Reports , 2017 , 7 ( 1 ): 1 - 8 .

LIU S C , LAI J , HUANG J Y , et al . Predicting microvascular invasion in hepatocellular carcinoma: A deep learning model validated across hospitals [J ] . Cancer Imaging , 2021 , 21 ( 1 ): 1 - 16 .

HU J , SHEN L , SUN G . Squeeze-and-excitation networks‍ [C ] // Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 7132 - 7141 .

ZHANG Y L , LI K , LI K P , et al . MR image super-resolution with squeeze and excitation reasoning attention network [C ] // Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2021 : 13425 - 13434 .

CHEN Y , DU W W , DUAN X J , et al . Squeeze-and-excitation convolutional neural network for classification of malignant and benign lung nodules [J ] . Journal of Advances in Information Technology , 2021 , 12 ( 2 ): 153 - 158 .

WANG Q L , WU B G , ZHU P F , et al . ECA-Net: Efficient channel attention for deep convolutional neural networks [C ] // Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2020 : 11531 - 11539 .

KRIZHEVSKY A , SUTSKEVER I , HINTON G E . Imagenet classification with deep convolutional neural networks [J ] . Communications of the ACM , 2012 , 60 ( 6 ): 84 - 90 .

HE K M , ZHANG X Y , REN S Q , et al . Deep residual learning for image recognition [C ] // Conference on computer vision and pattern recognition . Piscataway : IEEE , 2016 : 770 - 778 .

YANG D W , JIA X B , XIAO Y J , et al . Noninvasive evaluation of the pathologic grade of hepatocellular carcinoma using MCF-3DCNN: A pilot study [J ] . BioMed Research International , 2019 , 2019( 1 ): 1 - 12 .

WOO S , PARK J , LEE J Y , et al . CBAM: Convolutional block attention module [C ] // European Conference on Computer Vision . Berlin : Springer , 2018 : 3 - 19 .

QIN Z , ZHANG P Y , WU F , et al . FcaNet: Frequency channel attention networks [C ] // International Conference on Computer Vision . Piscataway : IEEE , 2021 : 763 - 772 .

JIA X B , SUN Z , MI Q , et al . A multimodality-contribution-aware tripnet for histologic grading of hepatocellular carcinoma [J ] . IEEE/ACM Transactions on Computational Biology and Bioinformatics , 2021 , 19 ( 4 ): 2003 - 2016 .

GUO Z , LI X , HUANG H , et al . Deep learning-based image segmentation on multimodal medical imaging [J ] . IEEE Transactions on Radiation and Plasma Medical Sciences , 2019 , 3 ( 2 ): 162 - 169 .

ZHOU Q , ZHOU Z Y , CHEN C M , et al . Grading of hepatocellular carcinoma using 3D SE-DenseNet in dynamic enhanced MR images [J ] . Computers in Biology and Medicine , 2019 , 107 : 47 - 57 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于内容语义感知多模态融合的图像增强方法

基于层次化一致性语义学习的多模态意图识别

基于松弛Hadamard矩阵的多模态融合哈希方法