Efficient Multimodal Contribution Aware Network for Assessment of Microvascular Invasion in Hepatocellular Carcinoma

JIA Xi-bin; YU Gao-yuan; WANG Luo; DENG Yu-hui; YANG Da-wei; YANG Zheng-han

doi:10.12263/DZXB.20220919

您当前的位置：

首页 >

文章列表页 >

Efficient Multimodal Contribution Aware Network for Assessment of Microvascular Invasion in Hepatocellular Carcinoma

PAPERS | 更新时间：2025-12-11

- Efficient Multimodal Contribution Aware Network for Assessment of Microvascular Invasion in Hepatocellular Carcinoma
- ACTA ELECTRONICA SINICA Vol. 52, Issue 6, Pages: 2053-2066(2024)
- 作者机构：
  
  1.北京工业大学信息学部,北京 100124
  2.多媒体与智能软件技术北京市重点实验室(北京工业大学),北京 100124
  3.北京人工智能研究院(北京工业大学),北京 100124
  4.首都医科大学附属北京友谊医院放射科,北京 100050
- 作者简介：
- 基金信息：
  
  The National Natural Science Foundation of China(82071876;62171298;61871276);Beijing Medical Management Center Young Talents Project(QML20200108)
- DOI：10.12263/DZXB.20220919
  CLC： TP18;
- Received：05 August 2022，
  
  Revised：2022-12-08，
  
  Published：25 June 2024
- 稿件说明：
移动端阅览
贾熹滨, 于高远, 王珞, 等. 面向肝细胞癌微血管侵犯评估的高效多模态贡献度感知网络研究[J]. 电子学报, 2024, 52(06): 2053-2066.

JIA Xi-bin, YU Gao-yuan, WANG Luo, et al. Efficient Multimodal Contribution Aware Network for Assessment of Microvascular Invasion in Hepatocellular Carcinoma[J]. Acta Electronica Sinica, 2024, 52(06): 2053-2066.
贾熹滨, 于高远, 王珞, 等. 面向肝细胞癌微血管侵犯评估的高效多模态贡献度感知网络研究[J]. 电子学报, 2024, 52(06): 2053-2066. DOI：10.12263/DZXB.20220919

JIA Xi-bin, YU Gao-yuan, WANG Luo, et al. Efficient Multimodal Contribution Aware Network for Assessment of Microvascular Invasion in Hepatocellular Carcinoma[J]. Acta Electronica Sinica, 2024, 52(06): 2053-2066. DOI：10.12263/DZXB.20220919

摘要

微血管侵犯（MicroVascular Invasion，MVI）是肝细胞癌（HepatoCellular Carcinoma，HCC）切除或移植患者出现早期复发和长期预后不良的重要影响因素，因此在HCC患者术前评估是否存在MVI具有非常重要的临床价值.近年来，深度学习为MVI影像诊断评估提供了有价值的解决方法，但受数据标注收集困难等因素的影响，目前研究多独立利用电子计算机断层扫描（Computed Tomography，CT）或核磁共振成像（Magnetic Resonance Imaging，MRI）手段采集影像中的单模态序列，缺乏对各成像手段中多模态序列的综合应用.在小样本场景下，为有效利用多模态序列数据，提高诊断效能，本文提出一种高效多模态贡献度感知网络.该网络可以利用模态分组卷积和高效多模态自适应加权模块，在极少计算开销的引入下，学习CT或MRI的各模态影像信息在复杂多样的MVI表象下的诊断贡献.本文在三甲医院收集的临床数据集上进行实验，结果表明该网络模型可以在少量有标注数据的支持下，取得优于多种基于注意力机制的神经网络模型的MVI诊断性能，为专业医师的诊断分析提供了有效参考.

Abstract

Microvascular invasion (MVI) is an important factor for early recurrence and poor long-term prognosis in patients with hepatocellular carcinoma (HCC) after resection or transplantation. Therefore

it is of great clinical value to evaluate whether MVI exists in patients with HCC before operation. In recent years

deep learning has provided a valuable solution for MVI image diagnosis and evaluation. Nevertheless

due to the difficulties of data annotation and collection

the current researches mostly use computed tomography (CT) or magnetic resonance imaging (MRI) methods to collect single modal sequences in images independently

which lacks the comprehensive application of multimodal sequences in various imaging methods. In order to make more effective use of multimodal data of CT and MRI images and improve diagnosis efficiency under few-shot scenarios

an efficient multimodal montribution aware network is proposed in this paper. The modality grouping convolution and efficient multimodal adaptive weighting module in this network are used to to learn the diagnostic contribution of each modal information of CT or MRI under complex and diverse MVI representation with little computational cost introduced. The experiment is carried out on the clinical dataset collected by the third-class hospital. Result show that with the support of a small amount of labeled data

our method can achieve better MVI diagnostic performance than many deep neural networks based on attention mechanism

which provides an effective reference for professional doctors’ diagnostic analysis.

关键词

Keywords

references

GOLABI P , FAZEL S , OTGONSUREN M , et al . Mortality assessment of patients with hepatocellular carcinoma according to underlying disease and treatment modalities [J ] . Medicine , 2017 , 96 ( 9 ): e5904 .

EL-SERAG H B . Epidemiology of viral hepatitis and hepatocellular carcinoma [J ] . Gastroenterology , 2012 , 142 ( 6 ): 1264 - 1273 .

IGUCHI T , SHIRABE K , AISHIMA S , et al . New pathologic stratification of microvascular invasion in hepatocellular carcinoma: Predicting prognosis after living-donor liver transplantation [J ] . Transplantation , 2015 , 99 ( 6 ): 1236 - 1242 .

IMAMURA H , MATSUYAMA Y , TANAKA E , et al . Risk factors contributing to early and late phase intrahepatic recurrence of hepatocellular carcinoma after hepatectomy [J ] . Journal of Hepatology , 2003 , 38 ( 2 ): 200 - 207 .

CUCCHETTI A , PISCAGLIA F , CATURELLI E , et al . Comparison of recurrence of hepatocellular carcinoma after resection in patients with cirrhosis to its occurrence in a surveilled cirrhotic population [J ] . Annals of Surgical Oncology , 2009 , 16 ( 2 ): 413 - 422 .

SUMIE S , KUROMATSU R , OKUDA K , et al . Microvascular invasion in patients with hepatocellular carcinoma and its predictable clinicopathological factors [J ] . Annals of Surgical Oncology , 2008 , 15 ( 5 ): 1375 - 1382 .

OISHI K , ITAMOTO T , AMANO H , et al . Clinicopathologic features of poorly differentiated hepatocellular carcinoma [J ] . Journal of Surgical Oncology , 2007 , 95 ( 4 ): 311 - 316 .

MIN J H , LEE M W , PARK H S , et al . Interobserver variability and diagnostic performance of gadoxetic acid-enhanced MRI for predicting microvascular invasion in hepatocellular carcinoma [J ] . Radiology , 2020 , 297 ( 3 ): 573 - 581 .

WEI J W , JIANG H Y , ZENG M S , et al . Prediction of microvascular invasion in hepatocellular carcinoma via deep learning: A multi-center and prospective validation study‍ [J ] . Cancers , 2021 , 13 ( 10 ): 2368 .

ZHOU W , JIAN W W , CEN X P , et al . Prediction of microvascular invasion of hepatocellular carcinoma based on contrast-enhanced MR and 3D convolutional neural networks [J ] . Frontiers in Oncology , 2021 , 11 : 588010 .

ZHANG Y X , LV X F , QIU J L , et al . Deep learning with 3D convolutional neural network for noninvasive prediction of microvascular invasion in hepatocellular carcinoma [J ] . Journal of Magnetic Resonance Imaging , 2021 , 54 ( 1 ): 134 - 143 .

HU H T , ZHENG Q , HUANG Y , et al . A non-smooth tumor margin on preoperative imaging assesses microvascular invasion of hepatocellular carcinoma: A systematic review and meta-analysis [J ] . Scientific Reports , 2017 , 7 ( 1 ): 1 - 8 .

LIU S C , LAI J , HUANG J Y , et al . Predicting microvascular invasion in hepatocellular carcinoma: A deep learning model validated across hospitals [J ] . Cancer Imaging , 2021 , 21 ( 1 ): 1 - 16 .

HU J , SHEN L , SUN G . Squeeze-and-excitation networks‍ [C ] // Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 7132 - 7141 .

ZHANG Y L , LI K , LI K P , et al . MR image super-resolution with squeeze and excitation reasoning attention network [C ] // Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2021 : 13425 - 13434 .

CHEN Y , DU W W , DUAN X J , et al . Squeeze-and-excitation convolutional neural network for classification of malignant and benign lung nodules [J ] . Journal of Advances in Information Technology , 2021 , 12 ( 2 ): 153 - 158 .

WANG Q L , WU B G , ZHU P F , et al . ECA-Net: Efficient channel attention for deep convolutional neural networks [C ] // Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2020 : 11531 - 11539 .

KRIZHEVSKY A , SUTSKEVER I , HINTON G E . Imagenet classification with deep convolutional neural networks [J ] . Communications of the ACM , 2012 , 60 ( 6 ): 84 - 90 .

HE K M , ZHANG X Y , REN S Q , et al . Deep residual learning for image recognition [C ] // Conference on computer vision and pattern recognition . Piscataway : IEEE , 2016 : 770 - 778 .

YANG D W , JIA X B , XIAO Y J , et al . Noninvasive evaluation of the pathologic grade of hepatocellular carcinoma using MCF-3DCNN: A pilot study [J ] . BioMed Research International , 2019 , 2019( 1 ): 1 - 12 .

WOO S , PARK J , LEE J Y , et al . CBAM: Convolutional block attention module [C ] // European Conference on Computer Vision . Berlin : Springer , 2018 : 3 - 19 .

QIN Z , ZHANG P Y , WU F , et al . FcaNet: Frequency channel attention networks [C ] // International Conference on Computer Vision . Piscataway : IEEE , 2021 : 763 - 772 .

JIA X B , SUN Z , MI Q , et al . A multimodality-contribution-aware tripnet for histologic grading of hepatocellular carcinoma [J ] . IEEE/ACM Transactions on Computational Biology and Bioinformatics , 2021 , 19 ( 4 ): 2003 - 2016 .

GUO Z , LI X , HUANG H , et al . Deep learning-based image segmentation on multimodal medical imaging [J ] . IEEE Transactions on Radiation and Plasma Medical Sciences , 2019 , 3 ( 2 ): 162 - 169 .

ZHOU Q , ZHOU Z Y , CHEN C M , et al . Grading of hepatocellular carcinoma using 3D SE-DenseNet in dynamic enhanced MR images [J ] . Computers in Biology and Medicine , 2019 , 107 : 47 - 57 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Image Enhancement via Content Semantic-Aware Multimodal Fusion

Multimodal Intent Recognition Based on Hierarchical Semantic-Consistency Learning

Multimodal Fusion Hash Learning Method Based on Relaxed Hadamard Matrix

Related Author

LI Lei-da

ZHU Han-cheng

LIU Xin-yu

YAO Rui

SHAO Zhi-wen

ZHOU Yong

WANG Lan

ZHANG Huan-xiang

Related Institution

School of Artificial Intelligence, Xidian University

Mine Digitization Engineering Research Center of the Ministry of Education

School of Computer Science and Technology / School of Artificial Intelligence, China University of Mining and Technology

School of Innovation and Entrepreneurship Education, Inner Mongolia University of Science and Technology

School of Computer Engineering and Science, Shanghai University

⁰