医学图像关键点检测深度学习方法研究与挑战

李居朋; 王颖慧; 李刚

doi:10.12263/DZXB.20200725

您当前的位置：

首页 >

文章列表页 >

医学图像关键点检测深度学习方法研究与挑战

综述评论 | 更新时间：2025-12-08

- 医学图像关键点检测深度学习方法研究与挑战
- Research and Challenges of Medical Image Landmark Detection Based on Deep Learning
- 电子学报 2022年50卷第1期页码：226-237
- 作者机构：
  
  1.北京交通大学电子信息工程学院，北京 100044
  2.北京大学口腔医学院，北京 100081
- 作者简介：
  
  [ "李居朋男，1981年8月出生于江苏省沛县.2009年获得北京交通大学大学博士学位.2019年赴北卡罗来纳大学教堂山分校研修，现为北京交通大学副教授、研究生导师，主要研究方向为医学图像分割及多模态配准等. E-mail:lijupeng@bjtu.edu.cn" ]
  [ "王颖慧女，1992年1月出生于河北省保定市.2018年毕业于首都医科大学长学制（本硕连读）口腔医学专业.现为北京大学口腔医学院口腔颌面医学影像学在读博士研究生，主要研究方向为数字化口腔颌面医学影像." ]
  [ "李刚男，1970年2月出生于黑龙江省哈尔滨市.2004年于瑞典卡罗林斯卡医学院获博士学位.现为北京大学教授、博士生导师，中国电子学会会员，中华口腔医学会口腔颌面放射专委会副主任委员，口腔医学院医学影像科副主任，主要研究方向为数字化口腔医学影像的基础和临床研究等.E-mail:kqgang@bjmu.edu.cn" ]
- 基金信息：
  
  国家自然科学基金(81671034);军队后勤科研重大项目(AWS17J011);国家留学基金(201907095037)
- DOI：10.12263/DZXB.20200725
  中图分类号： TP391.4;
- 收稿：2020-07-16，
  
  修回：2020-12-17，
  
  纸质出版：2022-01-25
- 稿件说明：
移动端阅览
李居朋,王颖慧,李刚.医学图像关键点检测深度学习方法研究与挑战[J].电子学报,2022,50(01):226-237.

LI Ju-peng,WANG Ying-hui,LI Gang.Research and Challenges of Medical Image Landmark Detection Based on Deep Learning[J].ACTA ELECTRONICA SINICA,2022,50(01):226-237.
李居朋,王颖慧,李刚.医学图像关键点检测深度学习方法研究与挑战[J].电子学报,2022,50(01):226-237. DOI： 10.12263/DZXB.20200725.

LI Ju-peng,WANG Ying-hui,LI Gang.Research and Challenges of Medical Image Landmark Detection Based on Deep Learning[J].ACTA ELECTRONICA SINICA,2022,50(01):226-237. DOI： 10.12263/DZXB.20200725.

摘要

作为众多医学图像处理的前提和关键，医学图像关键点检测具有重要的理论研究和应用价值.由于个体间差异性和个体内歧义性的影响，以及更高的临床应用定位精度的要求，医学解剖关键点检测面临着巨大的挑战.鉴于深度学习技术在医学图像关键点检测乃至整个医学图像处理领域都表现出了强大的实力，本文全面检索发表于顶级医学期刊和会议论文集中的医学图像关键点研究成果并进行了详细的梳理和综述.从计算机视觉任务角度简述医学图像关键点检测及其存在的难点；总结了深度学习技术在医学图像关键点检测中的基本框架，详细论述了医学图像关键点检测的分类问题和回归分析两种不同类型的解决思路；最后探讨了医学图像关键点检测深度学习方法面临的挑战、主要应对策略和开放的研究方向.

Abstract

As an entrance and challenge for many medical images processing

it is clinically essential to study on the medical image landmark detection and localization. Due to inter-individual variability and intra-individual ambiguity

as well as higher accuracy requirements of clinical application

the detection of medical anatomical landmarks was facing enormous challenges. In view of strength of deep learning of medical image landmark detection and the entire medical image processing field

we comprehensively retrieves relevant papers published in the top medical journals and conference proceedings to conduct a detailed review of these research findings. First of all

we briefly introduce difficulties in medical image landmark detection from the view of computer vision tasks. Secondly

we describe basic framework in medical image landmark detection

and discuss two different categories: classification and regression landmark detection solutions. Finally

we discuss the challenges and practicable strategies in deep learning for medical image landmark detection

as well as open research.

关键词

Keywords

references

HANAOKA S , SHIMIZU A , NEMOTO M , et al . Automatic detection of over 100 anatomical landmarks in medical CT images: A framework with independent detectors and combinatorial optimization [J]. Medical Image Analysis , 2017 , 35 : 192 - 214 .

GE J W , SAEIDI H , OPFERMANN J D , et al . Landmark-guided deformable image registration for supervised autonomous robotic tumor resection [C]// Medical Image Computing and Computer Assisted Intervention-MICCAI 2019 . Cham, GER : Springer , 2019 : 320 - 328 .

PETERLÍK I , COURTECUISSE H , ROHLING R , et al . Fast elastic registration of soft tissues under large deformations [J]. Medical Image Analysis , 2018 , 45 : 24 - 40 .

MADER A O , BERG J , FABRITZ A , et al . Localization and labeling of posterior ribs in chest radiographs using a CRF-regularized FCN with local refinement [C]// Medical Image Computing and Computer Assisted Intervention-MICCAI 2018 . Cham, GER : Springer , 2018 : 562 - 570 .

TOROSDAGLI N , LIBERTON D K , VERMA P , et al . Deep geodesic learning for segmentation and anatomical landmarking [J]. IEEE Transactions on Medical Imaging , 2019 , 38 ( 4 ): 919 - 931 .

ZHANG J , LIU M , WANG L , et al . Joint craniomaxillofacial bone segmentation and landmark digitization by context-guided fully convolutional networks [J]. Medical Image Computing and Computer-Assisted Intervention , 2017 , 10434 : 720 - 728 .

ZHANG J , LIU M X , WANG L , et al . Context-guided fully convolutional networks for joint craniomaxillofacial bone segmentation and landmark digitization [J]. Medical Image Analysis , 2020 , 60 : 101621 .

ZHANG J , SAHA A , ZHU Z , et al . Hierarchical convolutional neural networks for segmentation of breast tumors in MRI with application to radiogenomics [J]. IEEE Transactions on Medical Imaging , 2019 , 38 ( 2 ): 435 - 447 .

LIU C B , XIE H T , ZHANG S C , et al . Misshapen pelvis landmark detection by spatial local correlation mining for diagnosing developmental dysplasia of the hip [C]// Medical Image Computing and Computer Assisted Intervention-MICCAI 2019 . Cham, GER : Springer , 2019 : 441 - 449 .

ZHENG Y F , LIU D , GEORGESCU B , et al . 3D deep learning for efficient and robust landmark detection in volumetric data [C]// Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015 . Cham, GER : Springer , 2015 : 565 - 572 .

AUBERT B , VAZQUEZ C , CRESSON T , et al . Toward automated 3D spine reconstruction from biplanar radiographs using CNN for statistical spine model fitting [J]. IEEE Transactions on Medical Imaging , 2019 , 38 ( 12 ): 2796 - 2806 .

NOOTHOUT J M H , VOS B D D , WOLTERINK J M , et al . CNN-based landmark detection in cardiac CTA scans [EB/OL]. [2018] . https://arxiv.org/abs/1804.04963 https://arxiv.org/abs/1804.04963 .

GREWAL M , DEIST T M , WIERSMA J , et al . An end-to-end deep learning approach for landmark detection and matching in medical images [EB/OL]. [2020] . https://arxiv.org/abs/2001.07434 https://arxiv.org/abs/2001.07434 .

ZHANG J , LIU M X , SHEN D G . Detecting anatomical landmarks from limited medical imaging data using two-stage task-oriented deep neural networks [J]. IEEE Transactions on Image Processing , 2017 , 26 ( 10 ): 4753 - 4764 .

DUAN J M , BELLO G , SCHLEMPER J , et al . Automatic 3D Bi-ventricular segmentation of cardiac images by a shape-refined multi- task deep learning approach [J]. IEEE Transactions on Medical Imaging , 2019 , 38 ( 9 ): 2151 - 2164 .

TIULPIN A , MELEKHOW I , SAARAKKALA S . Knee anatomical landmark localization using hourglass networks [EB/OL]. [2019] . https://arxiv.org/abs/1907.12237 https://arxiv.org/abs/1907.12237 .

IBRAGIMOV B , LIKAR B , PEMUS F , et al . Automatic cephalometric x-ray landmark detection by applying game theory and random forests [C]// Proceedings of ISBI International Symposium on Biomedical Imaging . Cham, GER : Springer , 2014 : 1 - 8 .

LINDNER C , COOTES T F . Fully automatic cephalometric evaluation using random forest regression-voting [C]// Proceedings of ISBI International Symposium on Biomedical Imaging . Cham, GER : Springer , 2015 : 1 - 8 .

WANG C W , HHANG C T , HSIEH M C , et al . Evaluation and comparison of anatomical landmark detection methods for cephalometric X-ray images: A grand challenge [J]. IEEE Transactions on Medical Imaging , 2015 , 34 ( 9 ): 1890 - 1900 .

WANG C W , HUANG C T , LEE J H , et al . A benchmark for comparison of dental radiography analysis algorithms [J]. Medical Image Analysis , 2016 , 31 : 63 - 76 .

BIER B , GOLDMANN F , ZAECH J N , et al . Learning to detect anatomical landmarks of the pelvis in X-rays from arbitrary views [J]. International Journal of Computer Assisted Radiology and Surgery , 2019 , 14 ( 9 ): 1463 - 1473 .

YANG D , XIONG T , XU D G , et al . Automatic vertebra labeling in large-scale 3D CT using deep image-to-image network with message passing and sparsity regularization [C]// Information Processing in Medical Imaging . Cham, GER : Springer , 2017 : 633 - 644 .

PROBST T , MANINIS K K , CHHATKULI A , et al . Automatic tool landmark detection for stereo vision in robot-assisted retinal surgery [J]. IEEE Robotics and Automation Letters , 2018 , 3 ( 1 ): 612 - 619 .

TEIXEIRA B , TAMERSOY B , SINGH V , et al . Adaloss: Adaptive loss function for landmark localization [EB/OL]. [2019] . http://arxiv-export-lb.library.cornell.edu/pdf/1908.01070 http://arxiv-export-lb.library.cornell.edu/pdf/1908.01070 .

ZHANG D Q , WANG J N , NOBLE J H , et al . HeadLocNet: Deep convolutional neural networks for accurate classification and multi-landmark localization of head CTs [J]. Medical Image Analysis , 2020 , 61 : 101659 .

URSCHLER M , EBNER T , ŠTERN D . Integrating geometric configuration and appearance information into a unified framework for anatomical landmark localization [J]. Medical Image Analysis , 2018 , 43 : 23 - 36 .

CHEN R N , MA Y X , CHEN N L , et al . Cephalometric landmark detection by attentive feature pyramid fusion and regression-voting [C]// Medical Image Computing and Computer Assisted Intervention-MICCAI 2019 . Cham, GER : Springer , 2019 : 873 - 881 .

LÉVY-MANDEL A D , VENETSANOPOULOS A N , TSOTSOS J K . Knowledge-based landmarking of cephalograms [J]. Computers and Biomedical Research , 1986 , 19 ( 3 ): 282 - 309 .

EL-FEGHI I , SID-AHMED M A , AHMADI M . Automatic localization of craniofacial landmarks for assisted cephalometry [J]. Pattern Recognition , 2004 , 37 ( 3 ): 609 - 621 .

CARDILLO J , SID-AHMED M A . An image processing system for locating craniofacial landmarks [J]. IEEE Transactions on Medical Imaging , 1994 , 13 ( 2 ): 275 - 289 .

COOTES T F , TAYLOR C J , COOPER D H , et al . Active shape models-their training and application [J]. Computer Vision and Image Understanding , 1995 , 61 ( 1 ): 38 - 59 .

COOTES T F , EDWARDS G J , TAYLOR C J . Active appearance models [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2001 , 23 ( 6 ): 681 - 685 .

YUE W , YIN D , LI C , et al . Automated 2-D cephalometric analysis on X-ray images by a model-based approach [J]. IEEE Transactions on Biomedical Engineering , 2006 , 53 ( 8 ): 1615 - 1623 .

LINDNER C , WANG C W , HUANG C T , et al . Fully automatic system for accurate localisation and analysis of cephalometric landmarks in lateral cephalograms [J]. Scientific Reports , 2016 , 6 : 33581 .

ARlK S Ö , IBRAGIMOV B , XING L . Fully automated quantitative cephalometry using convolutional neural networks [J]. Journal of Medical Imaging (Bellingham , Wash.), 2017 , 4 ( 1 ): 014501 .

DROSTE R , CAI Y F , SHARMA H , et al . Ultrasound image representation learning by modeling sonographer visual attention [C]// Information Processing in Medical Imaging . Cham, GER : Springer , 2019 : 592 - 604 .

LITJENS G , KOOI T , BEJNORDI B E , et al . A survey on deep learning in medical image analysis [J]. Medical Image Analysis , 2017 , 42 : 60 - 88 .

田娟秀 , 刘国才 , 谷珊珊 , 等 . 医学图像分析深度学习方法研究与挑战 [J]. 自动化学报 , 2018 , 44 ( 3 ): 401 - 424 .

TIAN J X , LIU G C , GU S S , et al . Deep learning in medical image analysis and its challenges [J]. Acta Automatica Sinica , 2018 , 44 ( 3 ): 401 - 424 . (in Chinese)

WELLS W M III . Medical Image Analysis - past, present, and future [J]. Medical Image Analysis , 2016 , 33 : 4 - 6 .

SHEN D , WU G , SUK H I . Deep learning in medical image analysis [J]. Annual Review of Biomedical Engineering , 2017 , 19 : 221 - 248 .

BOOKSTEIN F L . Morphometric Tools for Landmark Data [M]. Cambridge, UK : Cambridge University Press , 1992 .

TOMPSON J , JAIN A , LECUN Y , et al . Joint training of a convolutional network and a graphical model for human pose estimation [C]// Proceedings of the 27th International Conference on Neural Information Processing Systems . New York, USA : ACM , 2014 : 1799 - 1807 .

WU Y , JI Q . Facial landmark detection: A literature survey [J]. International Journal of Computer Vision , 2019 , 127 ( 2 ): 115 - 142 .

KRIZHEVSKY A , SUTSKEVER I , HINTON G E . ImageNet classification with deep convolutional neural networks [J]. Communications of the ACM , 2017 , 60 ( 6 ): 84 - 90 .

DANG Q , YIN J Q , WANG B , et al . Deep learning based 2D human pose estimation: A survey [J]. Tsinghua Science and Technology , 2019 , 24 ( 6 ): 663 - 676 .

LI R , LIU Z Y , TAN J R . A survey on 3D hand pose estimation: Cameras, methods, and datasets [J]. Pattern Recognition , 2019 , 93 : 251 - 272 .

CHENG W H , SONG S J , CHEN C Y , et al . Fashion meets computer vision [J]. ACM Computing Surveys , 2021 , 54 ( 4 ): 1 - 41 .

TUYSUZOGLU A , TAN J , EISSA K , et al . Deep adversarial context-aware landmark detection for ultrasound imaging [C]// Medical Image Computing and Computer Assisted Intervention-MICCAI 2018 . Cham, GER : Springer , 2018 : 151 - 158 .

ANDERMATT S , PEZOLD S , AMANN M , et al . Multi-dimensional gated recurrent units for automated anatomical landmark localization [EB/OL].[2017] . https://arxiv.org/abs/1708.02766 https://arxiv.org/abs/1708.02766 .

PAYER C , ŠTERN D , BISCHOF H , et al . Regressing heatmaps for multiple landmark localization using CNNs [C]// Medical Image Computing and Computer-Assisted Intervention-MICCAI 2016 . Cham, GER : Springer , 2016 : 230 - 238 .

PAYER C , ŠTERN D , BISCHOF H , et al . Integrating spatial configuration into heatmap regression based CNNs for landmark localization [J]. Medical Image Analysis , 2019 , 54 : 207 - 219 .

ZHONG Z S , LI J , ZHANG Z X , et al . An attention-guided deep regression model for landmark detection in cephalograms [C]// Medical Image Computing and Computer Assisted Intervention-MICCAI 2019 . Cham, GER : Springer , 2019 : 540 - 548 .

ESLAMI M , NEUSCHAEFER-RUBE C , Serrurier A . Automatic vocal tract landmark localization from midsagittal MRI data [EB/OL]. [2020] . https://arxiv.org/abs/1907.07951 https://arxiv.org/abs/1907.07951 .

XU Z B , HUO Y K , PARK J , et al . Less is more: Simultaneous view classification and landmark detection for abdominal ultrasound images [C]// Medical Image Computing and Computer Assisted Intervention-MICCAI 2018 . Cham, GER : Springer , 2018 : 711 - 719 .

LI Y W , ALANSARY A , CERROLAZA J J , et al . Fast multiple landmark localisation using a patch-based iterative network [C]// Medical Image Computing and Computer Assisted Intervention-MICCAI 2018 . Cham, GER : Springer , 2018 : 563 - 571 .

GREENSPAN H , VAN GINNEKEN B , SUMMERS R M . Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique [J]. IEEE Transactions on Medical Imaging , 2016 , 35 ( 5 ): 1153 - 1159 .

SHORTEN C , KHOSHGOFTAAR T M . A survey on image data augmentation for deep learning [J]. Journal of Big Data , 2019 , 6 ( 1 ): 1 - 48 .

DROSTE R , CHATELAIN P , DRUKKER L , et al . Discovering salient anatomical landmarks by predicting human gaze [C]// 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI) . Iowa City, IA, USA : IEEE , 2020 : 1711 - 1714 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

图像压缩感知的特征域优化及自注意力增强神经网络重构算法

基于空间结构化推理深度融合网络的RGB-D场景解析

基于神经进化的深度学习模型研究综述

面向时序异常检测的可变视距多向扫描方法