Research and Challenges of Medical Image Landmark Detection Based on Deep Learning

LI Ju-peng; WANG Ying-hui; LI Gang

doi:10.12263/DZXB.20200725

您当前的位置：

首页 >

文章列表页 >

Research and Challenges of Medical Image Landmark Detection Based on Deep Learning

SURVEYS AND REVIEWS | 更新时间：2025-12-08

- Research and Challenges of Medical Image Landmark Detection Based on Deep Learning
- ACTA ELECTRONICA SINICA Vol. 50, Issue 1, Pages: 226-237(2022)
- 作者机构：
  
  1.北京交通大学电子信息工程学院，北京 100044
  2.北京大学口腔医学院，北京 100081
- 作者简介：
- 基金信息：
- DOI：10.12263/DZXB.20200725
  CLC： TP391.4;
- Received：16 July 2020，
  
  Revised：2020-12-17，
  
  Published：25 January 2022
- 稿件说明：
移动端阅览
李居朋,王颖慧,李刚.医学图像关键点检测深度学习方法研究与挑战[J].电子学报,2022,50(01):226-237.

LI Ju-peng,WANG Ying-hui,LI Gang.Research and Challenges of Medical Image Landmark Detection Based on Deep Learning[J].ACTA ELECTRONICA SINICA,2022,50(01):226-237.
李居朋,王颖慧,李刚.医学图像关键点检测深度学习方法研究与挑战[J].电子学报,2022,50(01):226-237. DOI： 10.12263/DZXB.20200725.

LI Ju-peng,WANG Ying-hui,LI Gang.Research and Challenges of Medical Image Landmark Detection Based on Deep Learning[J].ACTA ELECTRONICA SINICA,2022,50(01):226-237. DOI： 10.12263/DZXB.20200725.

摘要

作为众多医学图像处理的前提和关键，医学图像关键点检测具有重要的理论研究和应用价值.由于个体间差异性和个体内歧义性的影响，以及更高的临床应用定位精度的要求，医学解剖关键点检测面临着巨大的挑战.鉴于深度学习技术在医学图像关键点检测乃至整个医学图像处理领域都表现出了强大的实力，本文全面检索发表于顶级医学期刊和会议论文集中的医学图像关键点研究成果并进行了详细的梳理和综述.从计算机视觉任务角度简述医学图像关键点检测及其存在的难点；总结了深度学习技术在医学图像关键点检测中的基本框架，详细论述了医学图像关键点检测的分类问题和回归分析两种不同类型的解决思路；最后探讨了医学图像关键点检测深度学习方法面临的挑战、主要应对策略和开放的研究方向.

Abstract

As an entrance and challenge for many medical images processing

it is clinically essential to study on the medical image landmark detection and localization. Due to inter-individual variability and intra-individual ambiguity

as well as higher accuracy requirements of clinical application

the detection of medical anatomical landmarks was facing enormous challenges. In view of strength of deep learning of medical image landmark detection and the entire medical image processing field

we comprehensively retrieves relevant papers published in the top medical journals and conference proceedings to conduct a detailed review of these research findings. First of all

we briefly introduce difficulties in medical image landmark detection from the view of computer vision tasks. Secondly

we describe basic framework in medical image landmark detection

and discuss two different categories: classification and regression landmark detection solutions. Finally

we discuss the challenges and practicable strategies in deep learning for medical image landmark detection

as well as open research.

关键词

Keywords

references

HANAOKA S , SHIMIZU A , NEMOTO M , et al . Automatic detection of over 100 anatomical landmarks in medical CT images: A framework with independent detectors and combinatorial optimization [J]. Medical Image Analysis , 2017 , 35 : 192 - 214 .

GE J W , SAEIDI H , OPFERMANN J D , et al . Landmark-guided deformable image registration for supervised autonomous robotic tumor resection [C]// Medical Image Computing and Computer Assisted Intervention-MICCAI 2019 . Cham, GER : Springer , 2019 : 320 - 328 .

PETERLÍK I , COURTECUISSE H , ROHLING R , et al . Fast elastic registration of soft tissues under large deformations [J]. Medical Image Analysis , 2018 , 45 : 24 - 40 .

MADER A O , BERG J , FABRITZ A , et al . Localization and labeling of posterior ribs in chest radiographs using a CRF-regularized FCN with local refinement [C]// Medical Image Computing and Computer Assisted Intervention-MICCAI 2018 . Cham, GER : Springer , 2018 : 562 - 570 .

TOROSDAGLI N , LIBERTON D K , VERMA P , et al . Deep geodesic learning for segmentation and anatomical landmarking [J]. IEEE Transactions on Medical Imaging , 2019 , 38 ( 4 ): 919 - 931 .

ZHANG J , LIU M , WANG L , et al . Joint craniomaxillofacial bone segmentation and landmark digitization by context-guided fully convolutional networks [J]. Medical Image Computing and Computer-Assisted Intervention , 2017 , 10434 : 720 - 728 .

ZHANG J , LIU M X , WANG L , et al . Context-guided fully convolutional networks for joint craniomaxillofacial bone segmentation and landmark digitization [J]. Medical Image Analysis , 2020 , 60 : 101621 .

ZHANG J , SAHA A , ZHU Z , et al . Hierarchical convolutional neural networks for segmentation of breast tumors in MRI with application to radiogenomics [J]. IEEE Transactions on Medical Imaging , 2019 , 38 ( 2 ): 435 - 447 .

LIU C B , XIE H T , ZHANG S C , et al . Misshapen pelvis landmark detection by spatial local correlation mining for diagnosing developmental dysplasia of the hip [C]// Medical Image Computing and Computer Assisted Intervention-MICCAI 2019 . Cham, GER : Springer , 2019 : 441 - 449 .

ZHENG Y F , LIU D , GEORGESCU B , et al . 3D deep learning for efficient and robust landmark detection in volumetric data [C]// Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015 . Cham, GER : Springer , 2015 : 565 - 572 .

AUBERT B , VAZQUEZ C , CRESSON T , et al . Toward automated 3D spine reconstruction from biplanar radiographs using CNN for statistical spine model fitting [J]. IEEE Transactions on Medical Imaging , 2019 , 38 ( 12 ): 2796 - 2806 .

NOOTHOUT J M H , VOS B D D , WOLTERINK J M , et al . CNN-based landmark detection in cardiac CTA scans [EB/OL]. [2018] . https://arxiv.org/abs/1804.04963 https://arxiv.org/abs/1804.04963 .

GREWAL M , DEIST T M , WIERSMA J , et al . An end-to-end deep learning approach for landmark detection and matching in medical images [EB/OL]. [2020] . https://arxiv.org/abs/2001.07434 https://arxiv.org/abs/2001.07434 .

ZHANG J , LIU M X , SHEN D G . Detecting anatomical landmarks from limited medical imaging data using two-stage task-oriented deep neural networks [J]. IEEE Transactions on Image Processing , 2017 , 26 ( 10 ): 4753 - 4764 .

DUAN J M , BELLO G , SCHLEMPER J , et al . Automatic 3D Bi-ventricular segmentation of cardiac images by a shape-refined multi- task deep learning approach [J]. IEEE Transactions on Medical Imaging , 2019 , 38 ( 9 ): 2151 - 2164 .

TIULPIN A , MELEKHOW I , SAARAKKALA S . Knee anatomical landmark localization using hourglass networks [EB/OL]. [2019] . https://arxiv.org/abs/1907.12237 https://arxiv.org/abs/1907.12237 .

IBRAGIMOV B , LIKAR B , PEMUS F , et al . Automatic cephalometric x-ray landmark detection by applying game theory and random forests [C]// Proceedings of ISBI International Symposium on Biomedical Imaging . Cham, GER : Springer , 2014 : 1 - 8 .

LINDNER C , COOTES T F . Fully automatic cephalometric evaluation using random forest regression-voting [C]// Proceedings of ISBI International Symposium on Biomedical Imaging . Cham, GER : Springer , 2015 : 1 - 8 .

WANG C W , HHANG C T , HSIEH M C , et al . Evaluation and comparison of anatomical landmark detection methods for cephalometric X-ray images: A grand challenge [J]. IEEE Transactions on Medical Imaging , 2015 , 34 ( 9 ): 1890 - 1900 .

WANG C W , HUANG C T , LEE J H , et al . A benchmark for comparison of dental radiography analysis algorithms [J]. Medical Image Analysis , 2016 , 31 : 63 - 76 .

BIER B , GOLDMANN F , ZAECH J N , et al . Learning to detect anatomical landmarks of the pelvis in X-rays from arbitrary views [J]. International Journal of Computer Assisted Radiology and Surgery , 2019 , 14 ( 9 ): 1463 - 1473 .

YANG D , XIONG T , XU D G , et al . Automatic vertebra labeling in large-scale 3D CT using deep image-to-image network with message passing and sparsity regularization [C]// Information Processing in Medical Imaging . Cham, GER : Springer , 2017 : 633 - 644 .

PROBST T , MANINIS K K , CHHATKULI A , et al . Automatic tool landmark detection for stereo vision in robot-assisted retinal surgery [J]. IEEE Robotics and Automation Letters , 2018 , 3 ( 1 ): 612 - 619 .

TEIXEIRA B , TAMERSOY B , SINGH V , et al . Adaloss: Adaptive loss function for landmark localization [EB/OL]. [2019] . http://arxiv-export-lb.library.cornell.edu/pdf/1908.01070 http://arxiv-export-lb.library.cornell.edu/pdf/1908.01070 .

ZHANG D Q , WANG J N , NOBLE J H , et al . HeadLocNet: Deep convolutional neural networks for accurate classification and multi-landmark localization of head CTs [J]. Medical Image Analysis , 2020 , 61 : 101659 .

URSCHLER M , EBNER T , ŠTERN D . Integrating geometric configuration and appearance information into a unified framework for anatomical landmark localization [J]. Medical Image Analysis , 2018 , 43 : 23 - 36 .

CHEN R N , MA Y X , CHEN N L , et al . Cephalometric landmark detection by attentive feature pyramid fusion and regression-voting [C]// Medical Image Computing and Computer Assisted Intervention-MICCAI 2019 . Cham, GER : Springer , 2019 : 873 - 881 .

LÉVY-MANDEL A D , VENETSANOPOULOS A N , TSOTSOS J K . Knowledge-based landmarking of cephalograms [J]. Computers and Biomedical Research , 1986 , 19 ( 3 ): 282 - 309 .

EL-FEGHI I , SID-AHMED M A , AHMADI M . Automatic localization of craniofacial landmarks for assisted cephalometry [J]. Pattern Recognition , 2004 , 37 ( 3 ): 609 - 621 .

CARDILLO J , SID-AHMED M A . An image processing system for locating craniofacial landmarks [J]. IEEE Transactions on Medical Imaging , 1994 , 13 ( 2 ): 275 - 289 .

COOTES T F , TAYLOR C J , COOPER D H , et al . Active shape models-their training and application [J]. Computer Vision and Image Understanding , 1995 , 61 ( 1 ): 38 - 59 .

COOTES T F , EDWARDS G J , TAYLOR C J . Active appearance models [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2001 , 23 ( 6 ): 681 - 685 .

YUE W , YIN D , LI C , et al . Automated 2-D cephalometric analysis on X-ray images by a model-based approach [J]. IEEE Transactions on Biomedical Engineering , 2006 , 53 ( 8 ): 1615 - 1623 .

LINDNER C , WANG C W , HUANG C T , et al . Fully automatic system for accurate localisation and analysis of cephalometric landmarks in lateral cephalograms [J]. Scientific Reports , 2016 , 6 : 33581 .

ARlK S Ö , IBRAGIMOV B , XING L . Fully automated quantitative cephalometry using convolutional neural networks [J]. Journal of Medical Imaging (Bellingham , Wash.), 2017 , 4 ( 1 ): 014501 .

DROSTE R , CAI Y F , SHARMA H , et al . Ultrasound image representation learning by modeling sonographer visual attention [C]// Information Processing in Medical Imaging . Cham, GER : Springer , 2019 : 592 - 604 .

LITJENS G , KOOI T , BEJNORDI B E , et al . A survey on deep learning in medical image analysis [J]. Medical Image Analysis , 2017 , 42 : 60 - 88 .

田娟秀 , 刘国才 , 谷珊珊 , 等 . 医学图像分析深度学习方法研究与挑战 [J]. 自动化学报 , 2018 , 44 ( 3 ): 401 - 424 .

TIAN J X , LIU G C , GU S S , et al . Deep learning in medical image analysis and its challenges [J]. Acta Automatica Sinica , 2018 , 44 ( 3 ): 401 - 424 . (in Chinese)

WELLS W M III . Medical Image Analysis - past, present, and future [J]. Medical Image Analysis , 2016 , 33 : 4 - 6 .

SHEN D , WU G , SUK H I . Deep learning in medical image analysis [J]. Annual Review of Biomedical Engineering , 2017 , 19 : 221 - 248 .

BOOKSTEIN F L . Morphometric Tools for Landmark Data [M]. Cambridge, UK : Cambridge University Press , 1992 .

TOMPSON J , JAIN A , LECUN Y , et al . Joint training of a convolutional network and a graphical model for human pose estimation [C]// Proceedings of the 27th International Conference on Neural Information Processing Systems . New York, USA : ACM , 2014 : 1799 - 1807 .

WU Y , JI Q . Facial landmark detection: A literature survey [J]. International Journal of Computer Vision , 2019 , 127 ( 2 ): 115 - 142 .

KRIZHEVSKY A , SUTSKEVER I , HINTON G E . ImageNet classification with deep convolutional neural networks [J]. Communications of the ACM , 2017 , 60 ( 6 ): 84 - 90 .

DANG Q , YIN J Q , WANG B , et al . Deep learning based 2D human pose estimation: A survey [J]. Tsinghua Science and Technology , 2019 , 24 ( 6 ): 663 - 676 .

LI R , LIU Z Y , TAN J R . A survey on 3D hand pose estimation: Cameras, methods, and datasets [J]. Pattern Recognition , 2019 , 93 : 251 - 272 .

CHENG W H , SONG S J , CHEN C Y , et al . Fashion meets computer vision [J]. ACM Computing Surveys , 2021 , 54 ( 4 ): 1 - 41 .

TUYSUZOGLU A , TAN J , EISSA K , et al . Deep adversarial context-aware landmark detection for ultrasound imaging [C]// Medical Image Computing and Computer Assisted Intervention-MICCAI 2018 . Cham, GER : Springer , 2018 : 151 - 158 .

ANDERMATT S , PEZOLD S , AMANN M , et al . Multi-dimensional gated recurrent units for automated anatomical landmark localization [EB/OL].[2017] . https://arxiv.org/abs/1708.02766 https://arxiv.org/abs/1708.02766 .

PAYER C , ŠTERN D , BISCHOF H , et al . Regressing heatmaps for multiple landmark localization using CNNs [C]// Medical Image Computing and Computer-Assisted Intervention-MICCAI 2016 . Cham, GER : Springer , 2016 : 230 - 238 .

PAYER C , ŠTERN D , BISCHOF H , et al . Integrating spatial configuration into heatmap regression based CNNs for landmark localization [J]. Medical Image Analysis , 2019 , 54 : 207 - 219 .

ZHONG Z S , LI J , ZHANG Z X , et al . An attention-guided deep regression model for landmark detection in cephalograms [C]// Medical Image Computing and Computer Assisted Intervention-MICCAI 2019 . Cham, GER : Springer , 2019 : 540 - 548 .

ESLAMI M , NEUSCHAEFER-RUBE C , Serrurier A . Automatic vocal tract landmark localization from midsagittal MRI data [EB/OL]. [2020] . https://arxiv.org/abs/1907.07951 https://arxiv.org/abs/1907.07951 .

XU Z B , HUO Y K , PARK J , et al . Less is more: Simultaneous view classification and landmark detection for abdominal ultrasound images [C]// Medical Image Computing and Computer Assisted Intervention-MICCAI 2018 . Cham, GER : Springer , 2018 : 711 - 719 .

LI Y W , ALANSARY A , CERROLAZA J J , et al . Fast multiple landmark localisation using a patch-based iterative network [C]// Medical Image Computing and Computer Assisted Intervention-MICCAI 2018 . Cham, GER : Springer , 2018 : 563 - 571 .

GREENSPAN H , VAN GINNEKEN B , SUMMERS R M . Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique [J]. IEEE Transactions on Medical Imaging , 2016 , 35 ( 5 ): 1153 - 1159 .

SHORTEN C , KHOSHGOFTAAR T M . A survey on image data augmentation for deep learning [J]. Journal of Big Data , 2019 , 6 ( 1 ): 1 - 48 .

DROSTE R , CHATELAIN P , DRUKKER L , et al . Discovering salient anatomical landmarks by predicting human gaze [C]// 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI) . Iowa City, IA, USA : IEEE , 2020 : 1711 - 1714 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Feature-Space Optimization-Inspired and Self-Attention Enhanced Neural Network Reconstruction Algorithm for Image Compressive Sensing

RGB-D Scene Parsing Based on Spatial Structured Inference Deep Fusion Networks

A Review of Deep Learning Models Based on Neuroevolution

Variable Horizon Multi-Directional Scanning Method for Time Series Anomaly Detection

Related Author

CHEN Wen-jun

YANG Chun-ling

WANG Ze-yu

WU Yan-xia

ZHANG Guo-yin

BU Shu-hui

HAN Chong

WANG Jun-li

Related Institution

School of Electronic and Information Engineering, South China University of Technology

College of Computer Science and Technology, Harbin Engineering University

School of Aeronautics, Northwestern Polytechnical University

College of Computer Science and Technology Harbin Engineering University Harbin Heilongjiang China

School of Aeronautics Northwestern Polytechnical University Xi'an Shaanxi China

⁰