基于细粒度检索增强的医学报告生成

牛庆港; 张晓伟; 吕千一; 李真; 闫中敏; 王峻; 郭茂祖; 余国先

doi:10.12263/DZXB.20260621

您当前的位置：

首页 >

文章列表页 >

基于细粒度检索增强的医学报告生成

更新时间：2026-10-13

- 基于细粒度检索增强的医学报告生成
  增强出版
- Medical Report Generation With Fine-grained Retrieval-based Refinement
- 电子学报 2026年页码：1-14
- 作者机构：
  
  1.山东大学软件学院，山东济南 250101
  2.山东大学人工智能国际联合研究院，山东济南 250101
  3.山东大学齐鲁医院，山东济南 250012
  4.北京建筑大学电气与信息工程学院，北京 100044
- 作者简介：
  
  牛庆港男，1997年生，山东临沂人。现为山东大学硕士研究生。主要研究方向为机器学习和计算病理学。E-mail: qgniu@mail.sdu.edu.cn
  张晓伟男，1997年生，山东临沂人。现为山东大学博士研究生。主要研究方向为AI for Science。Email: zhangxiaowei@mail.sdu.edu.cn
  吕千一男，2002年生，山东临沂人。现为山东大学硕士研究生。主要研究领域为计算病理学、机器学习。Email: qylv@mail.sdu.edu.cn
  李真女，1984年生，山东济南人。现为山东大学齐鲁医院主任医师。主要研究方向为胰腺疾病的超声内镜介入诊疗、人工智能在消化内镜质量控制中的应用。E-mail: qilulizhen@sdu.edu.cn
  闫中敏女，1977年生，山东济宁人。现为山东大学副教授。主要研究方向为Web数据集成与智能数据分析。E-mail: yzm@sdu.edu.cn
  王峻女，1983年生，重庆人。现为山东大学人工智能国际联合研究院教授。主要研究方向为数据挖掘和生物信息学。E-mail: kingjun@sdu.edu.cn
  郭茂祖男，1966年生，山东德州人。现为北京建筑大学电气与信息工程学院教授。主要研究方向为机器学习、数据挖掘和生物信息学。E-mail: guomaozu@bucea.edu.cn
  余国先男，1985年生，湖北孝感人。现为山东大学软件学院教授。主要研究方向为机器学习、数据挖掘和生物信息学。E-mail: gxyu@sdu.edu.cn
- 基金信息：
  
  国家自然科学基金重点项目(62531013);国家自然科学基金项目(62272276)
- DOI：10.12263/DZXB.20260621
  中图分类号： TP391.4;R445
- 收稿：2026-07-24，
  
  录用：2026-08-30，
  
  网络首发：2026-10-13，
- 稿件说明：
移动端阅览
牛庆港, 张晓伟, 吕千一, 等. 基于细粒度检索增强的医学报告生成[J/OL]. 电子学报, 2026,1-14.

NIU Qinggang, ZHANG Xiaowei, LV Qianyi, et al. Medical Report Generation With Fine-grained Retrieval-based Refinement[J/OL]. ACTA ELECTRONICA SINICA, 2026, 1-14.
牛庆港, 张晓伟, 吕千一, 等. 基于细粒度检索增强的医学报告生成[J/OL]. 电子学报, 2026,1-14. DOI： 10.12263/DZXB.20260621.

NIU Qinggang, ZHANG Xiaowei, LV Qianyi, et al. Medical Report Generation With Fine-grained Retrieval-based Refinement[J/OL]. ACTA ELECTRONICA SINICA, 2026, 1-14. DOI： 10.12263/DZXB.20260621.

摘要

医学诊断报告的自动生成能减轻医生工作负担、提高报告生成效率，并降低主观偏差。现有检索增强方法将相似病例报告作为参考文本引入生成过程，为模型提供诊断语义提示，来提升生成报告的质量，但多数方法仍停留在粗粒度文本检索层面，易出现“全局相似而局部病灶不一致”的问题，导致参考报告中的无关病灶描述干扰生成结果。此外，不同医生的报告书写风格差异显著，直接引入多个检索文本会因表述风格不一致带来噪声干扰，影响生成结果的准确性。为解决上述问题，本文提出一种基于细粒度检索增强的医学报告生成框架（Fine-grained Retrieval-enhanced Medical Report Generation，FRMRG）。首先，设计解剖感知细粒度校正模块，对检索得到的粗粒度文本进行细粒度校正，保留与当前影像相关的有效信息，削弱不一致病灶描述的干扰；其次，构建结构化提示驱动的初始报告生成模块，将多个相关报告映射到统一语义空间，并通过固定提示模板组织相关诊断报告，削弱报告书写风格差异的影响，生成初始报告；最后，设计多模态最终报告生成模块，将初始报告作为语义先验，结合视觉特征引导大语言模型生成连贯且符合临床事实的医学报告。通过上述设计，FRMRG能够充分挖掘检索报告中的有效诊断信息，同时降低无关病灶描述及书写风格差异带来的干扰，并结合视觉信息对报告生成过程进行约束，从而提升生成内容与医学影像之间的语义一致性。实验结果显示，FRMRG在印第安纳大学胸部X线数据集（Indiana University Chest X-Ray Collection，IU X-Ray）、重症监护医学信息数据库胸部X线数据集（Medical Information Mart for Intensive Care Chest X-Ray，MIMIC-CXR）和私有临床胃镜数据集上的BLEU-1分别较次优方法提升2.8%、2.1%和1.7%，METEOR分别提升1.4%、3.0%和3.2%，表明FRMRG能够提升医学报告生成质量，并具有较好的泛化能力。

Abstract

Automatic generation of medical diagnostic reports can reduce physicians’ workload

improve reporting efficiency

and mitigate subjective variations in report writing. Existing retrieval-enhanced methods usually introduce reports from similar cases as reference texts into the generation process to provide diagnostic semantic cues and improve report quality. However

most existing methods remain at the level of coarse-grained text retrieval

which can lead to the problem of “global similarity but local lesion inconsistency”

causing irrelevant lesion descriptions in retrieved reports to interfere with the generated results. In addition

substantial differences in terminology and writing styles among physicians may introduce additional noise when multiple retrieved reports are directly incorporated

thereby affecting the accuracy of report generation. To address these issues

this paper proposes a fine-grained retrieval-enhanced medical report generation framework

termed FRMRG(Fine-grained Retrieval-enhanced Medical Report Generation). First

an anatomy-aware fine-grained correction module is designed to refine the coarse-grained retrieved texts

preserve information relevant to the current medical image

and suppress interference from inconsistent lesion descriptions. Second

a structured prompt-driven initial report generation module is constructed to map multiple relevant reports into a unified semantic space and organize diagnostic information using fixed prompt templates

thereby reducing the influence of differences in report-writing styles and generating an initial report. Finally

a multimodal final report generation module is developed

in which the initial report serves as a semantic prior and is integrated with visual features to guide a large language model in generating coherent medical reports consistent with clinical findings. Through the above design

FRMRG can effectively exploit diagnostic information contained in retrieved reports

reduce interference from irrelevant lesion descriptions and writing-style variations

and further constrain the report generation process with visual information

thereby improving the semantic consistency between generated content and medical images. Experimental results demonstrate that FRMRG consistently outperforms the second-best methods on the IU-X-Ray(Indiana University Chest X-Ray Collection)

MIMIC-CXR(Medical Information Mart for Intensive Care Chest X-Ray)

and a private clinical gastroscopy dataset

with BLEU-1 improvements of 2.8%

2.1%

and 1.7%

respectively

and METEOR improvements of 1.4%

3.0%

and 3.2%

respectively. These results indicate that FRMRG can effectively improve the quality of medical report generation and exhibits good generalization capability.

关键词

Keywords

references

Chen M , Gholamrezanezhad A . Burnout in radiology [J ] . Academic Radiology , 2023 , 30 ( 6 ): 1031 - 1032 . DOI: 10.1016/j.acra.2023.03.025 http://dx.doi.org/10.1016/j.acra.2023.03.025

Chen Zhihong , Song Yan , Chang T H , et al . Generating radiology reports via memory-driven transformer [C ] // Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing . ACL , 2020 : 1439 - 1449 . DOI: 10.18653/v1/2020.emnlp-main.112 http://dx.doi.org/10.18653/v1/2020.emnlp-main.112

Liu Fenglin , Wu Xian , Ge Shen , et al . Exploring and distilling posterior and prior knowledge for radiology report generation [C ] // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2021 : 13748 - 13757 . DOI: 10.1109/cvpr46437.2021.01354 http://dx.doi.org/10.1109/cvpr46437.2021.01354

Jin Haibo , Che Haoxuan , Lin Yi , et al . PromptMRG: Diagnosis-driven prompts for medical report generation [C ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2024 , 38 ( 3 ): 2607 - 2615 . DOI: 10.1609/aaai.v38i3.28038 http://dx.doi.org/10.1609/aaai.v38i3.28038

Li Mingjie , Lin Bingqian , Chen Zicong , et al . Dynamic graph enhanced contrastive learning for chest X-ray report generation [C ] // 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2023 : 3334 - 3343 . DOI: 10.1109/cvpr52729.2023.00325 http://dx.doi.org/10.1109/cvpr52729.2023.00325

Wu Ruiqi , Zhang Chenran , Zhang Jianle , et al . MM-retinal: Knowledge-enhanced foundational pretraining with fundus image-text expertise [C ] // Medical Image Computing and Computer Assisted Intervention - MICCAI 2024 . ChamSpringer Nature Switzerland2024 : 722 - 732 . DOI: 10.1007/978-3-031-72378-0_67 http://dx.doi.org/10.1007/978-3-031-72378-0_67

Liu Fenglin , You Chenyu , Wu Xian , et al . Auto-encoding knowledge graph for unsupervised medical report generation [C ] // Proceedings of NeurIPS . 2021 : 16266 - 16279 . https://proceedings.neurips.cc/paper_files/paper/2021/hash/876e1c59023b1a0e95808168e1a8ff89-Abstract.html https://proceedings.neurips.cc/paper_files/paper/2021/hash/876e1c59023b1a0e95808168e1a8ff89-Abstract.html .

Tanida T , Müller P , Kaissis G , et al . Interactive and explainable region-guided radiology report generation [C ] // 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2023 : 7433 - 7442 . DOI: 10.1109/cvpr52729.2023.00718 http://dx.doi.org/10.1109/cvpr52729.2023.00718

Wang Siyuan , Peng Bo , Liu Yichao , et al . Fine-grained medical vision-language representation learning for radiology report generation [C ] // Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing . Stroudsburg : ACL , 2023 : 15949 - 15956 . DOI: 10.18653/v1/2023.emnlp-main.989 http://dx.doi.org/10.18653/v1/2023.emnlp-main.989

樊琳 , 龚勋 , 郑岑洋 . 基于文本引导下的多模态医学图像分析算法 [J ] . 电子学报 , 2024 , 52 ( 7 ): 2341 - 2355 .

Fan Lin , Gong Xun , Zheng Cenyang . A multi-modal medical image analysis algorithm based on text guidance [J ] . Acta Electronica Sinica , 2024 , 52 ( 7 ): 2341 - 2355 . (in Chinese)

才华 , 易亚希 , 付强 , 等 . 基于跨模态引导和对齐的多模态预训练方法 [J ] . 电子学报 , 2024 , 52 ( 10 ): 3368 - 3381 .

Cai Hua , Yi Yaxi , Fu Qiang ,, et al . Multimodal pretraining with cross-modal guidance and alignment [J ] . Acta Electronica Sinica , 2024 , 52 ( 10 ): 3368 - 3381 . (in Chinese)

Chen Zhihong , Shen Yaling , Song Yan , et al . Cross-modal memory networks for radiology report generation [C/OL ] // Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics . 2021 : 5904 - 5914 [2026-04-30] . https://aclanthology.org/2021.acl-long.459/ https://aclanthology.org/2021.acl-long.459/ . DOI: 10.18653/v1/2021.acl-long.459 http://dx.doi.org/10.18653/v1/2021.acl-long.459

Yang Shuxin , Wu Xian , Ge Shen , et al . Radiology report generation with a learned knowledge base and multi-modal alignment [J ] . Medical Image Analysis , 2023 , 86 : 102798 . DOI: 10.1016/j.media.2023.102798 http://dx.doi.org/10.1016/j.media.2023.102798

Shentu J , Al Moubayed N . CXR-IRGen: An integrated vision and language model for the generation of clinically accurate chest X-ray image-report pairs [C ] // 2024 IEEE/CVF Winter Conference on Applications of Computer Vision . Piscataway : IEEE , 2024 : 5200 - 5209 . DOI: 10.1109/wacv57701.2024.00513 http://dx.doi.org/10.1109/wacv57701.2024.00513

周涛 , 叶鑫宇 , 刘凤珍 , 等 . 利用跨模态轻量级YOLOv5模型的PET/CT肺部肿瘤检测 [J ] . 电子与信息学报 , 2024 , 46 ( 2 ): 624 - 632 .

Zhou Tao , Ye Xinyu , Liu Fengzhen , et al . CL-YOLOv5: PET/CT lung cancer detection with cross-modal light-weight YOLOv5 model [J ] . Journal of Electronics & Information Technology , 2024 , 46 ( 2 ): 624 - 632 . (in Chinese)

Jing Baoyu , Xie Pengtao , Xing E . On the automatic generation of medical imaging reports [C ] // Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics . Stroudsburg : ACL , 2018 : 2577 - 2586 . DOI: 10.18653/v1/p18-1240 http://dx.doi.org/10.18653/v1/p18-1240

You Jingyi , Li Dongyuan , Okumura M , et al . JPG-Jointly learn to align: automated disease prediction and radiology report generation [C ] // Proceedings of ACL . 2022 : 5989 - 6001 . https://aclanthology.org/2022.coling-1.523/ https://aclanthology.org/2022.coling-1.523/ .

Chen Wenting , Shen Linlin , Lin Jingyang , et al . Fine-grained image-text alignment in medical imaging enables explainable cyclic image-report generation [C ] // Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics . Stroudsburg : ACL , 2024 : 9494 - 9509 . DOI: 10.18653/v1/2024.acl-long.514 http://dx.doi.org/10.18653/v1/2024.acl-long.514

Yin Heng , Wu Wei , Hao Yongtao . DKA-RG: Disease-knowledge-enhanced fine-grained image-text alignment for automatic radiology report generation [J ] . Electronics , 2024 , 13 ( 16 ): 3306 . DOI: 10.3390/electronics13163306 http://dx.doi.org/10.3390/electronics13163306

Singhal K , Tu Tao , Gottweis J , et al . Toward expert-level medical question answering with large language models [J ] . Nature Medicine , 2025 , 31 ( 3 ): 943 - 950 . DOI: 10.1038/s41591-024-03423-7 http://dx.doi.org/10.1038/s41591-024-03423-7

Luo Renqian , Sun Liai , Xia Yingce , et al . BioGPT: Generative pre-trained transformer for biomedical text generation and mining [J ] . Briefings in Bioinformatics , 2022 , 23 ( 6 ): bbac409 . DOI: 10.1093/bib/bbac409 http://dx.doi.org/10.1093/bib/bbac409

刘青海 , 吴钱林 , 罗佳 , 等 . 面向大语言模型的医学图像语义挖掘跨模态哈希 [J ] . 电子与信息学报 , 2026 , 48 ( 3 ): 1167 - 1179 .

Liu Qinghai , Wu Qianlin , Luo Jia , et al . Cross modal hashing of medical image semantic mining for large language model [J ] . Journal of Electronics & Information Technology , 2026 , 48 ( 3 ): 1167 - 1179 . (in Chinese)

Wang Zhanyu , Liu Lingqiao , Wang Lei , et al . R2GenGPT: Radiology report generation with frozen LLMs [J ] . Meta-Radiology , 2023 , 1 ( 3 ): 100033 . DOI: 10.1016/j.metrad.2023.100033 http://dx.doi.org/10.1016/j.metrad.2023.100033

Liu Chang , Tian Yuanhe , Chen Weidong , et al . Bootstrapping large language models for radiology report generation [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2024 , 38 ( 17 ): 18635 - 18643 . DOI: 10.1609/aaai.v38i17.29826 http://dx.doi.org/10.1609/aaai.v38i17.29826

Liu Ze , Lin Yutong , Cao Yue , et al . Swin transformer: Hierarchical vision transformer using shifted windows [C ] // 2021 IEEE/CVF International Conference on Computer Vision . Piscataway : IEEE , 2021 : 9992 - 10002 . DOI: 10.48550/arXiv.2103.14030 http://dx.doi.org/10.48550/arXiv.2103.14030

Seibold C , Jaus A , Fink M A , et al . Accurate fine-grained segmentation of human anatomy in radiographs via volumetric pseudo-labeling [EB/OL ] . 2023[ 2026-04-30 ] . https://arxiv.org/abs/2306.03934 https://arxiv.org/abs/2306.03934 . DOI: 10.21203/rs.3.rs-3687730/v1 http://dx.doi.org/10.21203/rs.3.rs-3687730/v1

He Kaiming , Zhang Xiangyu , Ren Shaoqing , et al . Deep residual learning for image recognition [C ] // 2016 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2016 : 770 - 778 . DOI: 10.1109/cvpr.2016.90 http://dx.doi.org/10.1109/cvpr.2016.90

Papineni K , Roukos S , Ward T , et al . Bleu: A method for automatic evaluation of machine translation [C ] // Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics . Morristown : ACL , 2002 : 311 - 318 . DOI: 10.3115/1073083.1073135 http://dx.doi.org/10.3115/1073083.1073135

Lin C Y . ROUGE: A package for automatic evaluation of summaries [C ] // Text Summarization Branches Out . Kerrville : Association for Computational Linguistics 2004 : 74 - 81 .

Banerjee S , Lavie A . METEOR: An automatic metric for MT evaluation with improved correlation with human judgments [C ] // Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization . Kerrville : Association for Computational Linguistics 2005 : 65 - 72 .

Lee J , Yoon W , Kim S , et al . BioBERT: A pre-trained biomedical language representation model for biomedical text mining [J ] . Bioinformatics , 2020 , 36 ( 4 ): 1234 - 1240 . DOI: 10.1093/bioinformatics/btz682 http://dx.doi.org/10.1093/bioinformatics/btz682

Wang Zhuhao , Sun Yihua , Li Zihan , et al . LLM-RG4: Flexible and factual radiology report generation across diverse input contexts [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2025 , 39 ( 8 ): 8250 - 8258 . DOI: 10.1609/aaai.v39i8.32890 http://dx.doi.org/10.1609/aaai.v39i8.32890

Nicolson A , Dowling J , Koopman B . Improving chest X-ray report generation by leveraging warm starting [J ] . Artificial Intelligence in Medicine , 2023 , 144 : 102633 . DOI: 10.1016/j.artmed.2023.102633 http://dx.doi.org/10.1016/j.artmed.2023.102633

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

MDW：最小双权差分特征搜索模型

面向扇出型晶圆级封装的芯片间RDL布线算法

知识增强的区域关联图像解说生成方法

低轨卫星网络自适应随机接入研究

基于多维DHS的AFDM卫星通信隐蔽波形