电子学报 ›› 2022, Vol. 50 ›› Issue (3): 681-690.DOI: 10.12263/DZXB.20201338
张凌明1,2, 赵悦1,2, 李鹏程1,2, 刘洋3, 高陈强1,2
收稿日期:
2020-11-26
修回日期:
2021-06-02
出版日期:
2022-03-25
发布日期:
2022-03-25
作者简介:
基金资助:
ZHANG Ling-ming1,2, ZHAO Yue1,2, LI Peng-cheng1,2, LIU Yang3, GAO Chen-qiang1,2
Received:
2020-11-26
Revised:
2021-06-02
Online:
2022-03-25
Published:
2022-03-25
摘要:
从三维牙齿模型中准确分割出牙齿部分是正畸计算机辅助诊疗的基础.由于现有的三维模型分割网络对局部特征建模方式相对简单,这些方法无法有效提取牙齿边缘区域更细节的局部特征信息,进而导致这些区域出现牙齿多分、漏分等情况.本文提出一种基于局部注意力机制的三维牙齿模型分割网络以提高牙齿边缘区域的分割性能.首先,对原始牙齿模型中的三维网格数据进行多尺度的局部空间区域构建.其次,根据每个局部区域内的网格空间分布和网格特征差异进行注意力权重的学习.最后,基于学习到的网格权重进行局部特征聚合,以使得网络能自适应地去关注各个局部区域内更具有表达性网格特征.在临床数据集上的实验结果表明,相对于现有方法,本文网络的分割结果在牙齿边界区域更加准确光滑.
中图分类号:
张凌明, 赵悦, 李鹏程, 刘洋, 高陈强. 基于局部注意力机制的三维牙齿模型分割网络[J]. 电子学报, 2022, 50(3): 681-690.
ZHANG Ling-ming, ZHAO Yue, LI Peng-cheng, LIU Yang, GAO Chen-qiang. Dental Model Segmentation Network Based on Local Attention Mechanism[J]. Acta Electronica Sinica, 2022, 50(3): 681-690.
模型 | Accuracy |
---|---|
PointNet[ | 0.807 |
PointNet++[ | 0.892 |
PointCNN[ | 0.879 |
MeshSegNet[ | 0.925 |
本文方法 | 0.943 |
表1 本文网络与现有方法在3折交叉验证下的分割准确率对比 (均值±标准差)
模型 | Accuracy |
---|---|
PointNet[ | 0.807 |
PointNet++[ | 0.892 |
PointCNN[ | 0.879 |
MeshSegNet[ | 0.925 |
本文方法 | 0.943 |
模型 | T0 | T1 | T2 | T3 | T4 | T5 | T6 | T7 | mIoU |
---|---|---|---|---|---|---|---|---|---|
PointNet[ | 0.826 | 0.413 | 0.495 | 0.528 | 0.619 | 0.621 | 0.567 | 0.684 | 0.590 |
PointNet++[ | 0.863 | 0.719 | 0.741 | 0.734 | 0.787 | 0.757 | 0.705 | 0.776 | 0.760 |
PointCNN[ | 0.849 | 0.696 | 0.731 | 0.725 | 0.786 | 0.748 | 0.651 | 0.721 | 0.738 |
MeshSegNet[ | 0.911 | 0.811 | 0.821 | 0.814 | 0.838 | 0.840 | 0.811 | 0.847 | 0.837 |
本文方法 | 0.915 | 0.848 | 0.877 | 0.856 | 0.880 | 0.868 | 0.831 | 0.848 | 0.865 |
表2 本文网络与现有方法在3折交叉验证下的分割交并比对比
模型 | T0 | T1 | T2 | T3 | T4 | T5 | T6 | T7 | mIoU |
---|---|---|---|---|---|---|---|---|---|
PointNet[ | 0.826 | 0.413 | 0.495 | 0.528 | 0.619 | 0.621 | 0.567 | 0.684 | 0.590 |
PointNet++[ | 0.863 | 0.719 | 0.741 | 0.734 | 0.787 | 0.757 | 0.705 | 0.776 | 0.760 |
PointCNN[ | 0.849 | 0.696 | 0.731 | 0.725 | 0.786 | 0.748 | 0.651 | 0.721 | 0.738 |
MeshSegNet[ | 0.911 | 0.811 | 0.821 | 0.814 | 0.838 | 0.840 | 0.811 | 0.847 | 0.837 |
本文方法 | 0.915 | 0.848 | 0.877 | 0.856 | 0.880 | 0.868 | 0.831 | 0.848 | 0.865 |
输入组合 | Accuracy | mIoU |
---|---|---|
顶点坐标 | 0.887 | 0.752 |
顶点坐标+网格法向量 | 0.919 | 0.820 |
顶点坐标+顶点法向量 | 0.934 | 0.851 |
顶点坐标+顶点法向量+网格法向量 | 0.943 | 0.865 |
表3 本文网络在不同输入组合下的分割指标
输入组合 | Accuracy | mIoU |
---|---|---|
顶点坐标 | 0.887 | 0.752 |
顶点坐标+网格法向量 | 0.919 | 0.820 |
顶点坐标+顶点法向量 | 0.934 | 0.851 |
顶点坐标+顶点法向量+网格法向量 | 0.943 | 0.865 |
模型 | Accuracy | mIoU |
---|---|---|
仅空间信息增强 | 0.910 | 0.802 |
仅局部注意机制 | 0.926 | 0.835 |
完整网络结构 | 0.943 | 0.865 |
表4 本文网络使用不同模块的分割指标
模型 | Accuracy | mIoU |
---|---|---|
仅空间信息增强 | 0.910 | 0.802 |
仅局部注意机制 | 0.926 | 0.835 |
完整网络结构 | 0.943 | 0.865 |
分辨率 | Accuracy | mIoU |
---|---|---|
0.884 | 0.768 | |
0.916 | 0.815 | |
0.927 | 0.841 | |
0.943 | 0.865 |
表5 本文网络在不同网格分辨率下的分割指标
分辨率 | Accuracy | mIoU |
---|---|---|
0.884 | 0.768 | |
0.916 | 0.815 | |
0.927 | 0.841 | |
0.943 | 0.865 |
1 | XU Xiao-jie, LIU Chang, ZHENG You-yi. 3D tooth segmentation and labeling using deep convolutional neural networks[J]. IEEE Transactions on Visualization and Computer Graphics, 2018, 25(7): 2336‑2348. |
2 | TIAN Su-kun, DAI Ning, ZHANG Bei, et al. Automatic classification and segmentation of teeth on 3D dental model using hierarchical deep learning networks[J]. IEEE Access, 2019, 7(1): 84817‑84828. |
3 | ZANJANI F G, MOIN D A, VERHEIJ B, et al. Deep learning approach to semantic segmentation in 3d point cloud intra-oral scans of teeth[C]//International Conference on Medical Imaging with Deep Learning. London: PMLR, 2019: 557‑571. |
4 | LIAN C F, WANG L, WU T H, et al. MeshSNet: deep multi-scale mesh feature learning for end-to-end tooth labeling on 3D dental surfaces[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2019: 837‑845. |
5 | QI C R, SU Hao, MO Kai-chun, et al. Pointnet: deep learning on point sets for 3d classification and segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE Press, 2017: 652‑660. |
6 | LI Yang-yan, BU Rui, SUN Ming-chao, et al. Pointcnn: convolution on x-transformed points[C]//Advances in Neural Information Processing Systems. New York: ACM Press, 2018: 820‑830. |
7 | QI C R, YI Li, SU H, et al. Pointnet++: deep hierarchical feature learning on point sets in a metric space[C]//Advances in Neural Information Processing Systems. New York: ACM Press, 2017: 5099‑5108. |
8 | YUAN Tian-ran, LIAO Wen-he, DAI Ning, et al. Single-tooth modeling for 3D dental model[J]. International Journal of Biomedical Imaging, 2010, 2010(1): 1‑14. |
9 | ZHAO Ming-xi, MA Li-zhuang, TAN Wu-zheng, et al. Interactive tooth segmentation of dental models[C]//2005 IEEE Engineering in Medicine and Biology 27th Annual Conference. Shanghai: IEEE Press, 2006: 654‑657. |
10 | KUMAR Y, JANARDAN R, LARSON B, et al. Improved segmentation of teeth in dental models[J]. Computer-Aided Design and Applications, 2011, 8(2): 211‑224. |
11 | KRONFELD T, BRUNNER D, BRUNNETT G. Snake-based segmentation of teeth from virtual dental casts[J]. Computer-Aided Design and Applications, 2010, 7(2): 221‑233. |
12 | WU Kan, CHEN Li, LI Jing, et al. Tooth segmentation on dental meshes using morphologic skeleton[J]. Computers & Graphics, 2014, 38(1): 199‑211. |
13 | SINTHANAYOTHIN C, THARANONT W. Orthodontics treatment simulation by teeth segmentation and setup[C]//2008 5th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology. Los Alamitos: IEEE Press, 2008: 81‑84 |
14 | MA Ya-qi, LI Zhong-ke. Computer aided orthodontics treatment by virtual segmentation and adjustment[C]// 2010 International Conference on Image Analysis and Signal Processing. Xiameng: IEEE Press, 2010: 336‑339. |
15 | ZOU Bei-ji, LIU Shi-jian, LIAO Sheng-hui, et al. Interactive tooth partition of dental mesh base on tooth-target harmonic field[J]. Computers in Biology and Medicine, 2015, 56(1): 132‑144. |
16 | KONDO T, ONG S H, FOONG K W C. Tooth segmentation of dental study models using range images[J]. IEEE Transactions on Medical Imaging, 2004, 23(3): 350‑362. |
17 | WONGWAEN N, SINTHANYOTHIN C. Computerized algorithm for 3D teeth segmentation[C]//2010 International Conference on Electronics and Information Engineering. Kyoto: IEEE Press, 2010: 277‑280. |
18 | 罗会兰, 张云. 基于深度网络的图像语义分割综述[J]. 电子学报, 2019, 47(10): 2211‑2220. |
LUO Hui‑lan, ZHANG Yun. A survey of image semantic segmentation based on deep network[J]. Acta Electronica Sinica, 2019, 47(10): 2211‑2220. (in Chinese) | |
19 | 梁新宇, 林洗坤, 权冀川, 肖铠鸿. 基于深度学习的图像实例分割技术研究进展[J]. 电子学报, 2020, 48(12): 2476‑2486. |
LIANG Xin‑yu, LIN Xi‑kun, QUAN Ji‑chuan, XIAO Kai‑hong. Research on the progress of image instance segmentation based on deep learning[J]. Acta Electronica Sinica, 2020, 48(12): 2476‑2486. (in Chinese) | |
20 | WANG Peng-Shuai, LIU Yang, GUO Yu-xiao, et al. O-cnn: Octree-based convolutional neural networks for 3d shape analysis[J]. ACM Transactions on Graphics, 2017, 36(4): 1‑11. |
21 | LE T, BUI G, Duan Ye. A multi-view recurrent neural network for 3D mesh segmentation[J]. Computers & Graphics, 2017, 66(1): 103‑112. |
22 | KALOGERAKIS E, AVERKIOU M, MAJI S, et al. 3D shape segmentation with projective convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE Press 2017: 3779‑3788. |
23 | DAI A, NIESSNER M. 3dmv: Joint 3d-multi-view prediction for 3d semantic scene segmentation[C]// Proceedings of the European Conference on Computer Vision. Munich: IEEE Press, 2018: 452‑468. |
24 | YOU Hao-xuan, FENG Yi-fan, JI Rong-rong, et al. Pvnet: A joint convolutional network of point cloud and multi-view for 3d shape recognition[C]//Proceedings of the 26th ACM International Conference on Multimedia. New York: ACM Press, 2018: 1310‑1318. |
25 | KLOKOV R, LEMPITSKY V. Escape from cells: Deep kd-networks for the recognition of 3d point cloud models[C]//Proceedings of the IEEE International Conference on Computer Vision. Venice: IEEE Press, 2017: 863‑872. |
26 | RIEGLER G, OSMAN U A, GEIGER A. Octnet: Learning deep 3d representations at high resolutions[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE Press, 2017: 3577‑3586. |
27 | GRAHAM B, ENGELCKE M, MAATEN L VAN DER. 3d semantic segmentation with submanifold sparse convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE Press, 2018: 9224‑9232. |
28 | WANG Zong-ji, LU Feng. VoxSegNet: Volumetric CNNs for semantic part segmentation of 3D shapes[J]. IEEE Transactions on Visualization and Computer Graphics, 2019, 26(9): 2919‑2930. |
29 | MENG H Y, GAO Lin, LAI Yu-kun, et al. VV-Net: Voxel vae net with group convolutions for point cloud segmentation[C]//Proceedings of the IEEE International Conference on Computer Vision. Seoul: IEEE Press, 2019: 8500‑8508. |
30 | 彭秀平, 仝其胜, 林洪彬, 等. 一种面向散乱点云语义分割的深度残差-特征金字塔网络框架[J].自动化学报, 2021, 47(12): 2831‑2840. |
PENG Xiu-Ping, TONG Qi-Sheng, LIN Hong-Bin, et al. A deep residual — feature pyramid network framework for scattered point cloud semantic segmentation[J]. Acta Automatica Sinica, 2021, 47(12): 2831‑2840.(in Chinese) | |
31 | 盖杉, 王俊生. 基于深度学习的非局部注意力增强网络图像去雨算法研究[J]. 电子学报, 2020, 48(10): 1899‑ 1908. |
GAI Shan, WANG Jun‑sheng. Image Raindrop Algorithm Research Using Nonlocal Attention Enhanced Network Based on Deep Learning[J]. Acta Electronica Sinica, 2020, 48(10): 1899-1908. (in Chinese) | |
32 | 唐海桃, 薛嘉宾, 韩纪庆. 一种多尺度前向注意力模型的语音识别方法[J]. 电子学报, 2020, 48(07): 1255‑1260. |
TANG Hai-tao, XUE Jia-bin, HAN Ji-qing. A method of multi-scale forward attention model for speech recognition[J]. Acta Electronica.Sinica, 2020, 48(07): 1255‑1260.(in Chinese) | |
33 | 张志昌, 曾扬扬, 庞雅丽. 融合语义角色和自注意力机制的中文文本蕴含识别[J]. 电子学报, 2020, 48(11): 2162‑2169. |
ZHANG Zhi-chang, ZENG Yang-yang, PANG Ya-li. A chinese textual entailment recognition method incorporating se mantic role and self-attention[J]. Acta Electronica.Sinica, 2020, 48(11): 2162‑2169. (in Chinese) |
[1] | 王延达, 陈炜通, 皮德常, 岳琳. 一种自适应记忆神经网络多跳读取与覆盖度机制结合的药物推荐模型[J]. 电子学报, 2022, 50(4): 943-953. |
[2] | 伍邦谷, 张苏林, 石红, 朱鹏飞, 王旗龙, 胡清华. 基于多分支结构的不确定性局部通道注意力机制[J]. 电子学报, 2022, 50(2): 374-382. |
[3] | 王波, 黄冕, 刘利军, 黄青松, 单文琦. 基于多层聚焦Inception-V3卷积网络的细粒度图像分类[J]. 电子学报, 2022, 50(1): 72-78. |
[4] | 赵琰, 赵凌君, 匡纲要. 基于注意力机制特征融合网络的SAR图像飞机目标快速检测[J]. 电子学报, 2021, 49(9): 1665-1674. |
[5] | 周东明, 张灿龙, 李志欣, 王智文. 基于多层级视觉融合的图像描述模型[J]. 电子学报, 2021, 49(7): 1286-1290. |
[6] | 李志欣, 孙亚茹, 唐素勤, 张灿龙, 马慧芳. 双路注意力引导图卷积网络的关系抽取[J]. 电子学报, 2021, 49(2): 315-323. |
[7] | 江泽涛, 钱艺, 伍旭, 张少钦. 一种基于ARD‑GAN的低照度图像增强方法[J]. 电子学报, 2021, 49(11): 2160-2165. |
[8] | 石义乐, 杨文忠, 杜慧祥, 王丽花, 王婷, 理珊珊. 基于深度学习的图像描述综述[J]. 电子学报, 2021, 49(10): 2048-2060. |
[9] | 唐海桃, 薛嘉宾, 韩纪庆. 一种多尺度前向注意力模型的语音识别方法[J]. 电子学报, 2020, 48(7): 1255-1260. |
[10] | 张志昌, 曾扬扬, 庞雅丽. 融合语义角色和自注意力机制的中文文本蕴含识别[J]. 电子学报, 2020, 48(11): 2162-2169. |
[11] | 盖杉, 王俊生. 基于深度学习的非局部注意力增强网络图像去雨算法研究[J]. 电子学报, 2020, 48(10): 1899-1908. |
[12] | 尤洪峰, 田生伟, 禹龙, 吕亚龙. 基于Word Embedding的遥感影像检测分割[J]. 电子学报, 2020, 48(1): 75-83. |
[13] | 冯建周, 宋沙沙, 王元卓, 刘亚坤, 武红颖, 龚昊. 基于改进注意力机制的实体关系抽取方法[J]. 电子学报, 2019, 47(8): 1692-1700. |
[14] | 郎丛妍;须德;李兵. 一种基于模糊信息粒化的视频时空显著单元提取方法[J]. 电子学报, 2007, 35(10): 2023-2028. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||