ED-NAS: 基于神经网络架构搜索的陶瓷晶粒SEM图像分割方法

doi:10.12263/DZXB.20201416

电子学报 ›› 2022, Vol. 50 ›› Issue (2): 461-469.DOI: 10.12263/DZXB.20201416

ED-NAS: 基于神经网络架构搜索的陶瓷晶粒SEM图像分割方法

蔡超丽¹, 李纯纯², 黄琳¹, 杨铁军¹

^1.桂林理工大学广西嵌入式技术与智能系统重点实验室, 广西桂林 541006
^2.桂林理工大学材料科学与工程学院, 广西桂林 541006

收稿日期:2020-12-11 修回日期:2021-10-19 出版日期:2022-02-25
- 通讯作者:
- 杨铁军
- 作者简介:
- 蔡超丽　女，1996年11月出生，湖北十堰人.现为桂林理工大学信息学院硕士研究生，主要研究方向为计算机视觉、深度学习.E‑mail: kiko.c@qq.com
  杨铁军（通讯作者）　男，1979年6月出生，湖南宁乡人.桂林理工大学信息学院教授，硕士生导师，研究方向为计算机视觉、深度学习.E‑mail: yattie@foxmail.com
- 基金资助:
- 国家自然科学基金 (61941202); 广西自然科学基金 (2018GXNSFBA281081); 广西嵌入式技术与智能系统重点实验室开放基金 (2020-2-2)

ED-NAS: Ceramic Grain Segmentation Based on Neural Architecture Search Using SEM Images

CAI Chao-li¹, LI Chun-chun², HUANG Lin¹, YANG Tie-jun¹

^1.Guangxi Key Laboratory of Embedded Technology and Intelligent System, Guilin University of Technology, Guilin, Guangxi 541006, China
^2.School of Materials Science and Engineering, Guilin University of Technology, Guilin, Guangxi 541006, China

Received:2020-12-11 Revised:2021-10-19 Online:2022-02-25 Published:2022-02-25
- Corresponding author:
- YANG Tie-jun
- Supported by:
- National Natural Science Foundation of China (61941202); Natural Science Foundation of Guangxi Zhuang Autonomous Region, China (2018GXNSFBA281081); Open Fund of Guangxi Key Laboratory of Embedded Technology and Intelligent Systems (2020-2-2)

摘要/Abstract

摘要：

为了提高深度卷积神经网络（Convolutional Neural Network，CNN）设计的自动化程度并进一步提高陶瓷晶粒扫描电子显微镜（Scanning Electron Microscope，SEM）图像分割的准确性，提出了一种基于神经网络架构搜索的陶瓷晶粒图像分割方法.该方法设计多分支结构编码空间和链式结构解码空间，并构造多分支结构编码Cell和链式结构解码Cell；同时基于强化学习分别搜索最佳编码Cell和解码Cell；此外，基于编码-解码神经网络架构堆叠最佳Cell构建陶瓷晶粒图像分割CNN，并采用池化索引在解码阶段恢复丢失的细节信息.实验在包含了629张的陶瓷晶粒SEM图像数据集上进行，搜索最佳Cell耗时约148 GPU-时.与U-Net、SegNet等SOTA方法相比，该方法在陶瓷晶粒测试集上获得了更高的分割准确性（mIoU $≈$ 68.9%）.

关键词: 神经网络架构搜索, 编码-解码神经网络架构, 陶瓷晶粒, 图像分割, 编码Cell, 解码Cell

Abstract:

In order to improve the automation degree of deep convolutional neural network(CNN) design and further improve the accuracy of ceramic grainsegmentation using scanning electron microscope(SEM) images,a ceramic grain segmentation method is proposed based on neural architecture search. This method designs searching spaces including the ones of multi-branch structure for encoding and chain structure for decoding, where encoding cells(E-cell) and decoding cells(D-cell) are constructed. The best E-cell and D-cell are found using reinforcement learning. Moreover, an encoding-decoding neural architecture-based CNN is built for ceramic grain segmentation by stacking the best cells, and the pooling indices are adopted to recover the lost details in the decoding stage. The experiment was carried out on a dataset of 629 SEM images of ceramic grain, and the searching process took about 148 GPU-hours. Compared with SOTA methods such as U-Net and SegNet, the proposed method obtained higher segmentation accuracy(mIoU≈68.9%) on a ceramic grains test dataset.

Key words: neural architecture search, encoding-decoding neural architecture, ceramic grains, image segmentation, encoding cell, decoding cell

中图分类号:

TP389.1

蔡超丽, 李纯纯, 黄琳, 等. ED-NAS: 基于神经网络架构搜索的陶瓷晶粒SEM图像分割方法[J]. 电子学报, 2022, 50(2): 461-469.

Chao-li CAI, Chun-chun LI, Lin HUANG, et al. ED-NAS: Ceramic Grain Segmentation Based on Neural Architecture Search Using SEM Images[J]. Acta Electronica Sinica, 2022, 50(2): 461-469.

图/表 14

图1 陶瓷晶粒SEM图像

图2 ED-NAS方法总体流程图

图3 Cell空间的表示

表1 候选操作集合

操作	操作码
3×3 dw conv	0
5×5 dw conv	1
3×3 conv	2
3×3 dl conv，dilation rate=2	3
3×3 max pooling	4

图4 堆叠Cell构建基于EDNA的神经网络(编码器采用多个E-cell堆叠构成,解码器采用相同数量的D-cell堆叠构成,每个E-cell紧跟一个Max-pooling层，并且将Pooling indices复制给解码器对应的D-cell供其使用)

图5 原始图像（左侧）和裁剪后的图像（右侧）

图6 可视化后的标注结果示例(a) 图像样本　(b) 可视化后的GT

表2 数据集数量分布及标签

类别	训练集	验证集	测试集	总数	标签
板条状	422	60	119	601	1
圆柱状	435	63	125	623	2

表3 候选网络最佳验证准确性比较

候选网络	分割精度（mIoU）/%
N=3，L=4，链式结构D-cell	54.9
N=4，L=3，链式结构D-cell	56.6
N=3，L=4，多分支结构D-cell	51.3
N=4，L=3，多分支结构D-cell	53.5

图7 D-cell采用链式结构，不同N值时的最佳Cell结构

图8 D-cell采用多分支结构，不同N值时的最佳Cell结构

图9 不同N值下堆叠L个最佳Cell的EDNA神经网络的准确性比较

表4 不同方法在陶瓷晶粒数据集上的分割准确性

方法	分割精度（mIoU）/%
ERFNet^［34］	40.4
U-Net^［11］	47.7
ENet^［35］	48.8
FCN-16^［12］	58.1
DeepLabv3^［36］	59.1
SegNet^［29］	68.0
ED-NAS	68.9

图10 几种方法在陶瓷晶粒数据集上的分割结果示例

参考文献 36

1	张伟儒, 李伶, 王坤. 先进陶瓷材料研究现状及发展趋势[J]. 新材料产业, 2016, (1): 2-8.
	ZHANGWeiru, LILing, WANGKun. Research status and development trend of advanced ceramic materials[J]. New Materials Industry, 2016, (1): 2-8. (in Chinese)
2	CHENY W, MOUSSIJ, DRURYJ L, et al. Zirconia in biomedical applications[J]. Expert Rev Med Devices, 2016: 945-963.
3	VIOLAG, CHONGK B, ERIKSSONM, et al. Effect of grain size on domain structures, dielectric and thermal depoling of Nd-substituted bismuth titanate ceramics[J]. Applied Physics Letters, 2013, 103(18): 899.
4	雷涛, 李云彤, 周文政, 等. 数据与模型联合驱动的陶瓷材料晶粒分割[J]. 自动化学报, 2020, 46(x): 1-16. DOI:10.16383/j.aas.c200277.
	LEITao, LIYuntong, ZHOUWenzheng, et al. Grain segmentation of ceramic materials using data-driven jointing model-driven[J]. Acta Automatica Sinica, 2020, 46(x): 1-16. DOI:10.16383/j.aas.c200277. (in Chinese)
5	BODNARJ L, NICOLASJ L, CANDORÉJ C. Non-destructive testing by infrared thermography under random excitation and ARMA analysis[J]. International Journal of Thermophysics, 2012, 33(10-11): 2011-2015.
6	ZHANGL X, XUZ, WEIS, et al. Grain size automatic determination for 7050 Al alloy based on a fuzzy logic method[J]. Rare Metal Materials and Engineering, 2016, 45: 548-554.
7	LIQ, NIX, LIUG. Ceramic image processing using the second curvelet transform and watershed algorithm[C]//IEEE International Conference on Robotics and Biomimetics(ROBIO). Sanya, China: IEEE, 2007: 2037-2042.
8	VINCENTL, SOILLEP. Watersheds in digital spaces: An efficient algorithm based on immersion simulations[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1991, 13(6): 583-598.
9	HEILBRONNERR. Automatic grain boundary detection and grain size analysis using polarization micrographs or orientation images[J]. Journal of Structural Geology, 2000, 22: 969-981.
10	SCHMIDHUBERJ. Deep learning in neural networks: An overview[J]. Neural networks, 2015, 61: 85-117.
11	RONNEBERGERO, FISCHERP, BROXT. U-Net: Convolutional networks for biomedicalimage segmentation[EB/OL]. (2015-05-18). .
12	LONGJ, SHELHAMERE, DARRELLT. Fully convolutional networks for semantic segmentation[C]//IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Boston, MA, USA: IEEE, 2015: 3431-3440.
13	JIANGF, GUQ, HAOH, et al. Feature extraction and grain segmentation of sandstone images based on convolutional neural networks[C]//24th International Conference on Pattern Recognition(ICPR). Beijing, China: IEEE, 2018: 2636-2641.
14	MAB, LIUC, WEIX, et al. WPU-Net: Boundary learning by usi-ng weighted propagation in convolution network[EB/OL]. (2019-05-22). .
15	LIUY, CHENGMM, HUX, et al. Richer convolutional features for edge detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA: IEEE, 2017: 3000-3009.
16	ZOPHB, LEQ V. Neural architecture search with reinforcement learning[EB/OL]. 78, 2017-02-15.
17	LIUC, CHENLC, SCHROFFF, et al. Auto-deeplab: Hierarchical neural architecture search for semantic image segmentation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Long Beach, CA, USA: IEEE, 2019: 82-92.
18	ZHANGY, QIUZ, LIUJ, et al. Customizable Architecture Search for Semantic Segmentation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Long Beach, CA, USA: IEEE, 2019: 11633-11642.
19	ELSKENT, METZENJ H, HUTTERF. Neural architecture search: A survey[EB/OL]. (2019-04-26). .
20	ZHONGZ, YANJ, WUW, et al. Practical block-wise neural network architecture generation[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018: 2423-2432.
21	ZOPHB, VASUDEVANV, SHLENSJ, et al. Learning transferable architectures for scalable image recognition[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018: 8697-8710.
22	CHENLC, ZHUY, PAPANDREOUG, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of the European Conference on Computer Vision(ECCV). Munich: Springer, 2018: 801-818.
23	ELSKENT, METZENJ H, HUTTERF. Simple and efficient architecture search for convolutional neural networks[EB/OL]. (2017-11-13). .
24	BAKERB, GUPTAO, NAIKN, et al. Designing neural network architectures using reinforcement learning[EB/OL]. (2017-03-22). .
25	HEK, ZHANGX, RENS, et al. Deep residual learning for image recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Seattle, WA, USA: IEEE, 2016: 770-778.
26	REALE, AGGARWALA, HUANGY, et al. Regularized evolution for image classifier architecture search[EB/OL]. (2019-02-16). .
27	LIUC, ZOPHB, SHLENSJ, et al. Progressive neural architecture search[EB/OL]. (2018-07-26). .
28	NOH H, HONGS, HANB. Learning deconvolution network for semantic segmentation[C]//IEEE International Conference on Computer Vision(ICCV). Santiago, Chile: IEEE, 2015: 1520-1528.
29	BADRINARAYANANV, KENDALLA, CIPOLLAR. SegNet: A deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39: 2481-2495.
30	PHAMH, GUANM Y, ZOPHB, et al. Efficient neural architecture search via parameter sharing[EB/OL]. (2018-02-12).
31	ZHAOH, SHIJ, QIX, et al. Pyramid scene parsing network[C]//IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Honolulu, HI, USA: IEEE, 2017: 6230-6239.
32	ZAREMBAW, SUTSKEVERI, VINYALSO. Recurrent neural network regularization[EB/OL]. (2015-02-19). .
33	SUTTONR S, MCALLESTERD, SINGHS, et al. Policy gradient methods for reinforcement learning with function approximation[J]. Advances in Neural Information Processing Systems, 1999, 12: 1057-1063.
34	ROMERAE, ALVAREZJ M, BERGASAL M, et al. ERFNet: Efficient residual factorized convnet for real-time semantic segmentation[J]. IEEE Transactions on Intelligent Transportation Systems, 2018, 19(1): 263-272.
35	PASZKEA, CHAURASIAA, KIMS, et al. ENet: A deep neural network architecture for real-time semantic segmentation[EB/OL].(2016-06-07). .
36	CHENLC, PAPANDREOUG, SCHROFFF, et al. Rethinking atrous convolution for semantic image segmentation[EB/OL]. (2017-12-05). .

[1]	唐利明, 熊点华, 方壮. 基于比尔朗伯定律的变分水平集模型[J]. 电子学报, 2023, 51(2): 416-426.
[2]	金天虎, 陶砚蕴, 李佐勇. 基于超像素图像分割的暗通道先验去雾改进算法[J]. 电子学报, 2023, 51(1): 146-159.
[3]	张淑军, 彭中, 李辉. SAU-Net：基于U-Net和自注意力机制的医学图像分割方法[J]. 电子学报, 2022, 50(10): 2433-2442.
[4]	吴绿, 张馨月, 唐茉, 王梓, 王永安. Focus+Context语义表征的场景图像分割[J]. 电子学报, 2021, 49(3): 596-604.
[5]	朱占龙, 刘永军. 融合混沌优化和改进模糊聚类的图像分割算法[J]. 电子学报, 2020, 48(5): 975-984.
[6]	梁新宇, 林洗坤, 权冀川, 肖铠鸿. 基于深度学习的图像实例分割技术研究进展[J]. 电子学报, 2020, 48(12): 2476-2486.
[7]	石雪, 李玉, 赵泉华. 自适应类别的层次高斯混合模型遥感影像分割[J]. 电子学报, 2020, 48(1): 131-136.
[8]	雷涛, 张肖, 加小红, 刘侍刚, 张艳宁. 基于模糊聚类的图像分割研究进展[J]. 电子学报, 2019, 47(8): 1776-1791.
[9]	侯小刚, 赵海英, 马严. 基于超像素多特征融合的快速图像分割算法[J]. 电子学报, 2019, 47(10): 2126-2133.
[10]	范虹, 张程程, 侯存存, 朱艳春, 姚若侠. 结合双树复小波变换和改进密度峰值快速搜索聚类的乳腺MR图像分割[J]. 电子学报, 2019, 47(10): 2149-2157.
[11]	张峰, 钟宝江. 基于兴趣目标的图像检索[J]. 电子学报, 2018, 46(8): 1915-1923.
[12]	李亚峰. 一种基于多字典学习的图像分割模糊方法[J]. 电子学报, 2018, 46(7): 1700-1709.
[13]	李磊, 董卓莉, 张德贤. 基于自适应区域限制FCM的图像分割方法[J]. 电子学报, 2018, 46(6): 1312-1318.
[14]	周则明, 胡彪, 孟勇, 陈超迁, 罗其祥. 基于流形特征与形状先验的红外直升机图像分割[J]. 电子学报, 2018, 46(4): 834-839.
[15]	王慧斌, 高国伟, 徐立中, 文成林. 基于纹理特征的多区域水平集图像分割方法[J]. 电子学报, 2018, 46(11): 2588-2596.

ED-NAS: 基于神经网络架构搜索的陶瓷晶粒SEM图像分割方法

ED-NAS: Ceramic Grain Segmentation Based on Neural Architecture Search Using SEM Images

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献 36

相关文章 15

编辑推荐

Metrics