双线阵无透镜超分辨率扫描成像方法研究

doi:10.12263/DZXB.20210946

摘要/Abstract

摘要：

针对无透镜成像系统分辨率低和单线阵扫描成像中由于细胞流速不准确造成的图像重构畸变问题，本文提出了一种基于双线阵图像传感器的超分辨率扫描成像方法.通过引入第二个线阵，利用双线阵的时间空间关系，实现了细胞流动速度的实时测量，为扫描图像的重构提供了精准参数.综合考虑了光源、衍射光路、细胞样品瞬态流速、图像传感器像素尺寸、倾斜角度和帧频等系统特性，建立了线阵扫描成像系统的成像模型，深入分析了影响双线阵成像质量的因素，建立了基于坐标变换、扫描图像插值计算的双线阵扫描成像系统的图像重构模型.综合两种模型，以条形图案为仿真对象，分析了该方法的成像特性，并从理论上验证了双线阵扫描成像系统的可行性和有效性.仿真结果表明，双线阵扫描成像系统的空间分辨率相比面阵系统至少提升了2.8倍，并且结构相似性和峰值信噪比均有5%~15%的提升.在此基础上，搭建了双线阵扫描成像系统，并以微球、红细胞和白细胞作为测试对象，对系统进行了实际测试.结果表明，双线阵扫描成像系统解决了单线阵扫描成像图像重构畸变的问题，同时相比面阵成像系统，20 μm微球的直径均值误差为2.04%，红细胞和白细胞的特征更多、更明显.

关键词: 双线阵图像传感器, 超分辨率扫描成像, 扫描成像模型, 图像重构模型, 坐标变换, 扫描图像插值

Abstract:

Aiming at the problems of the low resolution in lensless imaging systems and the reconstruction distortion in single-line array scanning imaging systems, a super-resolution scanning imaging method is proposed for lensless using a dual-line array image sensor. The cell’s velocity can be measured in real-time using the space-time displacement of the dual-line array, which provides accurate parameters for the reconstruction. Considering the system characteristics of optical wavelength, diffraction light path, cell sample flow rate, pixel size of the image sensor, tilt angle, and frame rate, the scanning imaging model of the scanning imaging system is established. The factors affecting the imaging quality of the dual-line array are analyzed, and the image reconstruction model of the dual-line array scanning imaging system is established by coordinate transformation and scanned image interpolation calculation. By synthesizing the two models and taking the strip pattern as the simulation object, the imaging characteristics are analyzed, and the feasibility and effectiveness of the dual-line array scanning imaging system are verified theoretically. The simulation results show that the spatial resolution of the scanning imaging system with a dual-line array is at least 2.8 times higher than that of the system with an area array, and the SSIM and PSNR are improved by 5%~15%. On this basis, a scanning imaging system for lensless with a dual-line array was built, and the microspheres, red blood cells, and white blood cells were used as test objects to test the system. The results show that the problems of the reconstruction distortion in single-line array scanning imaging systems can be solved with a dual-line array scanning imaging. The mean diameter error of 20 μm microsphere is 2.04% between the scanning imaging system with dual-line array and the area array imaging system, and the characteristics of red blood cells and white blood cells are more obvious under the scanning imaging system with the dual-line array.

Key words: dual-line array image sensor, super-resolution scanning imaging, scanning imaging model, image reconstruction model, coordinate transformation, scanning image interpolation

中图分类号:

TN911.7

田典, 余宁梅, 周广霖, 李娜. 双线阵无透镜超分辨率扫描成像方法研究[J]. 电子学报, 2022, 50(10): 2503-2516.

TIAN Dian, YU Ning-mei, ZHOU Guang-lin, LI Na. Research on Super-Resolution Scanning Imaging Method for Lensless with Dual-Line Array[J]. Acta Electronica Sinica, 2022, 50(10): 2503-2516.

图/表 26

图1 扫描成像模型的流程图

图2 细胞图像与双线阵的初始位置

图3 细胞在t时刻所处的位置

图4 纵向的四种边界情况示意图

图5 横向边界条件示意图

表1 流速对比仿真中圆形图案的流动参数

组号	时间t/s	初始速度 v/(μm/s)	加速度 a/(μm/s²)	流动方向θ/(°)
A	0~1	1 200	0	21
B	0 ~ 0.50	1 200	200	21
C	0 ~ 0.26	1 000	2 000	21
	0.26 ~ 0.30	-	5 000	21
	0.30 ~ 0.34	-	-5 000	21
	0.34 ~ 0.38	-	5 000	21
	0.38 ~ 0.50	-	-5 000	21

图6 圆形图案流速对比的仿真结果

表2 流向对比仿真中圆形图案的流动参数

组号	时间t/s	初始速度v/(μm/s)	加速度a/(μm/s²)	流动方向θ/(°)
D	0 ~ 1	1 200	0	30
E	0 ~ 0.26	1 000	2 000	21
	0.26 ~ 0.30	-	5 000	25
	0.30 ~ 0.34	-	-5 000	18
	0.34 ~ 0.38	-	5 000	13
	0.38 ~ 0.50	-	-5 000	20

图7 圆形图案流向对比的仿真结果

图8 扫描图像重构过程的流程图

图9 分段区间的自动检测算法流程图

图10 实心圆图案扫描图像的速度点检测

图11 图像重构过程中坐标系的建立

图12 物体流过线阵的数学模型

图13 物体上a点在线阵扫描图像上的映射关系

图14 物体上a点最近的四点在物体图像映射关系

表3 双线阵扫描成像仿真中的部分参数

参数	$d L / μ m$	$d R / μ m$	$d / μ m$	$λ / n m$	$z 1 / m m$	ƒ	$θ / (°)$
数值	2.2	0.11	4.4	470	0.75	1 000	21

表3 双线阵扫描成像仿真中的部分参数

参数	$d L / μ m$	$d R / μ m$	$d / μ m$	$λ / n m$	$z 1 / m m$	ƒ	$θ / (°)$
数值	2.2	0.11	4.4	470	0.75	1 000	21

图15 线条图案的仿真测试结果

表4 双线阵和面阵系统下线条图案成像的客观评价

图像	SSIM	$M S E$	PSNR/dB
面阵衍射图像	0.749 8	5.362 1	33.544 1
双线阵衍射图像	0.892 0	3.275 9	37.824 1
面阵衍射恢复图像	0.880 8	5.941 4	32.653 0
双线阵衍射恢复图像	0.959 4	4.275 0	35.512 0

表4 双线阵和面阵系统下线条图案成像的客观评价

图像	SSIM	$M S E$	PSNR/dB
面阵衍射图像	0.749 8	5.362 1	33.544 1
双线阵衍射图像	0.892 0	3.275 9	37.824 1
面阵衍射恢复图像	0.880 8	5.941 4	32.653 0
双线阵衍射恢复图像	0.959 4	4.275 0	35.512 0

图16 双线阵扫描成像结构验证测试方案

图17 系统测试的实验平台

图18 染色标记下微球样品的二维聚焦

图19 双线阵扫描成像系统下20μm微球的测试结果

图20 双线阵扫描成像系统下红细胞的成像结果

图21 双线阵扫描成像系统下白细胞的成像结果

图22 双线阵系统和面阵系统下热力图对比

参考文献 38

1	HENG X, ERICKSON D, BAUGH L R, et al. Optofluidic microscopy-A method for implementing a high resolution optical microscope on a chip[J]. Lab Chip, 2006, 6(10): 1274-1276.
2	张佳琳, 陈钱, 张翔宇, 等. 无透镜片上显微成像技术: 理论、发展与应用[J]. 红外与激光工程, 2019, 48(6): 121-153.
	ZHANG J L, CHEN Q, ZHANG X Y, et al. Lens-free on-chip microscopy: Theory, advances, and applications[J]. Infrared and Laser Engineering, 2019, 48(6): 121-153. (in Chinese)
3	MUDANYALI O, TSENG D, OH C, et al. Compact, light-weight and cost-effective microscope based on lensless incoherent holography for telemedicine applications[J]. Lab on a Chip, 2010, 10(11): 1417-1428.
4	LEE S A, LEITAO R, ZHENG G, et al. Color capable sub-pixel resolving optofluidic microscope and its application to blood cell imaging for malaria diagnosis[J]. PLoS One, 2011, 6(10): e26127.
5	ROY M, JIN G, PAN J, et al. Staining-free cell viability measurement technique using lens-free shadow imaging platform[J]. Sensors and Actuators, B: Chemical, 2016, 224: 577-583.
6	ROY M, JIN G, SEO D, et al. A simple and low-cost device performing blood cell counting based on lens-free shadow imaging technique[J]. Sensors and Actuators, B: Chemical, 2014, 201: 321-328.
7	HUANG X, LI Y, XU X, et al. High-Precision Lensless Microscope on a Chip Based on In-Line Holographic Imaging[J]. SENSORS, 2021, 21(3): 720.
8	LI J, DAI L, YU N, et al. Elliptocyte detection technology based on super-resolution algorithms for a lensless imaging system[J]. Measurement Science and Technology, 2021, 32(2): 025701.
9	LI J, DAI L, YU N, et al. Red blood cell recognition and posture estimation in microfluidic chip based on lensless imaging[J]. Biomicrofluidics, 2021, 15(3): 34109.
10	LI S, DAI L, LI J, et al. A Deep learning Feature Fusion Algorithm based on Lensless Cell detection system[C]// Proceedings of the IEEE 15th International Conference on Solid-State & Integrated Circuit Technology. Kunming: IEEE, 2020: 1-3.
11	LIAO Y, YU N, TIAN D, et al. An intelligent low-power low-cost mobile lab-on-chip yeast cell culture platform[J]. IEEE Access, 2020, 8: 70733-70745.
12	LIAO Y, YU N, TIAN D, et al. Toward embedded sensing automation and miniaturization for portable smart cost-effective algae monitor[J]. IEEE Sensors Journal, 2021, 21(4): 5230-5239.
13	MUDANYALI O, BISHARA W, OZCAN A. Lensfree super-resolution holographic microscopy using wetting films on a chip[J]. Optics Express, 2011, 19(18): 17378-17389.
14	ZHANG J, CHEN Q, LI J, et al. Lensfree dynamic super-resolved phase imaging based on active micro-scanning[J]. Optics Letters, 2018, 43(15): 3714.
15	TIAN D, YU N, LI Z, et al. A super-resolution scanning algorithm for lensless microfluidic imaging using the dual-line array image sensor[J]. PLoS One, 2020, 15(6): e235111.
16	张华, 曹良才, 金国藩, 等. 基于压缩感知算法的无透镜数字全息成像研究[J]. 激光与光电子学进展, 2020, 57(8): 9-19.
	ZHANG H, CAO L C, JIN G F, et al. Progress on lensless digital holography imaging based on compressive holographic algorithm[J]. Laser & Optoelectronics Progress, 2020, 57(8): 9-19. (in Chinese)
17	ZUO C, CHEN Q, GU G, et al. Laplacian reconstruction of one single hologram using two different reconstruction distances or wavelengths[J]. Chinese Optics Letters, 2012, 10(6): 50-53.
18	GUO C, ZHANG F, GENG Y, et al. High-quality multi-wavelength lensfree microscopy based on nonlinear optimization[J]. Optics and Lasers in Engineering, 2021, 137(2): 10640.
19	COSKUN A F, SENCAN I, SU T, et al. Wide-field lensless fluorescent microscopy using a tapered fiber-optic faceplate on a chip[J]. The Analyst, 2011, 136(17): 3512.
20	王相海, 赵晓阳, 毕晓昀, 等. 小波域多角度轮廓模板变分模型的单幅图像超分辨率重建[J]. 电子学报, 2018, 46(9): 2256-2262.
	WANG X H, ZHAO X Y, BI X Y, et al. Single image super-resolution reconstruction approach based on multi-angle contour templates variational calculus model in wavelet domain[J]. Acta Electronica Sinica, 2018, 46(9): 2256-2262. (in Chinese)
21	张成, 程鸿, 张芬, 等. 单次曝光频域振幅编码压缩成像[J]. 电子学报, 2014, 42(7): 1262-1267.
	ZHANG C, CHENG H, ZHANG F, et al. Single-exposure frequency-domain amplitude encoding compressive imaging[J]. Acta Electronica Sinica, 2014, 42(7): 1262-1267. (in Chinese)
22	李展, 陈清亮, 彭青玉, 等. 基于MAP的单帧字符图像超分辨率重建[J]. 电子学报. 2015, 43(1): 191-197.
	LI Z, CHEN Q L, PENG Q Y, et al. MAP-based single-frame super-resolution reconstruction for character image[J]. Acta Electronica Sinica, 2015, 43(1): 191-197. (in Chinese)
23	任福全, 邱天爽, 韩军, 等. 基于二阶广义全变差的多帧图像超分辨率重建[J]. 电子学报, 2015, 43(7): 1275-1280.
	REN F Q, QIU T S, HAN J, et al. Multiframe image super resolution based on second order total generalized variation[J]. Acta Electronica Sinica, 2015, 43(7): 1275-1280. (in Chinese)
24	唐艳秋, 潘泓, 朱亚平, 等. 图像超分辨率重建研究综述[J]. 电子学报. 2020, 48(7): 1407-1420.
	TANG Y Q, PAN H, ZHU Y P, et al. A survey of image super-resolution reconstruction[J]. Acta Electronica Sinica, 2020, 48(7): 1407-1420. (in Chinese)
25	ZUO C, SUN J, ZHANG J, et al. Lensless phase microscopy and diffraction tomography with multi-angle and multi-wavelength illuminations using a LED matrix[J]. Optics Express, 2015, 23(11): 14314.
26	FANG Y, YU N, JIANG Y. Super-Resolution Lensless Imaging of Cells Using Brownian Motion[J]. APPLIED SCIENCES-BASEL, 2019, 9(10): 2080.
27	Dai L, Li J, Yu N. Cell image reconstruction for a lens-free imaging system based on linear array sensor[C]//Proceedings of the IEEE International Conference on Electron Devices and Solid-State Circuits, Xi'an: IEEE, 2019: 1-3.
28	王伍娟. 基于片上细胞检测系统的细胞图像超分辨率技术的研究[D]. 西安: 西安理工大学, 2018.
	WANG W J. Research on Cell Image Super-Resolution Technology Based on On-Chip Cell Detection System[D]. Xi'an: Xi'an University of Technology, 2018.
29	方元. 无透镜显微成像及细胞检测关键技术研究[D]. 西安: 西安理工大学, 2019.
	FANG Y. Research on The Key Techniques of Lens-Less Microscopic Imaging and Cell Detection[D]. Xi'an: Xi'an University of Technology, 2019.
30	王广俊, 王大勇, 王华英. 数字全息显微中常见重建算法比较[J]. 激光与光电子学进展, 2010, 47(3): 83-88.
	WANG G J, WANG D Y, WANG H Y. Comparison of commonly used numerical reconstruction algorithms in digital holographic microscopy[J]. Laser & Optoelectronics Progress, 2010, 47(3): 83-88. (in Chinese)
31	KUMAR S, MAHADEVAPPA M, DUTTA P K. Lensless in-line holographic microscopy with light source of low spatio-temporal coherence[J]. IEEE Journal of Selected Topics in Quantum Electronics, 2021, 27(4): 1-8.
32	CHEN L, CHEN X, CUI H, et al. Image enhancement in lensless inline holographic microscope by inter-modality learning with denoising convolutional neural network[J]. Optics Communications, 2021, 484 (2): 126682.
33	WU J, CAO L, BARBASTATHIS G. DNN-FZA camera: A deep learning approach toward broadband FZA lensless imaging[J]. Optics Letters, 2021, 46(1): 130-133.
34	ZHOU H, FENG H, HU Z, et al. Lensless cameras using a mask based on almost perfect sequence through deep learning[J]. Optics Express, 2020, 28(20): 30248-30262.
35	戎路, 王大勇, 王云新,等. 同轴数字全息中的相位恢复算法[J]. 中国激光, 2014, 41(2): 63-72.
	RONG L, WANG D Y, WANG Y X, et al. Phase retrieval methods in in-line digital holography[J]. Chinese Journal of Lasers, 2014, 41(2): 63-72. (in Chinese)
36	ROSTEN E, DRUMMOND T. Machine learning for high-speed corner detection[C]//Proceedings of the 9th European Conference on Computer Vision. Graz: Springer, 2006: 430-443.
37	FANG Y, YU N, JIANG Y, et al. High-precision lens-less flow cytometer on a chip[J]. Micromachines, 2018, 9(5): 227.
38	ZHANG J, SUN J, CHEN Q, et al. Adaptive pixel-super-resolved lensfree in-line digital holography for wide-field on-chip microscopy[J]. Scientific Reports, 2017, 7(1): 11715-11777.

[1]	吕颖达, 申铉京, 陈海鹏. 具有几何不变性的图像复制-粘贴盲鉴别算法[J]. 电子学报, 2016, 44(11): 2592-2599.
[2]	饶伟;郭业才;汪胜前;谭文群;夏非;刘剑冰. 基于坐标变换的常数模盲均衡新算法[J]. 电子学报, 2011, 39(1): 7-12.
[3]	王文惠;孟兵;万建伟;周良柱. 利用不变量进行基于内容的图像检索[J]. 电子学报, 2002, 30(7): 949-951.
[4]	阮成礼. 椭圆锥带线的严格解[J]. 电子学报, 2001, 29(6): 829-831.
[5]	刘超;薛正辉;高本庆;刘瑞祥;杨仕明. 时域近场测试中的探头误差分析与修正[J]. 电子学报, 2001, 29(12): 1689-1692.