具有容错能力的L1最优化半自动2D转3D

doi:10.3969/j.issn.0372-2112.2018.02.026

电子学报 ›› 2018, Vol. 46 ›› Issue (2): 447-455.DOI: 10.3969/j.issn.0372-2112.2018.02.026

具有容错能力的L₁最优化半自动2D转3D

袁红星, 安鹏, 吴少群, 郑悠

宁波工程学院电子与信息工程学院, 浙江宁波 315211

收稿日期:2016-06-27 修回日期:2016-10-21 出版日期:2018-02-25
- 通讯作者:
- 袁红星
- 作者简介:
- 安鹏,男,1981年11月出生,山西太原人.宁波工程学院电子与信息工程学院副教授.2009年毕业于清华大学核科学与技术专业,获工学博士学位.主要从事信号处理、嵌入式系统设计、智能机器人等方面的研究工作.E-mail:anp04@nbut.edu.cn
- 基金资助:
- 国家自然科学基金 (No.61671260，No.61502256）; 浙江省自然科学基金 (No.LY16F010014，No.LY15F020011，No.LQ14F010001）; 浙江省教育厅科研项目 (No.Y201533511）; 宁波市自然科学基金 (No.2017A610109，No.2013A610114）

Error-Tolerant Semi-Automatic 2D-to-3D Conversion via L₁ Optimization

YUAN Hong-xing, AN Peng, WU Shao-qun, ZHENG You

School of Electronic and Information Engineering, Ningbo University of Technology, Ningbo, Zhejiang 315211, China

Received:2016-06-27 Revised:2016-10-21 Online:2018-02-25 Published:2018-02-25
- Corresponding author:
- YUAN Hong-xing
- Supported by:
- National Natural Science Foundation of China (No.61671260, No.61502256); National Natural Science Foundation of Zhejiang Province, China (No.LY16F010014, No.LY15F020011, No.LQ14F010001); Research Program of Education Department of Zhejiang Province (No.Y201533511); Ningbo Natural Science Fund (No.2017A610109, No.2013A610114)

摘要/Abstract

摘要： 半自动2D转3D的关键是将用户分配的稀疏深度转换为稠密深度.现有方法没有充分考虑纹理图像和深度图之间的结构差异，以及2D转3D对用户误标注的容错性.针对上述问题，借助L₁范数对异常数据的抵制，在一个统一框架下实现结构相关具有容错能力的稀疏深度稠密插值.首先，利用L₁范数表示估计深度和用户分配深度在标注位置的差异，建立数据项；其次，根据特征的相似性用L₁范数计算局部相邻像素点之间的深度差异，建立局部正则项；再次，对图像进行超像素分割，根据不同超像素内代表性像素点之间深度差异的L₁测度，建立全局正则项；最后，用上述数据项和正则项构建能量函数，并通过分裂Bregman算法予以求解.无误差和有误差情况下的实验结果表明，与边缘保持的最优化插值、随机游走、混合图割与随机游走、软分割约束的最优化插值和非局部化随机游走相比，本文估计深度图绘制的虚拟视点图像空洞和伪影损伤更小.在误操作情况下，本文比上述方法PSNR改善了0.9dB以上，且在视觉上屏蔽了用户误操作的影响.

关键词: 2D转3D, 最优化, 随机游走, 图割, L₁范数

Abstract: Sparse-to-dense depth conversion is an important task in semi-automatic 2D-to-3D conversion. Existing methods do not handle structural difference between texture image and depth map, and the error-tolerance of 2D-to-3D is not considered. Inspired by compressive sensing studies, we address these problems in an optimization framework via L₁ norm. First, data term is built with L₁ norm to measure the fidelity between estimated depth and user assigned depth. Second, local regularized term is defined by using feature weighted L₁ norm to measure difference between local neighboring pixels. Third, super-pixels are generated from input image and global regularized term is introduced by using feature weighted L₁ norm to measure difference between representative pixels from these super-pixels. Then, the energy function for sparse-to-dense depth conversion is defined based on the data term, local regularized term and global regularized term. The split Bregman algorithm is used to solve the energy. Experimental comparisons with optimization based interpolation, random-walks, hybrid graph-cuts and random-walks, soft segmentation constrained interpolation and nonlocal random-walks show that our method demonstrates significant advantages over hole and ghosting artifacts for viewpoint synthesis. The PSNR is improved by more than 0.9 dB compared with these methods when user assigns error depth.

Key words: 2D-to-3D conversion, optimization, random-walks, graph-cuts, L₁ norm

中图分类号:

TN911.73

袁红星, 安鹏, 吴少群, 等. 具有容错能力的L₁最优化半自动2D转3D[J]. 电子学报, 2018, 46(2): 447-455.

YUAN Hong-xing, AN Peng, WU Shao-qun, et al. Error-Tolerant Semi-Automatic 2D-to-3D Conversion via L₁ Optimization[J]. Acta Electronica Sinica, 2018, 46(2): 447-455.

参考文献

[1] 王琦.2D到3D图像颜色空间层转换技术研究及实现[J].微型电脑应用,2013,29(3):31-34. WANG Qi.2D to 3D image conversion technology for color space research and implementation[J].Microcomputer Applications,2013,29(3):31-34.(in Chinese)
[2] DOMINIC J M.Recent trends in 2D to 3D conversion:a survey[J].International Journal for Research in Applied Science and Engineering Technology,2014,2(4):388-395.
[3] 王丕涛,王化祥,孙犇渊.基于L₁范数的电容层析成像图像重建算法[J].中国电机工程学报,2015,35(18):4709-4714. WANG Pi-tao,WANG Hua-xiang,SUN Ben-yuan.L₁-norm-based image reconstruction algorithm for electrical capacitance tomography[J].Proceedings of The Chinese Society for Electrical Engineering,2015,35(18):4709-4714.(in Chinese)
[4] RZESZUTEK R,ANDROUTSOS D.Propagating sparse labels through edge-aware filters[J].Signal,Image and Video Processing,2015,9(12):17-24.
[5] VOSTERS L,DE HAAN G.Efficient and stable sparse-to-dense conversion for automatic 2-D to 3-D conversion[J].IEEE Transactions on Circuits and Systems for Video Technology,2013,23(3):373-386.
[6] LIU M Y,TUZEL O,TAGUCHI Y.Joint geodesic upsampling of depth images[A].Proceedings of IEEE Conference on Computer Vision and Pattern Recognition[C].Los Alamitos:IEEE Computer Society,2013.169-176.
[7] DAI L Q,ZHANG F H,MEI X,et al.Fast minimax path-based joint depth interpolation[J].IEEE Signal Processing Letters,2015,22(5):623-627.
[8] LI Y G,ZHANG L,ZHANG Y B,et al.Depth map super-resolution via iterative joint-trilateral-upsampling[A].Proceedings of IEEE Conference on Visual Communications and Image Processing Conference[C].Los Alamitos:IEEE Computer Society,2014.386-389.
[9] HUA K L,LO K H,WANG Y C.Extended guided filtering for depth map upsampling[J].IEEE Transactions on Multimedia,2016,23(2):72-83.
[10] LEVIN A,LISCHINSKI D,WEISS Y.Colorization using optimization[A].Proceedings of ACM SIGGRAPH[C].New York:ACM,2004.689-694.
[11] RZESZUTEK R,PHAN R,ANDROUTSOS D.Semi-automatic synthetic depth map generation for video using random walks[A].Proceedings of IEEE International Conference on Multimedia and Expo[C].Los Alamitos:IEEE Computer Society,2011.1-6.
[12] PHAN R,ANDOUTSOS D.Robust semi-automatic depth map generation in unconstrained images and video sequences for 2D to stereoscopic 3D conversion[J].IEEE Transactions on Multimedia,2014,16(1):122-136.
[13] 吴少群,袁红星,安鹏,等.软分割约束边缘保持插值的半自动2D转3D[J].电子学报,2015,43(11):2218-2224. WU Shao-qun,YUAN Hong-xing,AN Peng,et al.Semi-automatic 2D-to-3D conversion using soft segmentation constrained edge-aware interpolation[J].Acta Electronica Sinica,2015,43(11):2218-2224.(in Chinese)
[14] YUAN H X,WU S Q,CHENG P H,et al.Nonlocal random walks algorithm for semi-automatic 2D-to-3D image conversion[J].IEEE Signal Processing Letters,2015,22(3):371-374.
[15] LOPEZ A,GARCES E,GUTIERREZ D.Depth from a single image through user interaction[A].Spanish Computer Graphics Conference[C].New York:ACM,2014.1-10.
[16] FERSTL D,REINBACHER C,RANFTL R,et al.Image guided depth upsampling using anisotropic total generalized variation[A].Proceedings of International Conference on Computer Vision[C].Los Alamitos:IEEE Computer Society,2013.993-1000.
[17] KIM S,OH C,KIM Y,et al.Structure-aware depth super-resolution using Gaussian mixture model[A].Proceedings of SPIE[C].Bellingham:Society of Photo-Optical Instrumentation Engineers,2015.93930J.
[18] BAI S,HAN X G,YU Y Z.An L₁ image transform for edge-preserving smoothing and scene-level intrinsic decomposition[J].ACM Transactions on Graphics,2015,34(4):1-12.
[19] SENER O,UGUR K,ALATAN A A.Error-tolerant interactive image segmentation using dynamic and iterated graph-cuts[A].Proceedings of the 2nd ACM International Workshop on Interactive Multimedia on Mobile and Portable Devices[C].New York:ACM,2012.9-16.
[20] KARTIC S,SYLVAIN P,CYRIL S,et al.Accurate binary image selection from inaccurate user input[J].Computer Graphics Forum,2013,32(21):41-50.
[21] FELZENSZWALB P F,HUTTENLOCHER D P.Efficient graph-based image segmentation[J].International Journal of Computer Vision,2004,59(2):167-181.
[22] 郁理,郭立,袁红星.基于分级置信度传播的立体匹配新方法[J].中国图象图形学报,2011,16(1):103-109. YU Li,GUO Li,YUAN Hong-xing.A novel method of stereo matching based on hierarchical belief-propagation[J].Journal of Image and Graphics,2011,16(1):103-109.(in Chinese)
[23] 郁理,郭立,袁红星.基于深度图像的视点绘制新方法[J].中国科学院研究生院学报,2010,27(5):638-644. YU Li,GUO Li,YUAN Hong-xing.A novel method of depth-image-based view synthesis[J].Journal of the Graduate School of the Chinese Academy of Sciences,2010,27(5):638-644.(in Chinese)
[24] GOLDSTEIN T,OSHER S.The split Bregman method for l1 regularized problems[J].SIAM Journal on Imaging Sciences,2009,2(2):323-343.
[25] 李亚峰,冯象初.L₁投影问题的分裂Bregman方法[J].电子学报,2010,38(11):2471-2475. LI Ya-feng,FENG Xiang-chu.The split Bregman method for L₁ projection problems[J].Acta Electronica Sinica,2010,38(11):2471-2475.(in Chinese)
[26] HALE E,YIN W,ZHANG Y.Fixed-point continuation for l1-minimization:methodology and convergence[J].SIAM Journal on Optimization,2008,19(3):1107-1130.
[27] YUAN J,WANG D,CHERIYADAT A M.Factorization-based texture segmentation[J].IEEE Transactions on Image Processing,2015,24(11):3488-3497.
[28] LI Y,ADELSON E,AGARWALA A.ScribbleBoost:adding classification to edge-aware interpolation of local image and video adjustments[J].Computer Graphics Forum,2008,27(4):1255-1264.
[29] YANG J,ZHANG Y,YIN W.An efficient tvl1 algorithm for deblurring multichannel images corrupted by impulsive noise[J].SIAM Journal on Scientific Computing,2009,31(4):2842-2865.
[30] LU S,REN X F,LIU F.Depth enhancement via low-rank matrix completion[A].Proceedings of IEEE Conference on Computer Vision and Pattern Recognition[C].Los Alamitos:IEEE Computer Society,2014.3390-3397.
[31] 袁红星,吴少群,安鹏,等.对象引导的单幅散焦图像深度提取方法[J].电子学报,2014,42(10):2009-2015. YUAN Hong-xing,WU Shao-qun,AN Peng,et al.Object guided depth map recovery from a single defocused image[J].Acta Electronica Sinica,2014,42(10):2009-2015.(in Chinese)
[32] 刘延伟,刘金霞,慈松,等.3DQoE评价方法及其模型研究进展综述[J].电子学报,2015,43(3):568-576. LIU Yan-wei,LIU Jin-xia,CI Song,et al.Overview of 3DQoE assessment and modeling[J].Acta Electronica Sinica,2015,43(3):568-576.(in Chinese)

具有容错能力的L₁最优化半自动2D转3D

Error-Tolerant Semi-Automatic 2D-to-3D Conversion via L₁ Optimization

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	曹阳, 高旻, 余俊良, 范琪琳, 荣文戈, 文俊浩. 基于双图混合随机游走的社会化推荐模型[J]. 电子学报, 2023, 51(2): 286-296.
[2]	李青青, 马慧芳, 李举, 李志欣. 属性网络中相似性增强随机游走的多社区搜索[J]. 电子学报, 2021, 49(11): 2096-2100.
[3]	苗力仁, 扈红超, 霍树民, 程国振. 矢量数据包处理加速的动态防护系统设计与实现[J]. 电子学报, 2019, 47(8): 1724-1730.
[4]	李敏, 王晓桐, 罗慧敏, 孟祥茂, 王建新. 随机游走技术在网络生物学中的研究进展[J]. 电子学报, 2018, 46(8): 2035-2048.
[5]	马慧芳, 刘芳, 夏琴, 郝占军. 基于加权超图随机游走的文献关键词提取算法[J]. 电子学报, 2018, 46(6): 1410-1414.
[6]	方晨, 张恒巍, 王娜, 王晋东. 基于随机游走和多样性图排序的个性化服务推荐方法[J]. 电子学报, 2018, 46(11): 2773-2780.
[7]	牛新征, 牛嘉郡, 苏大壮, 佘堃. 基于加权内容-结构网络和随机游走的社团划分算法[J]. 电子学报, 2017, 45(9): 2135-2142.
[8]	葛婷, 牟宁, 李黎. 基于softmax回归与图割法的脑肿瘤分割算法[J]. 电子学报, 2017, 45(3): 644-649.
[9]	康世泽, 马宏, 黄瑞阳. 一种基于Opinosis图和马尔科夫随机游走模型的多文本情感摘要框架[J]. 电子学报, 2017, 45(12): 3005-3011.
[10]	李磊, 董卓莉, 张德贤, 费选. 基于区域限制的EM和图割的非监督彩色图像分割方法[J]. 电子学报, 2016, 44(6): 1349-1354.
[11]	施文灶, 毛政元. 基于图割与阴影邻接关系的高分辨率遥感影像建筑物提取方法[J]. 电子学报, 2016, 44(12): 2849-2854.
[12]	田野, 练秋生. 基于重加权l₁范数惩罚的远近场混合源定位算法[J]. 电子学报, 2016, 44(10): 2440-2448.
[13]	姜明新, 王培昌, 王洪玉. 基于视频多目标跟踪的高度测量算法[J]. 电子学报, 2015, 43(3): 591-596.
[14]	袁红星, 吴少群, 朱仁祥, 安鹏. 加权SIFT流深度迁移的单幅图像2D转3D[J]. 电子学报, 2015, 43(2): 242-247.
[15]	吴少群, 袁红星, 安鹏, 程培红. 软分割约束边缘保持插值的半自动2D转3D[J]. 电子学报, 2015, 43(11): 2218-2224.