软分割约束边缘保持插值的半自动2D转3D

doi:10.3969/j.issn.0372-2112.2015.11.012

电子学报 ›› 2015, Vol. 43 ›› Issue (11): 2218-2224.DOI: 10.3969/j.issn.0372-2112.2015.11.012

软分割约束边缘保持插值的半自动2D转3D

吴少群, 袁红星, 安鹏, 程培红

宁波工程学院电子与信息工程学院, 浙江宁波 315016

收稿日期:2014-05-12 修回日期:2014-12-01 出版日期:2015-11-25
- 通讯作者:
- 袁红星
- 作者简介:
- 吴少群 女,1981年12月出生,安徽安庆人.2011年毕业于华东师范大学职成教研究所,获教育学硕士学位.现为宁波工程学院实验师,主要从事图像与信号处理、3D视频处理等方面的研究工作.E-mail:vison101@163.com
- 基金资助:
- 浙江省自然科学基金 (No.LQ14A040002,No.LQ12D01001,No.LQ12D0100,No.LQ12F03001); 浙江省教育厅科研项目 (No.Y201431834); 宁波市自然科学基金 (No.2012A610048,No.2013A610114)

Semi-Automatic 2D-to-3D Conversion Using Soft Segmentation Constrained Edge-Aware Interpolation

WU Shao-qun, YUAN Hong-xing, AN Peng, CHENG Pei-hong

School of Electronics and Information Engineering, Ningbo University of Technology, Ningbo, Zhejiang 315016, China

Received:2014-05-12 Revised:2014-12-01 Online:2015-11-25 Published:2015-11-25

摘要/Abstract

摘要：

半自动2D转3D将用户标注的稀疏深度转换成稠密深度,是解决3D片源不足的主要手段之一.针对现有方法利用硬分割增强深度边缘引入误差的问题,提出像素点与超像素深度一致性约束的边缘保持插值方法.首先,建立像素点深度和超像素深度传播的能量模型,通过像素点与所属超像素间深度差异的约束项将二者关联起来;其次,利用矩阵表示形式将两个能量模型的最优化转换成一个稀疏线性方程组的求解问题.通过超像素提供的约束项,可避免深度传播穿过低对比度边缘区域,从而能保持对象边缘.实验结果表明,本文方法对象边缘处深度恢复的准确性优于融合图割的随机游走方法,PSNR改善了1.5dB以上.

关键词: 二维转三维, 图割, 随机游走, 软分割

Abstract:

Semi-automatic 2D-to-3D conversion is a promising solution to 3D stereoscopic content creation.Its main process is to estimate the dense depth map from user-defined strokes on the image.Existing methods preserve depth boundaries by incorporating hard segmentation.However,the inexact segmentation around object boundaries will decrease depth accuracy around these regions.To help solve this problem,an edge-aware interpolation method is developed which is constrained by depth consistency between pixels and superpixels.First,we formulate depth propagation in terms of two energy functions of pixels and superpixels,which are influenced by each other through the constraint of soft segmentation.Second,the energy functions are reformulated in matrix forms and they are solved jointly in a sparse linear equation.We recover depth boundaries with help of the superpixels constraint which prevents depth propagation across low contrast edge regions.Experimental comparisons with existing algorithms show that our method demonstrates significant advantages over object boundaries.The PSNR is improved by more than 1.5 dB compared with hybrid graph-cuts and random-walks approach.

Key words: 2D-to-3D conversion, graph-cuts, random-walks, soft segmentation

中图分类号:

TN911.73

吴少群, 袁红星, 安鹏, 程培红. 软分割约束边缘保持插值的半自动2D转3D[J]. 电子学报, 2015, 43(11): 2218-2224.

WU Shao-qun, YUAN Hong-xing, AN Peng, CHENG Pei-hong. Semi-Automatic 2D-to-3D Conversion Using Soft Segmentation Constrained Edge-Aware Interpolation[J]. Acta Electronica Sinica, 2015, 43(11): 2218-2224.

参考文献

[1] 刘伟,吴毅红.基于图层优化与融合的2D-3D视频转换方法[J]. 计算机辅助设计与图形学学报,2012,24(11):1426-1439. Liu Wei,Wu Yi-hong.A 2D-3D video conversion method based on layer optimization and intergration[J]. Journal of Computer-Aided Design & Computer Graphics,2012,24(11):1426-1439.(in Chinese)
[2] 赖文能,陈韦志.浅谈2D至3D视讯转换技术[J]. 影像与识别,2010,16(2):61-75. Lai Wen-eng,Chen Wei-zhi.2D-to-3D video conversion technologies overview[J]. Images & Recognition,2010,16(2):61-75.(in Chinese)
[3] Wang O,Lang M,Frei M,et al.StereoBrush:interactive 2D to 3D conversion using discontinuous warps[A]. Proceedings of Eurographics Symposium on Sketch-Based Interfaces and Modeling[C]. New York:ACM Press,2011.47-54.
[4] Guttmann M,Wolf L,Cohen-Or D.Semi-automatic stereo extraction from video footage[A]. Proceedings of IEEE International Conference on Computer Vision[C]. Los Alamitos:IEEE Computer Society Press,2009.136-142.
[5] Rzeszutek R,Phan R,Androutsos D.Semi-automatic synthetic depth map generation for video using random walks[A]. Proceedings of IEEE International Conference on Multimedia and Expo[C]. Los Alamitos:IEEE Computer Society Press,2011.1-6.
[6] Levin A,Lischinski D,Weiss Y.Colorization using optimization[A]. Proceedings of ACM SIGGRAPH[C]. New York:ACM Press,2004.689-694.
[7] 褚宏莉,李元祥,周则明,等.基于黑色通道的图像快速去雾优化算法[J]. 电子学报,2013,41(4):791-797. Chu Hong-li,Li Yuan-xiang,Zhou Ze-ming,et al.Optimized fast dehazing method based on dark channel prior[J]. Acta Electronica Sinica,2013,41(4):791-797.(in Chinese)
[8] Hu Wei,Dong Zhao,Yuan Guo-dong.Edit propagation via edge-aware filtering[J]. Journal of Computer Science and Technology,2012,27(4):830-840.
[9] 袁红星,吴少群,朱仁祥,等.融合对象性和视觉显著度的单目图像2D转3D[J]. 中国图象图形学报,2013,18(10):1478-1485. Yuan Hong-xing,Wu Shao-qun,Zhu Ren-xiang,et al.Single-view image 2D-to-3D conversion based on objectness and visual saliency[J]. Journal of Image and Graphics,2013,18(10):1478-1485.(in Chinese)
[10] Phan R,Androutsos D.Robust semi-automatic depth map generation in unconstrained images and video sequences for 2D to stereoscopic 3D conversion[J]. IEEE Transactions on Multimedia,2014,16(1):122-136.
[11] Xu X,Po L M,Ng K H,et al.Watershed and random walks based depth estimation for semi-automatic 2D to 3D image conversion[A]. Proceeding of International Conference on Signal Processing,Communications and Computing[C]. Los Alamitos:IEEE Computer Society Press,2012.84-87.
[12] Zhang L,Vazquez C,Knorr S.3D-TV content creation automatic 2D-to-3D video conversion[J]. IEEE Transactions on Broadcasting,2011,57(2):372-383.
[13] Achanta R,Shaji A,Smith K,et al.SLIC superpixels compared to state-of-the-art superpixel methods[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,34(11):2274-2282.
[14] Barrett R,Berry M,Chan T F,et al.Templates for the Solution of Linear Systems:Building Blocks for Iterative Methods[M]. Philadelphia:SIAM Press,1994.35-48.
[15] 曹汛.2D to 3D conversion test sequences[EB/OL]. http://media.au.tsinghua.edu.cn/2Dto3D/testsequence.html,2014-05-12/2014-05-12.
[16] Zitnick C L,Kang S B,Uyttendaele M,et al.High-quality video view interpolation using a layered representation[J]. ACM Transactions on Graphics,2004,23(3):600-608.
[17] Ranftl R,Gehrig S,Pock T,et al.Pushing the limits of stereo using variational stereo estimation[A]. Proceedings of IEEE Intelligent Vehicles Symposium[C]. Los Alamitos:IEEE Computer Society Press,2012.401-407.
[18] Vedaldi A,Fulkerson B.VLFeat:an open and portable library of computer vision algorithms[EB/OL]. http://www.vlfeat.org,2008-01-01/2014-05-12.
[19] 郁理,郭立,袁红星.基于深度图像的视点绘制新方法[J]. 中国科学院研究生院学报,2010,27(5):638-644. Yu Li,Guo Li,Yuan Hong-xing.A novel method of depth-image-based view synthesis[J]. Journal of the Graduate School of the Chinese Academy of Sciences,2010,27(5):638-644.(in Chinese)

软分割约束边缘保持插值的半自动2D转3D

Semi-Automatic 2D-to-3D Conversion Using Soft Segmentation Constrained Edge-Aware Interpolation

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	李青青, 马慧芳, 李举, 李志欣. 属性网络中相似性增强随机游走的多社区搜索[J]. 电子学报, 2021, 49(11): 2096-2100.
[2]	李敏, 王晓桐, 罗慧敏, 孟祥茂, 王建新. 随机游走技术在网络生物学中的研究进展[J]. 电子学报, 2018, 46(8): 2035-2048.
[3]	马慧芳, 刘芳, 夏琴, 郝占军. 基于加权超图随机游走的文献关键词提取算法[J]. 电子学报, 2018, 46(6): 1410-1414.
[4]	袁红星, 安鹏, 吴少群, 郑悠. 具有容错能力的L₁最优化半自动2D转3D[J]. 电子学报, 2018, 46(2): 447-455.
[5]	方晨, 张恒巍, 王娜, 王晋东. 基于随机游走和多样性图排序的个性化服务推荐方法[J]. 电子学报, 2018, 46(11): 2773-2780.
[6]	牛新征, 牛嘉郡, 苏大壮, 佘堃. 基于加权内容-结构网络和随机游走的社团划分算法[J]. 电子学报, 2017, 45(9): 2135-2142.
[7]	葛婷, 牟宁, 李黎. 基于softmax回归与图割法的脑肿瘤分割算法[J]. 电子学报, 2017, 45(3): 644-649.
[8]	康世泽, 马宏, 黄瑞阳. 一种基于Opinosis图和马尔科夫随机游走模型的多文本情感摘要框架[J]. 电子学报, 2017, 45(12): 3005-3011.
[9]	李磊, 董卓莉, 张德贤, 费选. 基于区域限制的EM和图割的非监督彩色图像分割方法[J]. 电子学报, 2016, 44(6): 1349-1354.
[10]	施文灶, 毛政元. 基于图割与阴影邻接关系的高分辨率遥感影像建筑物提取方法[J]. 电子学报, 2016, 44(12): 2849-2854.
[11]	姜明新, 王培昌, 王洪玉. 基于视频多目标跟踪的高度测量算法[J]. 电子学报, 2015, 43(3): 591-596.
[12]	陈圣国, 孙正兴, 周杰. 基于FCM和随机游走的地层图像分割方法[J]. 电子学报, 2013, 41(3): 526-531.
[13]	文韬, 葛泉波. 未建模系统基于观测值的实时分块Kalman滤波估计方法研究[J]. 电子学报, 2012, 40(10): 1958-1964.
[14]	李昱川;田铮;刘向增;徐海霞. SAR图像多尺度配准的小波域最小割模型方法[J]. 电子学报, 2010, 38(9): 2122-2127.
[15]	吴亚东, 孙世新, 张红英, 韩永国, 陈波. 一种基于图割的全变差图像去噪算法[J]. 电子学报, 2007, 35(2): 265-268.