Semi-automatic 2D-to-3D conversion is a promising solution to 3D stereoscopic content creation.Its main process is to estimate the dense depth map from user-defined strokes on the image.Existing methods preserve depth boundaries by incorporating hard segmentation.However,the inexact segmentation around object boundaries will decrease depth accuracy around these regions.To help solve this problem,an edge-aware interpolation method is developed which is constrained by depth consistency between pixels and superpixels.First,we formulate depth propagation in terms of two energy functions of pixels and superpixels,which are influenced by each other through the constraint of soft segmentation.Second,the energy functions are reformulated in matrix forms and they are solved jointly in a sparse linear equation.We recover depth boundaries with help of the superpixels constraint which prevents depth propagation across low contrast edge regions.Experimental comparisons with existing algorithms show that our method demonstrates significant advantages over object boundaries.The PSNR is improved by more than 1.5 dB compared with hybrid graph-cuts and random-walks approach.
[1] 刘伟,吴毅红.基于图层优化与融合的2D-3D视频转换方法[J]. 计算机辅助设计与图形学学报,2012,24(11):1426-1439. Liu Wei,Wu Yi-hong.A 2D-3D video conversion method based on layer optimization and intergration[J]. Journal of Computer-Aided Design & Computer Graphics,2012,24(11):1426-1439.(in Chinese)
[2] 赖文能,陈韦志.浅谈2D至3D视讯转换技术[J]. 影像与识别,2010,16(2):61-75. Lai Wen-eng,Chen Wei-zhi.2D-to-3D video conversion technologies overview[J]. Images & Recognition,2010,16(2):61-75.(in Chinese)
[3] Wang O,Lang M,Frei M,et al.StereoBrush:interactive 2D to 3D conversion using discontinuous warps[A]. Proceedings of Eurographics Symposium on Sketch-Based Interfaces and Modeling[C]. New York:ACM Press,2011.47-54.
[4] Guttmann M,Wolf L,Cohen-Or D.Semi-automatic stereo extraction from video footage[A]. Proceedings of IEEE International Conference on Computer Vision[C]. Los Alamitos:IEEE Computer Society Press,2009.136-142.
[5] Rzeszutek R,Phan R,Androutsos D.Semi-automatic synthetic depth map generation for video using random walks[A]. Proceedings of IEEE International Conference on Multimedia and Expo[C]. Los Alamitos:IEEE Computer Society Press,2011.1-6.
[6] Levin A,Lischinski D,Weiss Y.Colorization using optimization[A]. Proceedings of ACM SIGGRAPH[C]. New York:ACM Press,2004.689-694.
[7] 褚宏莉,李元祥,周则明,等.基于黑色通道的图像快速去雾优化算法[J]. 电子学报,2013,41(4):791-797. Chu Hong-li,Li Yuan-xiang,Zhou Ze-ming,et al.Optimized fast dehazing method based on dark channel prior[J]. Acta Electronica Sinica,2013,41(4):791-797.(in Chinese)
[8] Hu Wei,Dong Zhao,Yuan Guo-dong.Edit propagation via edge-aware filtering[J]. Journal of Computer Science and Technology,2012,27(4):830-840.
[9] 袁红星,吴少群,朱仁祥,等.融合对象性和视觉显著度的单目图像2D转3D[J]. 中国图象图形学报,2013,18(10):1478-1485. Yuan Hong-xing,Wu Shao-qun,Zhu Ren-xiang,et al.Single-view image 2D-to-3D conversion based on objectness and visual saliency[J]. Journal of Image and Graphics,2013,18(10):1478-1485.(in Chinese)
[10] Phan R,Androutsos D.Robust semi-automatic depth map generation in unconstrained images and video sequences for 2D to stereoscopic 3D conversion[J]. IEEE Transactions on Multimedia,2014,16(1):122-136.
[11] Xu X,Po L M,Ng K H,et al.Watershed and random walks based depth estimation for semi-automatic 2D to 3D image conversion[A]. Proceeding of International Conference on Signal Processing,Communications and Computing[C]. Los Alamitos:IEEE Computer Society Press,2012.84-87.
[12] Zhang L,Vazquez C,Knorr S.3D-TV content creation automatic 2D-to-3D video conversion[J]. IEEE Transactions on Broadcasting,2011,57(2):372-383.
[13] Achanta R,Shaji A,Smith K,et al.SLIC superpixels compared to state-of-the-art superpixel methods[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,34(11):2274-2282.
[14] Barrett R,Berry M,Chan T F,et al.Templates for the Solution of Linear Systems:Building Blocks for Iterative Methods[M]. Philadelphia:SIAM Press,1994.35-48.
[15] 曹汛.2D to 3D conversion test sequences[EB/OL]. http://media.au.tsinghua.edu.cn/2Dto3D/testsequence.html,2014-05-12/2014-05-12.
[16] Zitnick C L,Kang S B,Uyttendaele M,et al.High-quality video view interpolation using a layered representation[J]. ACM Transactions on Graphics,2004,23(3):600-608.
[17] Ranftl R,Gehrig S,Pock T,et al.Pushing the limits of stereo using variational stereo estimation[A]. Proceedings of IEEE Intelligent Vehicles Symposium[C]. Los Alamitos:IEEE Computer Society Press,2012.401-407.
[18] Vedaldi A,Fulkerson B.VLFeat:an open and portable library of computer vision algorithms[EB/OL]. http://www.vlfeat.org,2008-01-01/2014-05-12.
[19] 郁理,郭立,袁红星.基于深度图像的视点绘制新方法[J]. 中国科学院研究生院学报,2010,27(5):638-644. Yu Li,Guo Li,Yuan Hong-xing.A novel method of depth-image-based view synthesis[J]. Journal of the Graduate School of the Chinese Academy of Sciences,2010,27(5):638-644.(in Chinese)
2015, Vol. 43 
