加权SIFT流深度迁移的单幅图像2D转3D

doi:10.3969/j.issn.0372-2112.2015.02.006

电子学报 ›› 2015, Vol. 43 ›› Issue (2): 242-247.DOI: 10.3969/j.issn.0372-2112.2015.02.006

加权SIFT流深度迁移的单幅图像2D转3D

袁红星, 吴少群, 朱仁祥, 安鹏

宁波工程学院电子与信息工程学院, 浙江宁波 315016

收稿日期:2013-07-31 修回日期:2013-12-06 出版日期:2015-02-25
- 通讯作者:
- 袁红星
- 作者简介:
- 吴少群 女,1981年12月出生于安徽安庆,宁波工程学院电子与信息工程学院讲师.2011年毕业于华东师范大学职成教研究所,获教育学硕士学位.主要从事信号与信息处理、3D视觉等领域的研究开发工作. E-mail:vison101@163.com
- 基金资助:
- 国家自然科学基金 (No.61071173); 浙江省自然科学基金 (No.LY12F01001,No.Y1100253); 宁波市自然科学基金 (No.2012A610043,No.2012A610111,No.2011A610186); 浙江省教育厅科研项目 (No.Y201431834)

Single Image 2D-to-3D Conversion via Weighted SIFT Flow

YUAN Hong-xing, WU Shao-qun, ZHU Ren-xiang, AN Peng

School of Electronic and Information Engineering, Ningbo University of Technology, Ningbo, Zhejiang 315016, China

Received:2013-07-31 Revised:2013-12-06 Online:2015-02-25 Published:2015-02-25
- Supported by:
- National Natural Science Foundation of China (No.61071173); National Natural Science Foundation of Zhejiang Province, China (No.LY12F01001, No.Y1100253); Ningbo Natural Science Fund (No.2012A610043, No.2012A610111, No.2011A610186); Scientific Research Project of Zhejiang Education Department (No.Y201431834)

摘要/Abstract

摘要：

2D视频转3D视频是解决3D片源不足的主要手段,而单幅图像的深度估计是其中的关键步骤.提出基于加权SIFT流深度迁移和能量模型优化的单幅图像深度提取方法.首先利用图像的全局描述符从深度图数据库中检索出近邻图像;其次通过SIFT流建立输入图像和近邻图像之间像素级稠密对应关系;再次由SIFT流误差计算迁移权重,将近邻图像对应像素点的深度乘以权重后迁移到输入图像上;然后利用均值滤波对迁移后的近邻图像深度进行融合;最后建立深度图优化能量模型,在尽量接近迁移后近邻图像深度的前提下,平滑梯度较小区域的深度.实验结果表明,该方法降低了估计深度图的平均相对误差,增强了深度图的均匀性.

关键词: 2D转3D, 尺度不变特征变换流, 深度估计, 深度图优化, 能量模型

Abstract:

2D-to-3D conversion is one of the important ways to alleviate the lack of 3D content,in which the key technique is depth estimation from a single image.A depth extraction method based on weighted SIFT flow depth transferring and energy model optimization is proposed.First,k-nearest neighbor images were retrieved by evaluating the global image descriptors between the target image and the images from the RGBD database.Then,pixels of the target image and pixels of its neighbors in the database were matched using SIFT flow,which also generated matching errors used to determine the transferring weights of the neighbors.Next,depth maps of neighboring images were transferred to the target image according to the pixel matching and the transferring weights.The final depth map was obtained by depth refinement based on an energy model.Experimental results show that our scheme can significantly reduce the average relative error and improve the uniformity of the depth map.

Key words: 2D-to-3D conversion, scale invariant feature transform flow, depth estimation, depth map refinement, energy model

中图分类号:

TN911.73

袁红星, 吴少群, 朱仁祥, 等. 加权SIFT流深度迁移的单幅图像2D转3D[J]. 电子学报, 2015, 43(2): 242-247.

YUAN Hong-xing, WU Shao-qun, ZHU Ren-xiang, et al. Single Image 2D-to-3D Conversion via Weighted SIFT Flow[J]. Acta Electronica Sinica, 2015, 43(2): 242-247.

参考文献

[1] 王平,安平,王奎,等.基于区域融合的单视点图像深度信息提取[J].电视技术,2011,35(19):11-13. Wang P,An P,Wang K,et al.Depth extraction of single-view image using region merging[J].Video Engineering,2011,35(19):11-13.(in Chinese)
[2] Saxena A,et al.Make3D:Learning 3-D scene structure from a single still image[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2009,31(5):824-840.
[3] Konrad J,Brown G,Wang M,et al.Automatic 2D-to-3D image conversion using 3D examples from the internet[A].Proceedings of SPIE[C].Beningham:Society of Photo-Optical Instrumentation Engineers,2012.82880F.
[4] Bharathi S,Vasuki A.2D-to-3D conversion of images using edge information[A].IJCA Proceedings on International Conference in Recent trends in Computational Methods,Communication and Controls[C].New York:Foundation of Computer Science,2012.27-32.
[5] Zhang L,Vazquez C,Knorr S.3D-TV content creation automatic 2D-to-3D video conversion[J].IEEE Transactions on Broadcasting,2011,57(2):372-383.
[6] 苏庆华,等.单幅散焦图像深度计算方法[J].红外,2013,34(5):16-22. Su Q,et al.Calculation method of depth in single defocused image[J].Infrared,2013,34(5):16-22.(in Chinese)
[7] 史国凯,等.基于分割的离焦图像深度图提取方法[J].液晶与显示,2012,27(2):229-234. Shi G,et al.Method for depth map extraction of defocused image based on segmentation[J].Chinese Journal of Liquid Crystals and Displays,2012,27(2):229-234.(in Chinese)
[8] Zhuo S,Sim T.Defocus map estimation from a single image[J].Pattern Recognition,2011,44(9):1852-1858.
[9] 马吉权,马培军,苏小红.基于镜面高光检测的明暗恢复形状新方法[J].电子学报,2010,5(5):1084-1088. Ma Ji,Ma P,Su X.New method of shape from shading based on specular highlights detection[J].Acta Electronica Sinica,2010,5(5):1084-1088.(in Chinese)
[10] 王学梅,孙即祥.基于高阶LLF和WENO算法的透视SFS[J].中国图象图形学报,2011,16(2):300-304. Wang X,Sun J.Perspective shape from shading based on high-order LLF and WENO[J].Journal of Image and Graphics,2011,16(2):300-304.(in Chinese)
[11] Jung Y J,Baik A,Kim J,et al.A novel 2D-to-3D conversion technique based on relative height depth cue[A].Proceedings of SPIE[C].Beningham:Society of Photo-Optical Instrumentation Engineers,2009.7237-7244.
[12] Battiato S,et al.Depth map generation by image classification[A].Proceedings of SPIE[C].Beningham:Society of Photo-Optical Instrumentation Engineers,2004.95-104.
[13] Cheng C C,Li C T,Chen L G.A 2D-to-3D conversion system using edge information[A].Proceedings of International Conference on Consumer Electronics[C].Los Alamitos:IEEE Computer Society Press,2010.377-378.
[14] Liu B Y,Gould S,Koller D.Single image depth estimation from predicted semantic labels[A].Proceedings of IEEE Conference on Computer Vision and Pattern Recognition[C].Los Alamitos:IEEE Computer Society Press,2010.1253-1260.
[15] 李乐,张茂军,李永乐,等.前景时空域分析和背景理解相结合的2D/3D视频转换方法[J].计算机辅助设计与图形学学报,2011,23(5):899-908. Li L,Zhang M,Li Y,et al.2D/3D video conversion method based on foreground temporal-spatial analyzing and background understanding[J].Journal of Computer-Aided Design & Computer Graphics,2011,23(5):899-908.(in Chinese)
[16] 杨昭,高隽,谢昭,等.局部Gist特征匹配核的场景分类[J].中国图象图形学报,2013,18(3):264-270. Yang Z,Gao J,Xie Z,et al.Scene categorization of local Gist feature match kernel[J].Journal of Image and Graphics,2013,18(3):264-270.(in Chinese)
[17] Liu C,et al.SIFT flow:dense correspondence across scenes and its applications[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2011,33(5):978-994.
[18] Liu C.Beyond pixels:exploring new representations and applications for motion analysis[D].Cambridge:Dept of Electrical Engineering and Computer Science,Massachusetts Institute of Technology,2009.144-146.
[19] Tian D,et al.View syntesis techniques for 3D video[A].Proceedings of SPIE[C].Beningham:Society of Photo-Optical Instrumentation Engineers,2009.74430T-74430T-11.

加权SIFT流深度迁移的单幅图像2D转3D

Single Image 2D-to-3D Conversion via Weighted SIFT Flow

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 3

编辑推荐

Metrics

[1]	刘玉轩, 张力, 艾海滨, 许彪, 孙钰珊, 樊仲藜. 光场相机三维重建研究进展与展望[J]. 电子学报, 2022, 50(7): 1774-1792.
[2]	袁红星, 安鹏, 吴少群, 郑悠. 具有容错能力的L₁最优化半自动2D转3D[J]. 电子学报, 2018, 46(2): 447-455.
[3]	袁红星, 吴少群, 安鹏, 郑悠, 徐力. 对象引导的单幅散焦图像深度提取方法[J]. 电子学报, 2014, 42(10): 2009-2015.