基于遮挡优化的金字塔块匹配光流估计方法
Pyramid Patch-Matching Optical Flow Estimation Method Based on Occlusion Optimization
- 电子学报 2023年51卷第9期 页码：2539-2548
- 作者机构：
1.南昌航空大学无损检测教育部重点实验室，江西南昌 330063
2.南昌航空大学信息工程学院，江西南昌 330063
- 作者简介：
[ "张聪炫 男，1984年7月出生于河南省焦作市.博士， 教授. 主要研究方向图像处理与计算机视觉.中国电子学会会员编号：E190014621M. E-mail: zcxdsg@163.com" ]
[ "史世栋 男，1996年6月出生于山西省晋中市.硕士研究生. 主要研究方向为图像检测与智能识别. E-mail: 1367251730@qq.com" ]
[ "葛利跃 男，1992年10月出生于安徽省蚌埠市.硕士， 助理实验师. 主要研究方向为图像处理与计算机视觉. E-mail: lygeah@163.com" ]
[ "陈 震（通讯作者） 男，1969年11月出生于江西省九江市.博士， 教授. 主要研究方向图像理解与测量." ]
[ "黎 明 男，1965年2月出生于江西省樟树市.博士， 教授. 主要研究方向为图像处理、模式识别、高维多目标优化问题等． E-mail: liming@nchu.edu" ]
- 基金信息：国家重点研发计划(2020YFC2003800);国家自然科学基金(61866026;61866025);江西省优势科技创新团队计划(20165BCB19007);江西省技术创新引导类计划项目(20212AEI91005);江西省自然科学基金重点项目(20202ACB214007);航空科学基金(2018ZC56008);中国博士后科学基金(2019M650894);江西省教育厅科学技术研究项目(GJJ210910);江西省图像处理与模式识别重点实验室开放基金资助(ET202104413)
DOI：10.12263/DZXB.20211100
中图分类号： TP391收稿：2021-08-16，
修回：2022-02-18，
纸质出版：2023-09-25
- 稿件说明：
移动端阅览
张聪炫,史世栋,葛利跃等.基于遮挡优化的金字塔块匹配光流估计方法[J].电子学报,2023,51(09):2539-2548.
ZHANG Cong-xuan,SHI Shi-dong,GE Li-yue,et al.Pyramid Patch-Matching Optical Flow Estimation Method Based on Occlusion Optimization[J].ACTA ELECTRONICA SINICA,2023,51(09):2539-2548.
张聪炫,史世栋,葛利跃等.基于遮挡优化的金字塔块匹配光流估计方法[J].电子学报,2023,51(09):2539-2548. DOI： 10.12263/DZXB.20211100.
ZHANG Cong-xuan,SHI Shi-dong,GE Li-yue,et al.Pyramid Patch-Matching Optical Flow Estimation Method Based on Occlusion Optimization[J].ACTA ELECTRONICA SINICA,2023,51(09):2539-2548. DOI： 10.12263/DZXB.20211100.
摘要
针对现有光流计算方法在大位移和运动遮挡等场景下光流估计的准确性与鲁棒性问题，提出一种基于遮挡优化的金字塔块匹配光流估计方法.首先利用金字塔块匹配模型得到初始匹配运动场，构造遮挡检测模型优化匹配运动场，去除初始匹配运动场包含的误匹配点以获取鲁棒稀疏运动场；然后设计边缘约束的鲁棒插值算法获取稠密光流场，并构建全局能量泛函解决局部最优问题，得到最终的鲁棒稠密光流场.最后分别利用Middlebury、MPI-Sintel和KITTI数据集提供的测试图像序列对本文方法和具有代表性的变分光流估计方法、匹配插值光流估计方法、卷积神经网络光流估计方法进行对比分析.实验结果表明本文方法对大位移和运动遮挡场景光流估计的准确性与鲁棒性具有明显的提升效果.
Abstract
To improve the accuracy and robustness of optical flow estimation under large displacements and motion occlusions
this paper proposes a pyramid patch-matching optical flow estimation method based on occlusion optimization. First
the pyramid patch matching method is adopted to obtain the initial matching motion field
and the occlusion detection model is designed to optimize the matching motion field by removing the false matching points from the initial matching motion field. Second
a robust interpolation scheme with edge-preserving is planned to achieve the dense optical flow field. Third
the global energy function is projected to gain the global optimization optical flow. Finally
the Middlebury
MPI-Sintel and KITTI test datasets are employed to conduct a comprehensive comparison between the proposed method and the representative variation
patch matching and CNN-based optical flow methods. The experimental results show that the proposed method effectively improves the accuracy and robustness of optical flow estimation under large displacements and motion occlusions.
关键词
Keywords
references
XIE Q , REMIL O , GUO Y W , et al . Object detection and tracking under occlusion for object-level RGB-D video segmentation [J]. IEEE Transactions on Multimedia , 2018 , 20 ( 3 ): 580 - 592 .
吴晓军 , 鞠光亮 . 一种无标记点人脸表情捕捉与重现算法 [J]. 电子学报 , 2016 , 44 ( 9 ): 2141 - 2147 .
WU X J , JU G L . A markerless facial expression capture and reproduce algorithm [J]. Acta Electronica Sinica , 2016 , 44 ( 9 ): 2141 - 2147 . (in Chinese)
KE R M , LI Z B , TANG J J , et al . Real-time traffic flow parameter estimation from UAV video based on ensemble classifier and optical flow [J]. IEEE Transactions on Intelligent Transportation Systems , 2019 , 20 ( 1 ): 54 - 64 .
TU Z G , XIE W , ZHANG D J , et al . A survey of variational and CNN-based optical flow techniques [J]. Signal Processing: Image Communication , 2019 , 72 : 9 - 24 .
HUR J , ROTH S . Joint optical flow and temporally consistent semantic segmentation [C]// European Conference on Computer Vision . Cham : Springer , 2016 : 163 - 177 .
BAILER C , TAETZ B , STRICKER D . Flow fields: Dense correspondence fields for highly accurate large displacement optical flow estimation [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2019 , 41 ( 8 ): 1879 - 1892 .
葛利跃 , 张聪炫 , 陈震 , 等 . 相互结构引导滤波TV-L1变分光流估计 [J]. 电子学报 , 2019 , 47 ( 3 ): 707 - 713 .
GE L Y , ZHANG C X , CHEN Z , et al . Mutual-structure guided filtering based TV-L1 optical flow estimation [J]. Acta Electronica Sinica , 2019 , 47 ( 3 ): 707 - 713 . (in Chinese)
SUN D Q , ROTH S , BLACK M J . A quantitative analysis of current practices in optical flow estimation and the principles behind them [J]. International Journal of Computer Vision , 2014 , 106 ( 2 ): 115 - 137 .
DOSOVITSKIY A , FISCHER P , ILG E , et al . FlowNet: Learning optical flow with convolutional networks [C]// 2015 IEEE International Conference on Computer Vision . Piscataway : IEEE , 2015 : 2758 - 2766 .
SUN D Q , YANG X D , LIU M Y , et al . PWC-net: CNNs for optical flow using pyramid, warping, and cost volume [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE, : 8934 - 8943 .
HUI T W , LOY C C . LiteFlowNet3: Resolving correspondence ambiguity for more accurate optical flow estimation [C]// European Conference on Computer Vision . Cham : Springer , 2020 : 169 - 184 .
HUR J , ROTH S . Iterative residual refinement for joint optical flow and occlusion estimation [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2019 : 5747 - 5756 .
REVAUD J , WEINZAEPFEL P , HARCHAOUI Z , et al . EpicFlow: Edge-preserving interpolation of correspondences for optical flow [C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2015 : 1164 - 1172 .
HU Y L , SONG R , LI Y S . Efficient coarse-to-fine patch match for large displacement optical flow [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2016 : 5704 - 5712 .
ZHANG C X , CHEN Z , XIONG F , et al . STDC-Flow: Large displacement flow field estimation using similarity transformation-based dense correspondence [J]. IET Computer Vision , 2020 , 14 ( 5 ): 248 - 258 .
HU Y L , LI Y S , SONG R . Robust interpolation of correspondences for large displacement optical flow [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2017 : 4791 - 4799 .
HE K , ZHAO Y , LIU Z G , et al . Whole-pixel registration of non-rigid images using correspondences interpolation on sparse feature seeds [J]. The Visual Computer , 2022 , 38 ( 5 ): 1815 - 1832 .
HE J Z , ZHANG S L , YANG M , et al . Bi-directional cascade network for perceptual edge detection [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2019 : 3823 - 3832 .
WANNENWETSCH A S , ROTH S . Probabilistic pixel-adaptive refinement networks [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2020 : 11639 - 11648 .
TANG C Z , YUAN L , TAN P . LSM: Learning subspace minimization for low-level vision [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2020 : 6234 - 6245 .
0
浏览量
10
下载量
2
CSCD
- 网络PDF下载
- Meta-XML下载
- BibTeX下载
- Endnote下载
- RefMan 下载
- NoteExpress下载
- NoteFirst下载
关联资源
相关文章
相关作者
相关机构
- 技术支持由北京北大方正电子有限公司提供 京ICP备09064830号-19
京公网安备11010802024621 - 本系统建议在Chrome、 IE9+ 以上版本浏览器阅读本站内容，360浏览器请切换至极速模式
- Cookies帮助我们提供服务并提供个性化体验。使用本网站，即表示您同意我们使用Cookies