Pyramid Patch-Matching Optical Flow Estimation Method Based on Occlusion Optimization

ZHANG Cong-xuan; SHI Shi-dong; GE Li-yue; CHEN Zhen; LI Ming

doi:10.12263/DZXB.20211100

您当前的位置：

首页 >

文章列表页 >

Pyramid Patch-Matching Optical Flow Estimation Method Based on Occlusion Optimization

PAPERS | 更新时间：2025-12-08

- Pyramid Patch-Matching Optical Flow Estimation Method Based on Occlusion Optimization
- ACTA ELECTRONICA SINICA Vol. 51, Issue 9, Pages: 2539-2548(2023)
- 作者机构：
  
  1.南昌航空大学无损检测教育部重点实验室，江西南昌 330063
  2.南昌航空大学信息工程学院，江西南昌 330063
- 作者简介：
- 基金信息：
- DOI：10.12263/DZXB.20211100
  CLC： TP391
- Received：16 August 2021，
  
  Revised：2022-02-18，
  
  Published：25 September 2023
- 稿件说明：
移动端阅览
张聪炫,史世栋,葛利跃等.基于遮挡优化的金字塔块匹配光流估计方法[J].电子学报,2023,51(09):2539-2548.

ZHANG Cong-xuan,SHI Shi-dong,GE Li-yue,et al.Pyramid Patch-Matching Optical Flow Estimation Method Based on Occlusion Optimization[J].ACTA ELECTRONICA SINICA,2023,51(09):2539-2548.
张聪炫,史世栋,葛利跃等.基于遮挡优化的金字塔块匹配光流估计方法[J].电子学报,2023,51(09):2539-2548. DOI： 10.12263/DZXB.20211100.

ZHANG Cong-xuan,SHI Shi-dong,GE Li-yue,et al.Pyramid Patch-Matching Optical Flow Estimation Method Based on Occlusion Optimization[J].ACTA ELECTRONICA SINICA,2023,51(09):2539-2548. DOI： 10.12263/DZXB.20211100.

摘要

针对现有光流计算方法在大位移和运动遮挡等场景下光流估计的准确性与鲁棒性问题，提出一种基于遮挡优化的金字塔块匹配光流估计方法.首先利用金字塔块匹配模型得到初始匹配运动场，构造遮挡检测模型优化匹配运动场，去除初始匹配运动场包含的误匹配点以获取鲁棒稀疏运动场；然后设计边缘约束的鲁棒插值算法获取稠密光流场，并构建全局能量泛函解决局部最优问题，得到最终的鲁棒稠密光流场.最后分别利用Middlebury、MPI-Sintel和KITTI数据集提供的测试图像序列对本文方法和具有代表性的变分光流估计方法、匹配插值光流估计方法、卷积神经网络光流估计方法进行对比分析.实验结果表明本文方法对大位移和运动遮挡场景光流估计的准确性与鲁棒性具有明显的提升效果.

Abstract

To improve the accuracy and robustness of optical flow estimation under large displacements and motion occlusions

this paper proposes a pyramid patch-matching optical flow estimation method based on occlusion optimization. First

the pyramid patch matching method is adopted to obtain the initial matching motion field

and the occlusion detection model is designed to optimize the matching motion field by removing the false matching points from the initial matching motion field. Second

a robust interpolation scheme with edge-preserving is planned to achieve the dense optical flow field. Third

the global energy function is projected to gain the global optimization optical flow. Finally

the Middlebury

MPI-Sintel and KITTI test datasets are employed to conduct a comprehensive comparison between the proposed method and the representative variation

patch matching and CNN-based optical flow methods. The experimental results show that the proposed method effectively improves the accuracy and robustness of optical flow estimation under large displacements and motion occlusions.

关键词

Keywords

references

XIE Q , REMIL O , GUO Y W , et al . Object detection and tracking under occlusion for object-level RGB-D video segmentation [J]. IEEE Transactions on Multimedia , 2018 , 20 ( 3 ): 580 - 592 .

吴晓军 , 鞠光亮 . 一种无标记点人脸表情捕捉与重现算法 [J]. 电子学报 , 2016 , 44 ( 9 ): 2141 - 2147 .

WU X J , JU G L . A markerless facial expression capture and reproduce algorithm [J]. Acta Electronica Sinica , 2016 , 44 ( 9 ): 2141 - 2147 . (in Chinese)

KE R M , LI Z B , TANG J J , et al . Real-time traffic flow parameter estimation from UAV video based on ensemble classifier and optical flow [J]. IEEE Transactions on Intelligent Transportation Systems , 2019 , 20 ( 1 ): 54 - 64 .

TU Z G , XIE W , ZHANG D J , et al . A survey of variational and CNN-based optical flow techniques [J]. Signal Processing: Image Communication , 2019 , 72 : 9 - 24 .

HUR J , ROTH S . Joint optical flow and temporally consistent semantic segmentation [C]// European Conference on Computer Vision . Cham : Springer , 2016 : 163 - 177 .

BAILER C , TAETZ B , STRICKER D . Flow fields: Dense correspondence fields for highly accurate large displacement optical flow estimation [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2019 , 41 ( 8 ): 1879 - 1892 .

葛利跃 , 张聪炫 , 陈震 , 等 . 相互结构引导滤波TV-L1变分光流估计 [J]. 电子学报 , 2019 , 47 ( 3 ): 707 - 713 .

GE L Y , ZHANG C X , CHEN Z , et al . Mutual-structure guided filtering based TV-L1 optical flow estimation [J]. Acta Electronica Sinica , 2019 , 47 ( 3 ): 707 - 713 . (in Chinese)

SUN D Q , ROTH S , BLACK M J . A quantitative analysis of current practices in optical flow estimation and the principles behind them [J]. International Journal of Computer Vision , 2014 , 106 ( 2 ): 115 - 137 .

DOSOVITSKIY A , FISCHER P , ILG E , et al . FlowNet: Learning optical flow with convolutional networks [C]// 2015 IEEE International Conference on Computer Vision . Piscataway : IEEE , 2015 : 2758 - 2766 .

SUN D Q , YANG X D , LIU M Y , et al . PWC-net: CNNs for optical flow using pyramid, warping, and cost volume [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE, : 8934 - 8943 .

HUI T W , LOY C C . LiteFlowNet3: Resolving correspondence ambiguity for more accurate optical flow estimation [C]// European Conference on Computer Vision . Cham : Springer , 2020 : 169 - 184 .

HUR J , ROTH S . Iterative residual refinement for joint optical flow and occlusion estimation [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2019 : 5747 - 5756 .

REVAUD J , WEINZAEPFEL P , HARCHAOUI Z , et al . EpicFlow: Edge-preserving interpolation of correspondences for optical flow [C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2015 : 1164 - 1172 .

HU Y L , SONG R , LI Y S . Efficient coarse-to-fine patch match for large displacement optical flow [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2016 : 5704 - 5712 .

ZHANG C X , CHEN Z , XIONG F , et al . STDC-Flow: Large displacement flow field estimation using similarity transformation-based dense correspondence [J]. IET Computer Vision , 2020 , 14 ( 5 ): 248 - 258 .

HU Y L , LI Y S , SONG R . Robust interpolation of correspondences for large displacement optical flow [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2017 : 4791 - 4799 .

HE K , ZHAO Y , LIU Z G , et al . Whole-pixel registration of non-rigid images using correspondences interpolation on sparse feature seeds [J]. The Visual Computer , 2022 , 38 ( 5 ): 1815 - 1832 .

HE J Z , ZHANG S L , YANG M , et al . Bi-directional cascade network for perceptual edge detection [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2019 : 3823 - 3832 .

WANNENWETSCH A S , ROTH S . Probabilistic pixel-adaptive refinement networks [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2020 : 11639 - 11648 .

TANG C Z , YUAN L , TAN P . LSM: Learning subspace minimization for low-level vision [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2020 : 6234 - 6245 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

MAP-Based Multi-Information Flow Gradient Update and Aggregation for Video Compressed Sensing Reconstruction

On-Line Recognition of Actions in Daily Living

Dynamic Programming in Segments for Occlusion Detection/Stereo Matching

Phase Matching and Occlusion Detection Based on Dynamic Programming

Adaptive Response Kernel Correlation Target Tracking Method Guided by Motion Model

Related Author

YANG Xin

YANG Chun-ling

XU Guang-you

TAO Lin-mi

HUANG Fei-yue

CAO Yuan-yuan

CUI Ling-ling

JIAO Wei-dong

Related Institution

School of Electronic and Information Engineering， South China University of Technology

Key Laboratory of Pervasive Computing of Ministry of Education, Department of ComputerScience and Technology, Tsinghua University

Tencent Research

Key Laboratory of Pervasive Computing of Ministry of Education Department of ComputerScience and Technology Tsinghua University Beijing

Tencent Research Beijing

⁰