National Natural Science Foundation of China (No.61866026, No.61772255, No.61866025);Superior Science and Technology Innovation Team Project of Jiangxi Province (No.20152BCB24004, No.20165BCB19007);Youth Science Fund of Jiangxi Province (No.20171BAB212012);China Postdoctoral Science Foundation (No.2019M650894)
In order to address the issue of motion boundary blurring caused by the complex scenes
large displacement and motion occlusion
this paper proposes a binocular scene flow estimation method based on semantic segmentation. Firstly
by using the image semantic information
we classify the image regions into several categories with semantic labels through convolutional neural networks. Then we plan the motion models of various image regions according to the different semantic categories and compute the optical flow and disparity under the prior knowledge of semantic information. Secondly
we apply the superpixel segmentation to the input image and couple the optical flow and disparity information via least squares method to solve the motion parameters of each superpixel patch. Finally
we add the boundary information of semantic segmentation constraint to the optimization energy function
and estimate the scene flow by updating the mappings of pixels-to-superpixel and superpixel-to-plane. We evaluate the proposed approach and some state-of-the-art methods on the KITTI 2015 database to conduct a comparison experiment. The experimental results demonstrate that our method has high accuracy and good robustness
and especially has significant benefit of boundary preserving in the areas of complex scene