电子学报 ›› 2020, Vol. 48 ›› Issue (9): 1841-1849.DOI: 10.3969/j.issn.0372-2112.2020.09.023

• 综述评论 • 上一篇    下一篇

深度学习光流计算技术研究进展

张聪炫1,2, 周仲凯1, 陈震1, 葛利跃1, 黎明1, 江少锋1, 陈昊1   

  1. 1. 南昌航空大学无损检测技术教育部重点实验室, 江西南昌 330063;
    2. 中国科学院自动化研究所, 北京 100190
  • 收稿日期:2019-08-26 修回日期:2019-12-23 出版日期:2020-09-25 发布日期:2020-09-25
  • 通讯作者: 陈震
  • 作者简介:张聪炫 男,1984年7月出生于河南省焦作市.分别于2007和2014年在南昌航空大学和南京航空航天大学获得学士和博士学位.现为南昌航空大学副教授,硕士生导师,中国科学院自动化研究所博士后.主要研究方向为图像检测与智能识别.E-mail:zcxdsg@163.com
    周仲凯 男,1994年3月出生于江苏省常州市.现为南昌航空大学测试与光电工程学院硕士研究生.主要研究方向为图像检测与智能识别.E-mail:jsczzzk123@163.com
  • 基金资助:
    国家自然科学基金(No.61866026,No.61772255,No.61866025);江西省优势科技创新团队计划(No.20165BCB19007);江西省科技创新杰出青年人才计划(No.20192BCB23011);航空科学基金(No.2018ZC56008);中国博士后科学基金(No.2019M650894);江西省重点研发计划(No.20171BBG70052);江西省研究生创新专项资金项目资助(No.YC2018049)

Research Progress of Deep Learning Based Optical Flow Computation Technology

ZHANG Cong-xuan1,2, ZHOU Zhong-kai1, CHEN Zhen1, GE Li-yue1, LI Ming1, JIANG Shao-feng1, CHEN Hao1   

  1. 1. Key Laboratory of Nondestructive Testing, Ministry of Education, Nanchang Hangkong University, Nanchang, Jiangxi 330063, China;
    2. Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
  • Received:2019-08-26 Revised:2019-12-23 Online:2020-09-25 Published:2020-09-25

摘要: 图像序列光流计算是图像处理与计算机视觉等领域的重要研究方向.随着深度学习技术的快速发展,以卷积神经网络为代表的深度学习理论与方法成为光流计算技术研究的热点.本文主要对深度学习光流计算技术研究进行综述,首先介绍了有监督学习、无监督学习和半监督学习的光流计算网络模型与训练策略,然后重点阐述并分析了不同网络模型优化方法.针对光流计算模型的评估问题,分别介绍了Middlebury、MPI-Sintel和KITTI等数据库及评价基准,并对不同类型深度学习和传统变分光流模型进行对比与分析.最后,总结了深度学习光流计算技术在模型复杂度与泛化性、光流估计鲁棒性、小样本训练准确性等方面的关键技术问题,并指出了可能的解决方案与研究思路.

关键词: 光流计算, 深度学习, 卷积神经网络, 训练策略, 优化方法, 评价基准

Abstract: Optical flow computation is an important research direction in image processing and computer vision.With the rapid development of the deep learning technology,the convolutional neural network based deep learning theories and methodologies have been the research focus of optical flow computation.This article mainly reviews the research progress of the deep learning based optical flow estimation technologies.First,the typical models and training strategies of the optical flow computing networks with supervised learning,unsupervised learning and semi-supervised learning are introduced.Second,the optimization methods of various network models are described and analyzed.Third,the evaluation benchmarks of Middlebury,MPI-Sintel and KITTI databases are summarized,and the experimental comparison results and analysis between the different deep-learning and variational optical flow methods are conducted.Finally,we discuss some issues of the deep learning based optical flow computation technology including the model complexity and generalization,the robustness of optical flow estimation and the accuracy of the small sample training.Afterwards,we point out several possible solutions and research ideas to address the above mentioned issues.

Key words: optical flow computation, deep learning, convolutional neural network, training strategy, optimization method, evaluation benchmark

中图分类号: