电子学报 ›› 2021, Vol. 49 ›› Issue (4): 690-695.DOI: 10.12263/DZXB.20190135

• 学术论文 • 上一篇    下一篇

基于特征级联卷积网络的双目立体匹配

吴俊劼1, 陈震1, 张聪炫1,2, 江少锋1, 尚璇1   

  1. 1. 南昌航空大学无损检测技术教育部重点实验室, 江西南昌 330063;
    2. 中国科学院自动化研究所, 北京 100190
  • 收稿日期:2019-01-23 修回日期:2019-08-29 出版日期:2021-04-25 发布日期:2021-04-25
  • 通讯作者: 张聪炫
  • 作者简介:吴俊劼 男,1995年8月出生于江苏南京.现为南昌航空大学信息工程学院硕士研究生.主要研究方向为图像检测与智能识别.E-mail:junjiewu1023@163.com;陈震 男,1969年11月生于江西九江.分别于1993、2000和2003年在西北工业大学获得学士、硕士和博士学位.现为南昌航空大学教授,博士生导师.主要研究方向为计算机视觉、图像处理与模式识别.E-mail:dr_chenzhen@163.com
  • 基金资助:
    国家自然科学基金(No.61866026,No.61772255,No.61866025);江西省优势科技创新团队计划(No.20152BCB24004,No.20165BCB19007);江西省科技创新杰出青年人才计划(No.20192BCB23011);航空科学基金(No.2018ZC56008);江西省青年科学基金重点项目(No.20202ACB214007);中国博士后科学基金(No.2019M650894);江西省研究生创新专项资金(No.YC2018-S368)

Binocular Stereo Matching Based on Feature Cascade Convolutional Network

WU Jun-jie1, CHEN Zhen1, ZHANG Cong-xuan1,2, JIANG Shao-feng1, SHANG Xuan1   

  1. 1. Key Laboratory of Nondestructive Testing, Ministry of Education, Nanchang Hangkong University, Nanchang, Jiangxi 330063, China;
    2. Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
  • Received:2019-01-23 Revised:2019-08-29 Online:2021-04-25 Published:2021-04-25

摘要: 针对图像序列病态区域匹配歧义性以及稠密视差图连通性的问题,本文提出一种基于特征级联卷积神经网络的双目立体匹配计算方法.构造特征重用的全卷积密集块,利用"跳连接"机制将浅层提取的特征图级联到后续子层,对深层卷积丢失的局部特征信息进行补偿.引入指示函数划分一定大小的训练集,将其批量输入特征级联卷积网络模型进行前向传播,同时通过小批量梯度下降(Mini-Batch Gradient Descent,MBGD)策略更新初始权重和偏置参数.根据负连接神经元对网络模型的输出进行初始匹配代价计算,并利用十字交叉域代价聚合(Cross Based Cost Aggregation,CBCA)和半全局立体匹配(Semi-Global Matching,SGM)等算法对代价函数进行优化,求得精准稠密的视差图.分别采用Middlebury数据库提供的训练和测试立体图像集对本文方法和深度学习方法MC-CNN、CBMV、MC-CNN-WS等具有代表性方法进行对比测试.实验结果表明,本文方法具有较高的视差计算精度和鲁棒性,尤其对复杂场景、光照变化以及弱纹理等困难场景图像序列能有效提高匹配率和保持图像细节.

关键词: 图像序列, 稠密视差图, 双目立体匹配, 卷积神经网络, 全卷积密集块, 匹配代价, 前向传播

Abstract: In order to overcome the ambiguity of ill-posed regions matching while enhancing the connectivity of dense disparity map,this paper proposes a binocular stereo matching method based on feature cascade convolutional neural network.We constructed a fully convolutional densely block with feature reuse to utilize the "skip-connection" mechanism to transmit the feature maps extracted from the previous layers to all subsequent layers,and compensated the local feature information of deep convolution losing.At the forward propagation stage,we designed an indicator function to divide a certain size of the training set as the input of the feature cascade convolutional network model,and applied the Mini-Batch Gradient Descent (MBGD) strategy to update the initial weight and bias parameters.We computed the initial matching cost according to the output of the presented network model,and used the Cross Based Cost Aggregation (CBCA) and Semi-Global Matching (SGM) pipeline to optimize the cost function for generating the accurate and dense disparity map.We used the published training and test sets of Middlebury database to evaluate our approach and other representative stereo matching methods,such as MC-CNN,CBMV and MC-CNN-WS.The experimental results prove that the proposed method has higher accuracy and better robustness of disparity estimation,and especially improves the matching rate and preserves image details in the regions of complex scene,illumination change and weakly textured.

Key words: image sequence, dense disparity map, binocular stereo matching, convolutional neural network, fully convolutional densely block, matching cost, forward propagation

中图分类号: