基于运动视觉与视差协同计算的迫近感知神经网络

周浩艇; 房川凯; 刘稔远; 彭济根; 傅沁冰

doi:10.12263/DZXB.20250337

您当前的位置：

首页 >

文章列表页 >

基于运动视觉与视差协同计算的迫近感知神经网络

学术论文 | 更新时间：2025-12-27

- 基于运动视觉与视差协同计算的迫近感知神经网络
- A Bio-Plausible Neural Network Integrating Motion and Disparity Pathways for Looming Perception
- 电子学报 2025年53卷第8期页码：2899-2914
- 作者机构：
  
  1.广州大学数学与信息科学学院，广东广州 510006
  2.广州大学机器生命与智能研究中心，广东广州 510006
- 作者简介：
  
  [ "周浩艇男，2003年11月生于广东省汕头市.广州大学数学与信息科学学院本科生.主要研究方向为运动视觉感知、机器人多模态碰撞检测.E-mail: htzhou@e.gzhu.edu.cn" ]
  [ "房川凯男，2004年1月出生广东省深圳市.广州大学数学与信息科学学院本科生.主要研究方向为碰撞感知和计算机视觉.E-mail: chuankaifang@e.gzhu.edu.cn" ]
  [ "刘稔远男，2003年11月出生于河南省郑州市.广州大学数学与信息科学学院本科生.主要研究方向为神经建模、机器人运动感知和终身学习.E-mail: rliu@e.gzhu.edu.cn" ]
  [ "彭济根男，1967年5月生于江西省上饶市.广州大学数学与信息科学学院教授.主要研究方向为非线性泛函分析及其应用、机器学习理论以及稀疏信息处理.E-mail: jgpeng@gzhu.edu.cn" ]
  [ "傅沁冰男，1987年4月出生于四川省成都市.广州大学数学与信息科学学院副教授、机器生命与智能研究中心副主任.主要研究方向为生物启发式碰撞检测、运动感知、生物启发式机器人以及事件驱动的视觉处理.E-mail: qifu@gzhu.edu.cn|" ]
- 基金信息：
  
  国家自然科学基金(12031003;62376063;12571558)
- DOI：10.12263/DZXB.20250337
  中图分类号： TP183;TP242
- 收稿：2025-04-28，
  
  录用：2025-07-22，
  
  纸质出版：2025-08-25
- 稿件说明：
移动端阅览
周浩艇, 房川凯, 刘稔远, 等. 基于运动视觉与视差协同计算的迫近感知神经网络[J]. 电子学报, 2025, 53(08): 2899-2914.

ZHOU Hao-ting, FANG Chuan-kai, LIU Ren-yuan, et al. A Bio-Plausible Neural Network Integrating Motion and Disparity Pathways for Looming Perception[J]. Acta Electronica Sinica, 2025, 53(08): 2899-2914.
周浩艇, 房川凯, 刘稔远, 等. 基于运动视觉与视差协同计算的迫近感知神经网络[J]. 电子学报, 2025, 53(08): 2899-2914. DOI：10.12263/DZXB.20250337

ZHOU Hao-ting, FANG Chuan-kai, LIU Ren-yuan, et al. A Bio-Plausible Neural Network Integrating Motion and Disparity Pathways for Looming Perception[J]. Acta Electronica Sinica, 2025, 53(08): 2899-2914. DOI：10.12263/DZXB.20250337

摘要

从自然界中动物的正常生存到工业中机器的安全运作，碰撞感知能力始终至关重要.受蝗虫视觉神经元LGMD（Lobula Giant Movement Detector）的启发，许多仿生的计算模型已经被用于实时可靠的碰撞感知.然而，受限于二维单目的输入信号，目前的方法难以捕捉运动目标的深度特征，进而无法满足在复杂的真实动态场景下进行迫近感知的需求.因此，本研究提出一种融合生物似然性运动通路和视差通路的三维迫近感知模型.在突触前神经网络，通过对2种视觉通路从时空维度上进行实时神经信号整合，所提出的模型不仅能够有效排除大范围的背景杂波干扰，而且可以明显抑制前景非迫近运动所产生的视觉刺激，降低了对突然出现在感受野目标的关注度，进一步提高在未知现实环境中对迫近运动的选择.真实场景数据集的离线测试，以及在线机器人测试的实验结果显示，与目前最先进的方法相比，我们的模型在时间复杂度降低了一个数量级的前提下，准确率提升至96.09%，且能够协助移动机器人在自主导航时实时稳健检测，避免潜在的碰撞威胁.研究综合揭示出迫近感知神经网络对于运动通路的高效性以及视差通路的可靠性，具备显著的协同能力.

Abstract

The capacity to perceive collisions is essential

from the survival of animals in nature to the safe operation of machines in industrial environments. Inspired by the locust visual neuron LGMD (Lobula Giant Movement Detector)

numerous biomimetic computational models have been developed for real-time and reliable collision detection. However

constrained by two-dimensional monocular visual input

existing methods struggle to capture the depth features of moving objects

thus failing to meet the demands of looming perception in complex dynamic scenarios. To address this

this study proposes a 3D looming perception model that integrates bio-plausible motion and disparity pathways. In the presynaptic neural network

the proposed model achieves spatiotemporal integration of neural signals from both visual pathways. This not only effectively eliminates background clutter interference but also significantly suppresses visual stimuli caused by non-looming foreground motion

while reducing attention to targets suddenly appearing within the field of vision. Consequently

the model enhances selectivity for approaching objects in unknown realistic environments. The experimental results of offline tests on real scene datasets and online tests on robot validate that our model attains an accuracy of 96.09% while reducing time complexity by an order of magnitude compared with the state-of-art method. Furthermore

it enables mobile robots to detect and avoid potential collisions in real-time during autonomous navigation. The study demonstrates a significantly synergistic fusion of the motion pathway’s efficiency and the disparity pathway’s reliability accomplished by the proposed neural network.

关键词

Keywords

references

SHARKAWY A N , ALI M M . NARX neural network for safe human-robot collaboration using only joint position sensor [J ] . Logistics , 2022 , 6 ( 4 ): 75 .

RAMASAMY S , SABATINI R , GARDI A , et al . LIDAR obstacle warning and avoidance system for unmanned aerial vehicle sense-and-avoid [J ] . Aerospace Science and Technology , 2016 , 55 : 344 - 358 .

WESSENDORP N , DINAUX R , DUPEYROUX J , et al . Obstacle avoidance onboard MAVs using a FMCW radar [C ] // 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems . Piscataway : IEEE , 2021 : 117 - 122 .

FALANGA D , KLEBER K , SCARAMUZZA D . Dynamic obstacle avoidance for quadrotors with event cameras [J ] . Science Robotics , 2020 , 5 ( 40 ): eaaz9712 .

LU L , CARRIO A , SAMPEDRO C , et al . A robust and fast collision-avoidance approach for micro aerial vehicles using a depth sensor [J ] . Remote Sensing , 2021 , 13 ( 9 ): 1796 .

CARRIO A , TORDESILLAS J , VEMPRALA S , et al . Onboard detection and localization of drones using depth maps [J ] . IEEE Access , 2020 , 8 : 30480 - 30490 .

LIN J H , ZHU H , ALONSO-MORA J . Robust vision-based obstacle avoidance for micro aerial vehicles in dynamic environments [C ] // 2020 IEEE International Conference on Robotics and Automation . Piscataway : IEEE , 2020 : 2682 - 2688 .

陈昌川 , 丁锡烽 , 王亦何 , 等 . 基于光线碰撞检测的轻量化实时三维重建系统 [J ] . 电子学报 , 2024 , 52 ( 12 ): 4074 - 4086 .

CHEN C C , DING X F , WANG Y H , et al . Lightweight real-time 3D reconstruction system based on light collision detection [J ] . Acta Electronica Sinica , 2024 , 52 ( 12 ): 4074 - 4086 . (in Chinese)

TIPPETTS B , LEE D J , LILLYWHITE K , et al . Review of stereo vision algorithms and their suitability for resource-limited systems [J ] . Journal of Real-Time Image Processing , 2016 , 11 ( 1 ): 5 - 25 .

周晓清 , 王翔 , 郑锦 , 等 . 基于自适应空间稀疏化的高效多视图立体匹配 [J ] . 电子学报 , 2023 , 51 ( 11 ): 3079 - 3091 .

ZHOU X Q , WANG X , ZHENG J , et al . Adaptive spatial sparsification for efficient multi-view stereo matching [J ] . Acta Electronica Sinica , 2023 , 51 ( 11 ): 3079 - 3091 . (in Chinese)

WANG Y J , JI J L , WANG Q H , et al . Autonomous flights in dynamic environments with onboard vision [C ] // 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . New York : ACM , 2021 : 1966 - 1973 .

LU L , FASANO G , CARRIO A , et al . A comprehensive survey on non-cooperative collision avoidance for micro aerial vehicles: Sensing and obstacle detection [J ] . Journal of Field Robotics , 2023 , 40 ( 6 ): 1697 - 1720 .

WANG D S , LI W , LIU X G , et al . UAV environmental perception and autonomous obstacle avoidance: A deep learning and depth camera combined solution [J ] . Computers and Electronics in Agriculture , 2020 , 175 : 105523 .

SALEH S , MANOHARAN S , NINE J , et al . Towards robust perception depth information for collision avoidance [C ] // 2020 IEEE Congreso Bienal de Argentina . Piscataway : IEEE , 2021 : 1 - 4 .

FU Q B , WANG H X , HU C , et al . Towards computational models and applications of insect visual systems for motion perception: A review [J ] . Artificial Life , 2019 , 25 ( 3 ): 263 - 311 .

BAZAZI S , BUHL C , HALE J J , et al . Collective motion and cannibalism in locust migratory bands [J ] . Current Biology , 2008 , 18 ( 10 ): 735 - 739 .

KRONGAUZ D L , AYALI A , KAMINKA G A . Vision-based collective motion: A locust-inspired reductionist model [J ] . PLoS Computational Biology , 2024 , 20 ( 1 ): e1011796 .

O’SHEA M , WILLIAMS J L D . The anatomy and output connection of a locust visual interneurone; the lobular giant movement detector (LGMD) neurone [J ] . Journal of Comparative Physiology , 1974 , 91 ( 3 ): 257 - 266 .

YAKUBOWSKI J M , MCMILLAN G A , GRAY J R . Background visual motion affects responses of an insect motion-sensitive neuron to objects deviating from a collision course [J ] . Physiological Reports , 2016 , 4 ( 10 ): e12801 .

RIND F C . Non-directional, movement sensitive neurones of the locust optic lobe [J ] . Journal of Comparative Physiology A , 1987 , 161 ( 3 ): 477 - 494 .

SIMMONS P J , RIND F C . Responses to object approach by a wide field visual neurone, the LGMD2 of the locust: Characterization and image cues [J ] . Journal of Comparative Physiology A , 1997 , 180 ( 3 ): 203 - 214 .

KLAPOETKE N C , NERN A , PEEK M Y , et al . Ultra-selective looming detection from radial motion opponency [J ] . Nature , 2017 , 551 ( 7679 ): 237 - 241 .

RIND F C , SIMMONS P J . Seeing what is coming: Building collision-sensitive neurones [J ] . Trends in Neurosciences , 1999 , 22 ( 5 ): 215 - 220 .

YUE S G , RIND F C . Collision detection in complex dynamic scenes using an LGMD-based visual neural network with feature enhancement [J ] . IEEE Transactions on Neural Networks , 2006 , 17 ( 3 ): 705 - 716 .

GU B , FENG J F , SONG Z Y . Looming detection in complex dynamic visual scenes by interneuronal coordination of motion and feature pathways [J ] . Advanced Intelligent Systems , 2024 , 6 ( 9 ): 2470041 .

RIND F C , BRAMWELL D I . Neural network based on the input organization of an identified neuron signaling impending collision [J ] . Journal of Neurophysiology , 1996 , 75 ( 3 ): 967 - 985 .

GABBIANI F , KRAPP H G , KOCH C , et al . Multiplicative computation in a visual neuron sensitive to looming [J ] . Nature , 2002 , 420 ( 6913 ): 320 - 324 .

WHEATSTONE C . On some remarkable and hitherto unobserved phenomena of binocular vision [J ] . The Optometric Weekly , 1962 , 53 : 2311 - 2315 .

SUN X Y , JIANG Y Z , JI Y F , et al . Distance measurement system based on binocular stereo vision [J ] . IOP Conference Series: Earth and Environmental Science , 2019 , 252 ( 5 ): 052051 .

KELLNHOFER P , DIDYK P , RITSCHEL T , et al . Motion parallax in stereo 3D: Model and applications [J ] . ACM Transactions on Graphics , 2016 , 35 ( 6 ): 1 - 12 .

KEMPPAINEN J , SCALES B , HAGHIGHI K R , et al . Binocular mirror-symmetric microsaccadic sampling enables Drosophila hyperacute 3D vision [J ] . Proceedings of the National Academy of Sciences , 2022 , 119 ( 12 ): e2109717119 .

LAUGHLIN S B . The role of sensory adaptation in the retina [J ] . The Journal of Experimental Biology , 1989 , 146 : 39 - 62 .

READ J C A . Stereopsis without correspondence [J ] . Philosophical Transactions of the Royal Society of London. Series B , Biological sciences, 2023 , 378 ( 1869 ): 20210449 .

SOBEL E C . The locust’s use of motion parallax to measure distance [J ] . Journal of Comparative Physiology A , 1990 , 167 ( 5 ): 579 - 588 .

KRAL K , POTESER M . Motion parallax as a source of distance information in locusts and mantids [J ] . Journal of Insect Behavior , 1997 , 10 ( 1 ): 145 - 163 .

RIND F C . Recent advances in insect vision in a 3D world: Looming stimuli and escape behaviour [J ] . Current Opinion in Insect Science , 2024 , 63 : 101180 .

SILVA A , SANTOS C . Computational model of the LGMD neuron for automatic collision detection [C ] // 2013 IEEE 3rd Portuguese Meeting in Bioengineering . Piscataway : IEEE , 2013 : 1 - 4 .

FU Q B . Motion perception based on ON/OFF channels: A survey [J ] . Neural Networks , 2023 , 165 : 1 - 18 .

FU Q B , HU C , PENG J G , et al . Shaping the collision selectivity in a looming sensitive neuron model with parallel ON and OFF pathways and spike frequency adaptation [J ] . Neural Networks , 2018 , 106 : 127 - 143 .

FU Q B , HU C , PENG J G , et al . A robust collision perception visual neural network with specific selectivity to darker objects [J ] . IEEE Transactions on Cybernetics , 2020 , 50 ( 12 ): 5074 - 5088 .

HUA M , FU Q B , PENG J G , et al . Shaping the ultra-selectivity of a looming detection neural network from non-linear correlation of radial motion [C ] // 2022 International Joint Conference on Neural Networks . Piscataway : IEEE , 2022 : 1 - 8 .

FRANCESCHINI N . Small brains, smart machines: From fly vision to robot vision and back again [J ] . Proceedings of the IEEE , 2014 , 102 ( 5 ): 751 - 781 .

LEI F , PENG Z P , LIU M , et al . A robust visual system for looming cue detection against translating motion [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2023 , 34 ( 11 ): 8362 - 8376 .

LI L , ZHANG Z H , LU J X . Artificial fly visual joint perception neural network inspired by multiple-regional collision detection [J ] . Neural Networks , 2021 , 135 : 13 - 28 .

NITYANANDA V , BISSIANNA G , TARAWNEH G , et al . Small or far away? Size and distance perception in the praying mantis [J ] . Philosophical Transactions: Biological Sciences , 371 ( 1697 ): 20150262 .

ROSNER R , VON HADELN J , TARAWNEH G , et al . A neuronal correlate of insect stereopsis [J ] . Nature Communications , 2019 , 10 ( 1 ): 2845 .

ROSNER R , TARAWNEH G , LUKYANOVA V , et al . Binocular responsiveness of projection neurons of the praying mantis optic lobe in the frontal visual field [J ] . Journal of Comparative Physiology A , 2020 , 206 ( 2 ): 165 - 181 .

O’KEEFFE J , YAP S H , LLAMAS-CORNEJO I , et al . A computational model of stereoscopic prey capture in praying mantises [J ] . PLoS Computational Biology , 2022 , 18 ( 5 ): e1009666 .

ZHENG Y , WANG Y S , WU G R , et al . Enhancing LGMD-based model for collision prediction via binocular structure [J ] . Frontiers in Neuroscience , 2023 , 17 : 1247227 .

ZHAO J Y , XI S K , LI Y , et al . A fly inspired solution to looming detection for collision avoidance [J ] . iScience , 2023 , 26 ( 4 ): 106337 .

FU Q B , YUE S G . Modelling Drosophila motion vision pathways for decoding the direction of translating objects against cluttered moving backgrounds [J ] . Biological Cybernetics , 2020 , 114 ( 4 ): 443 - 460 .

FU Q B , YUE S G . Complementary visual neuronal systems model for collision sensing [C ] // 2020 5th International Conference on Advanced Robotics and Mechatronics . Piscataway : IEEE , 2020 : 609 - 615 .

ČÍŽEK P , FAIGL J . Self-supervised learning of the biologically-inspired obstacle avoidance of hexapod walking robot [J ] . Bioinspiration & Biomimetics , 2019 , 14 ( 4 ): 046002 .

ZHAO J N , WANG H X , BELLOTTO N , et al . Enhancing LGMD’s looming selectivity for UAV with spatial-temporal distributed presynaptic connections [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2023 , 34 ( 5 ): 2539 - 2553 .

ZHAO J N , XIE Q S , SHUANG F , et al . An angular acceleration based looming detector for moving UAVs [J ] . Biomimetics , 2024 , 9 ( 1 ): 22 .

SALT L , HOWARD D , INDIVERI G , et al . Parameter optimization and learning in a spiking neural network for UAV obstacle avoidance targeting neuromorphic processors [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2020 , 31 ( 9 ): 3305 - 3318 .

PETRÁČEK P , WALTER V , BÁČA T , et al . Bio-inspired compact swarms of unmanned aerial vehicles without communication and external localization [J ] . Bioinspiration & Biomimetics , 2020 , 16 ( 2 ): 026009 .

SANTOS M C P , SANTANA L V , BRANDÃO A S , et al . UAV obstacle avoidance using RGB-D system [C ] // 2015 International Conference on Unmanned Aircraft Systems . Piscataway : IEEE , 2015 : 312 - 319 .

VIENNE C , PLANTIER J , NEVEU P , et al . (Disparity-driven) Accommodation response contributes to perceived depth [J ] . Frontiers in Neuroscience , 2018 , 12 : 973 .

PIETRZIK S , CHANDRASEKARAN B . Testing autonomous path planning algorithms and setup for robotic vehicle navigation [C ] // 2018 9th IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference . Piscataway : IEEE , 2018 : 485 - 488 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于双目立体视觉的偏转线圈测量方法研究

基于图割的图像匹配算法

基于相位的尺度自适应立体匹配方法