一种基于帧图像的动态视觉传感器样本集建模方法

doi:10.3969/j.issn.0372-2112.2020.08.001

电子学报 ›› 2020, Vol. 48 ›› Issue (8): 1457-1464.DOI: 10.3969/j.issn.0372-2112.2020.08.001

• 学术论文 • 下一篇

一种基于帧图像的动态视觉传感器样本集建模方法

陆兴鹏^1,4, 王明羽^1,2, 曹扬³, 赵瑞莲⁴, 周武³, 李兆麟¹, 魏少军²

1. 清华大学计算机科学与技术系, 北京 100084;
2. 清华大学微电子学研究所, 北京 100084;
3. 北京航天晨信科技有限公司, 北京 102308;
4. 北京化工大学信息科学与技术学院, 北京 100029

收稿日期:2019-04-17 修回日期:2019-12-02 出版日期:2020-08-25 发布日期:2020-08-25
通讯作者: 王明羽
作者简介:陆兴鹏 男,安徽无为人.现为北京化工大学信息科学与技术学院硕士研究生,2018年4月至2020年1月于清华大学计算机科学与技术系从事实习工作,主要研究方向为图像处理、模式识别、深度学习等. E-mail:luxingpeng@foxmail.com;曹扬 女,辽宁营口人.现工作于北京航天晨信科技有限责任公司战略发展研究室,主要从事指挥信息系统智能化技术研究工作;周武 男,上海人.现任北京航天晨信科技有限公司副总经理,中国航天科工集团公司大数据与人工智能专业组副组长,指挥信息系统及装备智能化技术专家;李兆麟 男,黑龙江大庆人.清华大学计算机科学与技术系教授,清华大学移动计算研究中心副主任,电子元器件技术专家,十三五科技创新基础研究领域专家,主要研究方向为计算机体系结构、多核/众核处理器、人工智能芯片、嵌入式系统等.
基金资助:
中国博士后科学基金（No.2019M650030）；清华大学自主科研计划（No.2018Z05JDX005）

A Dynamic Vision Sensor Sample Set Modeling Method Based on Frame Images

LU Xing-peng^1,4, WANG Ming-yu^1,2, CAO Yang³, ZHAO Rui-lian⁴, ZHOU Wu³, LI Zhao-lin¹, WEI Shao-jun²

1. Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China;
2. Institute of Microelectronics, Tsinghua University, Beijing 100084, China;
3. Beijing Aerospace Chenxin Science and Technology Ltd., Beijing 102308, China;
4. School of Information Science and Technology, Beijing University of Chemical Technology, Beijing 100029, China

Received:2019-04-17 Revised:2019-12-02 Online:2020-08-25 Published:2020-08-25

摘要/Abstract

摘要： 动态视觉传感器（Dynamic Vision Sensor，DVS）利用事件驱动原理实现运动目标的快速提取，具有低延迟、低存储空间和高动态范围等优势.目前研究表明，基于DVS的神经网络在目标检测等领域具有明显的速度优势.但是，这类神经网络在训练时所需要的样本集主要依赖DVS相机产生，缺少高效的样本集生成方法，这制约了这类神经网络的应用与发展.本文根据DVS原理，提出了一种基于帧图像的DVS建模以及样本集建模方法.该方法设定每个像素单元独立工作，经过动态差分和逻辑判断后输出触发的地址-事件数据，这些数据通过编码和归一化生成神经网络训练时所需要的样本集.通过对MNIST和CIFAR-10样本集建模的实验结果表明，该建模方法效果与DVS相机基本一致；与基于帧图像的存储方式相比，该样本集可以明显减少存储空间.该方法所生成样本集已经通过卷积神经网络训练和测试验证.

关键词: 动态视觉传感器, 样本集建模, 事件驱动, 存储优化

Abstract: Dynamic vision sensor (DVS)shows significant advantages on low computational latency,low memory usage and high dynamic range by utilizing the event-driven principle to extract features from moving objects.Current research shows that DVS-based neural networks improve object detection speed obviously.However,the sample sets required by such neural networks mainly rely on specific DVS cameras while lacking efficient generation methods for the sample sets.It limits the application and development of those neural networks.According to the principle of DVS,this paper presents a DVS sample set modeling method based on frame images,in which the sample set can be generated by encoding and normalizing the address-event (AE)data after being trigged by dynamic differential comparisons and logical judgments.The experimental results for modeling the MNIST and CIFAR-10 sample sets show that,the sample set modeled by the proposed method is basically matched with the real DVS cameras.Compared with traditional frame image sample sets,this method can significantly reduce the memory usage.The sample set generated by the proposed modeling method has also been verified by training and testing a typical convolutional neural network.

Key words: dynamic vision sensor, sample set modeling, event-driven, memory optimization

中图分类号:

TP391

陆兴鹏, 王明羽, 曹扬, 赵瑞莲, 周武, 李兆麟, 魏少军. 一种基于帧图像的动态视觉传感器样本集建模方法[J]. 电子学报, 2020, 48(8): 1457-1464.

LU Xing-peng, WANG Ming-yu, CAO Yang, ZHAO Rui-lian, ZHOU Wu, LI Zhao-lin, WEI Shao-jun. A Dynamic Vision Sensor Sample Set Modeling Method Based on Frame Images[J]. Acta Electronica Sinica, 2020, 48(8): 1457-1464.

参考文献

[1] SZEGEDY C,LIU W,JIA Y,et al.Going deeper with convolutional[A].Proceeding of the 2015 IEEE Conference on Computer Vision and Pattern Recognition[C].Boston:IEEE,2015.1-9.
[2] HE K,ZHANG X,REN S,et al.Deep residual learning for image recognition[A].Proceeding of the 2016 IEEE Conference on Computer Vision and Pattern Recognition[C].Las Vegas:IEEE,2016.770-778.
[3] REDMON J,DIVVALA S,GIRSHICK R,et al.You only look once:unified,real-time object detection[A].Proceeding of the 2016 IEEE Conference on Computer Vision and Pattern Recognition[C].Las Vegas:IEEE,2016.779-788.
[4] LIU W,ANGUELOV D,ERHAN D,et al.SSD:single shot multibox detector[A].Proceeding of the 2016 European Conference on Computer Vision[C].Amsterdam:ECCV,2016.21-37.
[5] DELBRUCK T.Frame-free dynamic digital vision[A].Proceeding of Intl Symp on Source-Life Electronics,Advanced Electronics for Quality Life and Society[C].Tokyo:IEEE,2008.21-26.
[6] LUNGU L-A,CORRADI F,DELBRUCK T.Live demonstration:convolutional neural network driven by dynamic vision sensor playing RoShamBo[A].Proceeding of the 2017 IEEE International Symposium on Circuits and Systems (ISCAS)[C].Baltimore:IEEE,2017.1-1.
[7] MOEYS D P,CORRADI F,KERR E,et al.Steering a predator robot using a mixed frame/event-driven convolutional neural network[A].Proceeding of the 2016 Second International Conference on Event-based Control,Communication,and Signal Processing (EBCCSP)[C].Krakow:IEEE,2016.1-8.
[8] SIVILOTTI M A.Wiring considerations in Analog VLSI systems,with application to field-programmable networks[D].California USA:California Institute of Technology Pasadena,1991.
[9] MAHOWALD M.VLSI analogs of neuronal visual processing:a synthesis of form and function[D].California USA:California Institute of Technology Pasadena,1992.
[10] YANG M,LIU S-C,DELBRUCK T.A dynamic vision sensor with 1% temporal contrast sensitivity and in-pixel asynchronous delta modulator for event encoding[J].IEEE Journal of Solid-State Circuits,2015,50(9):2149-2160.
[11] BRANDLI C,MULLER L,DELBRUCK T.Real-time,high-speed video decompression using a frame-and event-based DAVIS sensor[A].Proceeding of the 2014 IEEE International Symposium on Circuits and Systems (ISCAS)[C].Melbourne:IEEE,2014.686-689.
[12] TERESA S-G,BERNABE L-B.Poker-DVS and MNIST-DVS.Their history,how they were made,and other details[J].Frontiers in Neuroscience,2015,9:481.
[13] LI H,LIU H,JI X,et al.CIFAR-10:an event-stream dataset for object classification[J].Frontiers in Neuroscience,2017,11:309.
[14] KRAMER J.An integrated optical transient sensor[J].IEEE Transactions on Circuits and Systems Ⅱ:Analog and Digital Signal Processing,2002,49(9):612-628.
[15] 隋晓红,高勇,赵阶喜,等.用于视觉假体的新型高动态范围图像传感器像素单元电路设计[J].电子学报,2011,39(8):1800-1804. SUI Xiao-hong,GAO Yong,ZHAO Jie-xi,et al.Circuit design of high-dynamic-range image sensor pixel for visual prostheses[J].Acta Electronica Sinica,2011,39(8):1800-1804.(in Chinese)
[16] WU Y,YANG M-H,LIM J.Online object tracking:a benchmark[A].Proceeding of the 2013 IEEE Conference on Computer Vision and Pattern Recognition[C].Portland:IEEE,2013.2411-2418.
[17] RUKUNDO O,CAO H.Nearest neighbor value interpolation[J].International Journal of Advanced Computer Science and Applications.2012,3(4):1-6.
[18] 胡正平,陈俊岭.多层融合深度局部PCA子空间稀疏优化特征提取模型[J].电子学报,2017,45(10):2383-2389. HU Zheng-ping,CHEN Jun-ling.Feature extraction model based on multi-layered deep local subspace optimization[J].Acta Electronica Sinica,2017,45(10):2383-2389.(in Chinese)
[19] JIA Y,SHELHAMER E,DONAHUE J,et al.Caffe:Convolutional architecture for fast feature embedding[A].Proceeding of the 22nd ACM International Conference on Multimedia[C].ACM,2014.675-678.
[20] LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-based learning applied to document recognition[J].Proceeding of the IEEE,1998,86(11):2278-2324.
[21] 杨勃,邵泉铭,李文杉,等.深度卷积神经网络鉴别正交特征生成及其应用[J].电子学报,2018,46(10):2376-2383. YANG Bo,SHAO Quan-ming,LI Wen-bin,et al.Deep convolutional neural networks controlled by discriminatively orthogonal feature generation and its application[J].Acta Electronica Sinica,2018,46(10):2376-2383.(in Chinese)

一种基于帧图像的动态视觉传感器样本集建模方法

A Dynamic Vision Sensor Sample Set Modeling Method Based on Frame Images

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 4

编辑推荐

Metrics

[1]	徐频捷, 陈逸杰, 李之南, 赵地. 基于事件驱动的车道线识别算法研究[J]. 电子学报, 2021, 49(7): 1379-1385.
[2]	高毅, 李云骥, 彭力. 离散时滞随机系统的一种均方有界事件驱动控制策略[J]. 电子学报, 2020, 48(3): 524-530.
[3]	蔺想红;张田文. 指数突触电导IF神经元模型及事件驱动模拟策略[J]. 电子学报, 2008, 36(8): 1495-1501.
[4]	龚海刚;刘明;余昌远;王晓敏;陈力军. 无线传感器网络环境下基于事件驱动应用的节能TDMA协议[J]. 电子学报, 2007, 35(10): 1843-1848.