基于多流架构与长短时记忆网络的组群行为识别方法研究

doi:10.3969/j.issn.0372-2112.2020.04.022

电子学报 ›› 2020, Vol. 48 ›› Issue (4): 800-807.DOI: 10.3969/j.issn.0372-2112.2020.04.022

基于多流架构与长短时记忆网络的组群行为识别方法研究

王传旭, 胡小悦, 孟唯佳, 闫春娟

青岛科技大学信息科学技术学院, 山东青岛 266001

收稿日期:2019-07-15 修回日期:2019-12-17 出版日期:2020-04-25 发布日期:2020-04-25
作者简介:王传旭 男,1968年1月出生,山东邹城人.教授、硕士生导师.1990年、2000年和2007年分别在石油大学(华东)、石油大学(北京)工业自动化和中国海洋大学获应用电子技术学士、硕士学位和博士学位.主要从事计算机视觉方面的有关研究.E-mail:Wangchuanxu_qd@163.com;胡小悦 女,1993年8月出生,山东济南人.2016年毕业于青岛工学院信息学院,取得学士学位,现为青岛科技大学信息学院在读硕士研究生,从事计算机视觉方面的有关研究.E-mail:1783910733@qq.com;孟唯佳 女,1995年4月出生,山东济南人.2017年毕业于山东师范大学应用化学系,取得学士学位,现为青岛科技大学信息学院在读硕士研究生,从事计算机视觉方面的有关研究.E-mail:386345026@qq.com;闫春娟 女,1969年1月出生,山东莱阳人.1991年毕业于中国石油大学物理勘探系,取得学士学位,2000年至今在青岛科技大学信息学院通信教研室工作,从事信息与通信系统方面的有关研究.E-mail:qdyancj@163.com
基金资助:
国家自然科学基金（No.61672305）

Research on Group Behavior Recognition Method Based on Multi-Stream Architecture and Long Short-Term Memory Network

WANG Chuan-xu, HU Xiao-yue, MENG Wei-jia, YAN Chun-juan

Institute of Information Science and Technology, Qingdao University of Science and Technology, Qingdao, Shandong 266001, China

Received:2019-07-15 Revised:2019-12-17 Online:2020-04-25 Published:2020-04-25

摘要/Abstract

摘要： 提出一种基于多流架构与长短时记忆网络的上下文建模框架，旨在克服组群行为识别的两个难点，其一对复杂场景中多视觉线索进行信息融合；其二对情景人物进行建模，以获得长视频上下文关系.并且，对基于全局信息和基于局部信息的识别结果进行决策融合，判定最终组群行为属性.该算法在CAD1和CAD2上分别取得93.2%和95.7%平均识别率.

关键词: 组群行为识别, 多视觉线索融合, 交互上下文建模, 全局-局部模型, 长短时记忆网络

Abstract: This paper proposes a context modeling framework based on multi-stream architecture and LSTM,which aims to overcome two difficulties for group behavior recognition.One is to fuse information from multiple visual cues in complex scenes,the other is to model situational characters to get the long-term temporal context in the video.In addition,decision fusion is performed on the behavior recognition results based on global information and local information to determine the final group behavior attributes.The algorithm achieved 93.2% and 95.7% average recognition rates on CAD1 and CAD2 respectively.

Key words: group behavior recognition, fusion of multiple visual cues, interactive context modeling, global-local model, long short-term memory network

中图分类号:

TP301.6

王传旭, 胡小悦, 孟唯佳, 闫春娟. 基于多流架构与长短时记忆网络的组群行为识别方法研究[J]. 电子学报, 2020, 48(4): 800-807.

WANG Chuan-xu, HU Xiao-yue, MENG Wei-jia, YAN Chun-juan. Research on Group Behavior Recognition Method Based on Multi-Stream Architecture and Long Short-Term Memory Network[J]. Acta Electronica Sinica, 2020, 48(4): 800-807.

参考文献

[1] Carreira J,Zisserman A.Quo Vadis,action recognition? A new model and the kinetics dataset[A].Proceedings of CVPR[C].USA:IEEE,2017.6299-6308.
[2] Sudhakaran S,Escalera S,Lanz O.LSTA:Long short-term attention for egocentric action recognition[A].Proceedings of CVPR[C].USA:IEEE,2019.9954-9963.
[3] YAN An,WANG Yali,LI Zhifeng,QIAO Yu.PA3D:Pose-Action 3D machine for video recognition[A].Proceedings of CVPR[C].USA:IEEE,2019.7922-7931.
[4] SHI Lei,ZHANG Yifan,CHENG Jian,LU Hanqing.Two-stream adaptive graph convolutional networks for skeleton-based action recognition[A].Proceedings of CVPR[C].USA:IEEE,2019.12026-12035.
[5] Yuan Y,Wang D,Wang Q.Memory-augmented temporal dynamic learning for action recognition[A].The Thirty-Third AAAI Conference on Artificial Intelligence[C].AAAI,2019.9167-9175.
[6] 田国会,尹建芹,闫云章,李国栋.基于混合高斯模型和主成分分析的轨迹分析行为识别方法[J].电子学报,2016,44(1):143-149. TIAN Guo-hui,YIN Jian-qin,YAN Yun-zhang,LI Guo-dong.Gaussian mixture models and principal component analysis based human trajectory behavior recognition[J].Acta Electronica Sinica,2016,44(1):143-149.(in Chinese)
[7] DING Wenwen,LIU Kai,XU Biao,CHENG Fei.Skeleton-based human action recognition via screw matrices[J].Chinese Journal of Electronics,2017,26(4):790-796.
[8] 罗会兰,王婵娟.行为识别中一种基于融合特征的改进VLAD编码方法[J].电子学报,2019,47(1):49-58. LUO Hui-lan,WANG Chan-juan.An improved VLAD coding method based on fusion feature in action recognition[J].Acta Electronica Sinica,2019,47(1):49-58.(in Chinese)
[9] 郑兴华,孙喜庆,吕嘉欣,鲜征征,李磊.基于深度学习和智能规划的行为识别[J].电子学报,2019,47(8):1661-1668. ZHENG Xing-hua,SUN Xi-qing,LU Jia-xin,XIAN Zheng-zheng,LI Lei.Action recognition based on deep learning and artificial intelligence planning[J].Acta Electronica Sinica,2019,47(8):1661-1668.(in Chinese)
[10] Shu T,Todorovic S,Zhu S C.CERN:Confidence-energy recurrent network for group activity recognition[A].Proceedings of CVPR[C].USA:IEEE,2017.4255-4263.
[11] Ibrahim M S,Muralidharan S,Deng Z,et al.A hierarchical deep temporal model for group activity recognition[A].Proceedings of CVPR[C].USA:IEEE,2016.1971-1980.
[12] Yan R,Tang J,Shu X,et al.Participation-contributed temporal dynamic model for group activity recognition[A].2018 ACM Multimedia Conference[C].USA:ACM,2018.1292-1300.
[13] Simonyan K,Zisserman A.Two-stream convolutional networks for action recognition in videos[A].NIPS 2014[C].NIPS,2014.568-576.
[14] Wang L,Xiong Y,Wang Z,et al.Temporal segment networks:towards good practices for deep action recognition[A].European Conference on Computer Vision[C].ECCV,2016.20-36.
[15] Zhu Y,Lan Z,Newsam S,et al.Hidden two-stream convolutional networks for action recognition[A].Asian Conference on Computer Vision[C].ACCV,2018.363-378.
[16] Ramanathan V,Huang J,Abu-El-Haija S,et al.Detecting events and key actors in multi-person videos[A].Proceedings of CVPR[C].USA:IEEE,2016.3043-3053.
[17] Qi M,Qin J,Li A,et al.stagNet:an attentive semantic rnn for group activity recognition[A].European Conference on Computer Vision[C].ECCV,2018.101-118.
[18] Deng Z,Vahdat A,Hu H,et al.Structure inference machines:recurrent neural networks for analyzing relations in group activity recognition[A].Proceedings of CVPR[C].USA:IEEE,2016.4772-4781.
[19] Wang M,Ni B,Yang X.Recurrent modeling of interaction context for collective activity recognition[A].Proceedings of CVPR[C].USA:IEEE,2017.3048-3056.
[20] Choi W,Shahid K,Savarese S.What are they doing?:Collective activity classification using spatio-temporal relationship among people[A].IEEE 12th International Conference on Computer Vision Workshops,ICCV Workshops[C].USA:IEEE,2009.1282-1289.
[21] Kong L,Qin J,Huang D,et al.Hierarchical attention and context modeling for group activity recognition[A].Proceedings of ICASSP[C].USA:IEEE,2018.1328-1332.
[22] Tang Y,Zhang P,Hu J F,et al.Latent embeddings for collective activity recognition[A].Proceedings of the 14th IEEE International Conference on Advanced Video and Signal based Surveillance[C].USA:IEEE,2017.1-6.
[23] 豆贺贺.基于视频特征的多人行为识别研究[D].南京:南京邮电大学,2016.1-61.

基于多流架构与长短时记忆网络的组群行为识别方法研究

Research on Group Behavior Recognition Method Based on Multi-Stream Architecture and Long Short-Term Memory Network

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 3

编辑推荐

Metrics

[1]	王传旭, 薛豪. 基于GFU和分层LSTM的组群行为识别研究方法[J]. 电子学报, 2020, 48(8): 1465-1471.
[2]	李鹏伟, 姜宇谦, 薛飞扬, 黄佳佳, 徐超. 一种基于深度学习的强对抗性Android恶意代码检测方法[J]. 电子学报, 2020, 48(8): 1502-1508.
[3]	田生伟, 秦越, 禹龙, 吐尔根·依布拉音, 冯冠军. 基于Bi-LSTM的维吾尔语人称代词指代消解[J]. 电子学报, 2018, 46(7): 1691-1699.