基于GFU和分层LSTM的组群行为识别研究方法

doi:10.3969/j.issn.0372-2112.2020.08.002

电子学报 ›› 2020, Vol. 48 ›› Issue (8): 1465-1471.DOI: 10.3969/j.issn.0372-2112.2020.08.002

基于GFU和分层LSTM的组群行为识别研究方法

王传旭, 薛豪

青岛科技大学信息科学技术学院, 山东青岛 266001

收稿日期:2019-09-19 修回日期:2020-03-31 出版日期:2020-08-25 发布日期:2020-08-25
作者简介:王传旭 男,1968年1月出生,山东邹城人.教授、硕士生导师.1990年、2000年和2007年分别在石油大学(华东)、石油大学(北京)工业自动化和中国海洋大学获应用电子技术学士、硕士学位和博士学位.主要从事计算机视觉方面的有关研究. E-mail:Wangchuanxu_qd@163.com;薛豪 男,1994年1月出生,山东临沂人.2016年毕业于潍坊学院信息与控制工程学院,取得学士学位,现为青岛科技大学信息学院在读硕士研究生,从事计算机视觉方面的有关研究. E-mail:xuehao1130@yeah.net
基金资助:
国家自然科学基金（No.61672305）

Group Activity Recognition Based on GFU and Hierarchical LSTM

WANG Chuan-xu, XUE Hao

Institute of Information Science and Technology, Qingdao University of Science and Technology, Qingdao, Shandong 266001, China

Received:2019-09-19 Revised:2020-03-31 Online:2020-08-25 Published:2020-08-25

摘要/Abstract

摘要： 提出一种以"关键人物"为核心，使用门控融合单元（GFU，Gated Fusion Unit）进行特征融合的组群行为识别框架，旨在解决两个问题：①组群行为信息冗余，重点关注关键人物行为特征，忽略无关人员对组群行为的影响；②组群内部交互行为复杂，使用GFU有效融合以关键人物为核心的交互特征，再通过LSTM时序建模成为表征能力更强的组群特征.最终，通过softmax分类器进行组群行为类别分类.该算法在排球数据集上取得了86.7%的平均识别率.

关键词: 组群行为识别, 关键人物建模, 交互特征建模, 门控融合单元

Abstract: This paper proposes a group behavior recognition framework with "key persons" as the core and Gated Fusing Unit (GFU)for feature fusion.Its aim is to solve the following two problems :1)Group behavior information is redundant,focusing on key person behavior characteristics,ignoring the influence of unrelated personels on group behavior.2)The internal interaction relationship is complex within group,GFU is used to effectively model interaction feature centered around the key characters and it is temporally evolved into the group characteristics via LSTM processing.Finally,the group behavior category is classified with Softmax.The algorithm achieves an average recognition rate of 86.7% on the volleyball dataset.

Key words: group behavior recognition, key person modeling, interaction feature modeling, gated fusion unit

中图分类号:

TP301.6

王传旭, 薛豪. 基于GFU和分层LSTM的组群行为识别研究方法[J]. 电子学报, 2020, 48(8): 1465-1471.

WANG Chuan-xu, XUE Hao. Group Activity Recognition Based on GFU and Hierarchical LSTM[J]. Acta Electronica Sinica, 2020, 48(8): 1465-1471.

参考文献

[1] Ibrahim M S,Muralidharan S,Deng Z,et al.A hierarchical deep temporal model for group activity recognition[A].Proceedings of CVPR[C].USA:IEEE,2016.1971-1980.
[2] Jain A,Zamir A R,Savarese S,et al.Structural-rnn:Deep learning on spatio-temporal graphs[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2016.5308-5317.
[3] Deng Z,Vahdat A,Hu H,et al.Structure inference machines:recurrent neural networks for analyzing relations in group activity recognition[A].Proceedings of CVPR[C].USA:IEEE,2016.4772-4781.
[4] Shu T,Todorovic S,Zhu S C.CERN:Confidence-energy recurrent network for group activity recognition[A].Proceedings of CVPR[C].USA:IEEE,2017.4255-4263.
[5] Biswas S,Gall J.Structural recurrent neural network (srnn)for group activity analysis[A]. IEEE Winter Conference on Applications of Computer Vision (WACV)[C].USA:IEEE,2018.1625-1632.
[6] Gammulle H,Denman S,Sridharan S,et al.Multi-Level Sequence GAN for Group Activity Recognition[A].Asian Conference on Computer Vision[C].Cham:Springer,2018.331-346.
[7] SHI Lei,ZHANG Yifan,CHENG Jian,LU Hanqing.Two-stream adaptive graph convolutional networks for skeleton-based action recognition[A].Proceedings of CVPR[C].USA:IEEE,2019.12026-12035.
[8] Wu J,Wang L,Wang L,et al.Learning actor relation graphs for group activity recognition[A].Proceedings of CVPR[C].USA:IEEE,2019.9964-9974.
[9] Ibrahim MS,Mori G.Hierarchical relational networks for group activity recognition and retrieval[A].European Conference on Computer Vision[C].USA:ECCV,2018.721-736.
[10] Yan R,Tang J,Shu X,et al.Participation-contributed temporal dynamic model for group activity recognition[A].ACM Multimedia Conference[C].USA:ACM,2018.1292-1300.
[11] Kong L,Qin J,Huang D,et al.Hierarchical attention and context modeling for group activity recognition[A].Proceedings of ICASSP[C].USA:IEEE,2018.1328-1332.
[12] Ramanathan V,Huang J,Abu-El-Haija S,et al.Detecting events and key actors in multi-person videos[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2016.3043-3053.
[13] 桑海峰,王传正,吕应宇,何大阔,刘晴.基于多信息流动卷积神经网络的行人再识别[J].电子学报,2019,47(2):351-357. SANG Hai-feng,WANG Chuan-zheng,LÜ Ying-yu,HE Da-kuo,LIU Qing.Person re-identification based on multi-information flow convolutional neural network[J].Acta Electronica Sinica,2019,47(2):351-357.(in Chinese)
[14] Danelljan M,Khan F,Felsberg M,et al.Accurate scale estimation for robust visual tracking[A].British Machine Vision Conference,Nottingham[C].[S.L.]:BMVA,2014.1-5.
[15] Russakovsky O,Deng J,Su H,et al.Imagenet large scale visual recognition challenge[J].International Journal of Computer Vision,2015,115(3):211-52.
[16] He K,Zhang X,Ren S,et al.Deep residual learning for image recognition[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2016.770-778.
[17] 吕品,李全刚,柳厅文,宁振虎,王玉斌,时金桥,方滨兴.基于双向LSTM的误植域名滥用检测方法[J].电子学报,2018,46(9):2081-2086. LÜ Pin,LI Quan-gang,LIU Ting-wen,NING Zhen-hu,WANG Yu-bin,SHI Jin-qiao,FANG Bin-xing.Towards typo squatting abuse detection using bi-directional LSTM[J].Acta Electronica Sinica,2018,46(9):2081-2086.(in Chinese)
[18] Brox T,Bruhn A,Weickert J,et al.High accuracy optical flow estimation based on a theory for warping[A].European Conference on Computer Vision[C].Berlin:Springer-Verlag,2004.25-36.
[19] 罗会兰,王婵娟.行为识别中一种基于融合特征的改进VLAD编码方法[J].电子学报,2019,47(1):49-58. LUO Hui-lan,WANG Chan-juan.An improved VLAD coding method based on fusion feature in action recognition[J].Acta Electronica Sinica,2019,47(1):49-58.(in Chinese)
[20] Arevalo J,Solorio T,et al.Gated Multimodal Units for Information Fusion[M].London:ICLR,2017.0941-0643.
[21] Kingma D P,Ba J.Adam:A method for stochastic optimization[A].International Conference on Learning Representations (ICLR)[C].Ithaca,NY:arXiv.org,2014.13.

基于GFU和分层LSTM的组群行为识别研究方法

Group Activity Recognition Based on GFU and Hierarchical LSTM

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 1

编辑推荐

Metrics