基于伪3D残差网络与交互关系建模的群组行为识别方法

doi:10.3969/j.issn.0372-2112.2020.07.004

电子学报 ›› 2020, Vol. 48 ›› Issue (7): 1269-1275.DOI: 10.3969/j.issn.0372-2112.2020.07.004

基于伪3D残差网络与交互关系建模的群组行为识别方法

丰艳, 张甜甜, 王传旭

青岛科技大学信息科学技术学院, 山东青岛 266061

收稿日期:2019-08-13 修回日期:2020-04-06 出版日期:2020-07-25 发布日期:2020-07-25
作者简介:丰艳女,1977年10月出生,山东曲阜人.现为青岛科技大学副教授,硕士生导师.主要从事虚拟现实、计算机视觉方面的研究.E-mail:fywmh@163.com;张甜甜 女,1993年3月出生,山东烟台人.2017年毕业于齐鲁工业大学信息学院,取得计算机科学与技术专业学士学位,现为青岛科技大学信息学院在读硕士研究生,从事计算机视觉方面的有关研究.E-mail:zhangtt0424@163.com;王传旭 男,1968年1月出生,山东邹城人.教授、硕士生导师.1990年、2000年和2007年分别在中国石油大学(华东)、中国石油大学(北京)和中国海洋大学获应用电子技术学士、硕士和博士学位.主要从事计算机视觉方面的有关研究.E-mail:Wangchuanxu_qd@163.com
基金资助:
国家自然科学基金面上项目（No.61672305）；国家自然科学基金青年科学基金（No.61702295）

Group Activity Recognition Method Based on Pseudo 3D Residual Network and Interaction Modeling

FENG Yan, ZHANG Tian-tian, WANG Chuan-xu

School of Information Science & Technology, Qingdao University of Science and Technology, Qingdao, Shandong 266061, China

Received:2019-08-13 Revised:2020-04-06 Online:2020-07-25 Published:2020-07-25

摘要/Abstract

摘要： 针对复杂场景下群组行为特征的多样性以及交互关系难以建模的问题，提出一种全新的分层网络架构.第一层网络，利用伪3D残差网络与图卷积网络相结合捕获交互关系特征；第二层网络，利用伪3D残差网络捕获群组全局场景时空特征.根据上述特征之间的互补作用对它们的群组行为决策输出，提出一种权重自适应调整决策融合算法，对上面两层网络的群组行为类别自适应计算重要性权重，实现决策融合.该方法在CAD和CAE上分别取得了91.4%和97.9%的平均识别精度.

关键词: 群组行为识别, 交互关系建模, 自适应决策融合

Abstract: For the diversity of group behavior characteristics in complex scenes and the problem of difficult interaction modeling,this paper proposes a new two-layered network architecture.The first layer of network combines a pseudo 3D residual network with a graph convolution network to capture the interaction characteristics.The second layer of network,uses the pseudo 3D residual network to capture the group global scene spatio-temporal characteristics.Based on the complementary role of the above features,their group behavior decisions are fused with a weight adaptive adjustment algorithm,which adaptively calculates importance weights for the group behavior categories predicted by the above two channels,and realizes decision fusion of the different prediction results.The method has achieved 91.4% and 97.9% average recognition accuracy on CAD and CAE respectively.

Key words: group behavior recognition, interaction modeling, adaptive decision fusion

中图分类号:

TP391

丰艳, 张甜甜, 王传旭. 基于伪3D残差网络与交互关系建模的群组行为识别方法[J]. 电子学报, 2020, 48(7): 1269-1275.

FENG Yan, ZHANG Tian-tian, WANG Chuan-xu. Group Activity Recognition Method Based on Pseudo 3D Residual Network and Interaction Modeling[J]. Acta Electronica Sinica, 2020, 48(7): 1269-1275.

参考文献

[1] 韩磊,李君峰,贾云得.基于时空单词的两人交互行为识别方法[J].计算机学报,2010,33(4):776-784. HAN Lei,LI Jun-feng,JIA Yun-de.Human interaction recognition using spatio-temporal words[J].Chinese Journal of Computers,2010,33(4):776-784.(in Chinese)
[2] 朱煜,赵江坤,王逸宁,郑兵兵.基于深度学习的人体行为识别算法综述[J].自动化学报,2016,42(6):848-857. ZHU Yu,ZHAO Jiang-kun,WANG Yi-ning,ZHENG Bing-bing.A review of human action recognition based on deep learning[J].Acta Automatica Sinica,2016,42(6):848-857.(in Chinese)
[3] 郑兴华,孙喜庆,吕嘉欣,等.基于深度学习和智能规划的行为识别[J].电子学报,2019,47(8):1661-1668. ZHENG Xing-hua,SUN Xi-qing,LU Jia-xin,et al.Action recognition based on deep learning and artificial intelligence planning[J].Acta Electronica Sinica,2019,47(8):1661-1668.(in Chinese)
[4] 王传旭,刘云,厉万庆.基于时空特征点的非监督姿态建模和行为识别的算法研究[J].电子学报,2011,39(8):1751-1756. WANG Chuan-xu,LIU Yun,LI Wan-qing.Research ofunsupervised posture modeling and action recognition based on spatial-temporal interesting points[J].Acta Electronica Sinica,2011,39(8):1751-1756.(in Chinese)
[5] Deng Z,Vahdat A,Hu H,et al.Structure inference machines:Recurrent neural networks for analyzing relations in group activity recognition[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2016.4772-4781.
[6] Tran D,Bourdev L,Fergus R,et al.Learning spatiotemporal features with 3d convolutional networks[A].Proceedings of the IEEE International Conference on Computer Vision[C]. USA:IEEE,2015.4489-4497.
[7] Simonyan K,Zisserman A.Two-stream convolutional networks for action recognition in videos[A].Advances in Neural Information Processing Systems[C].USA:Massachusetts Institute of Technology Press,2014.568-576.
[8] Ibrahim M S,Muralidharan S,Deng Z,et al.A hierarchical deep temporal model for group activity recognition[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2016.1971-1980.
[9] Vahora S,Chauhan N.Deep neural network model for group activity recognition using contextual relationship[J].Engineering Science and Technology,an International Journal,2019,22(1):47-54.
[10] Ramanathan V,Huang J,Abu-El-Haija S,et al.Detecting events and key actors in multi-person videos[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2016.3043-3053.
[11] Li W,Chang M-C,Lyu S.Who did what at where and when:simultaneous multi-person tracking and activity recognition[J].arXiv Preprint,2018,arXiv:1807.01253.
[12] Deng Z,Zhai M,Chen L,et al.Deep structured models for group activity recognition[J].arXiv Preprint,2015,arXiv:1506.04191.
[13] Wu J,Wang L,Wang L,et al.Learning actor relation graphs for group activity recognition[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2019.9964-9974.
[14] 罗会兰,王婵娟.行为识别中一种基于融合特征的改进VLAD编码方法[J].电子学报,2019,47(1):49-58. LUO Hui-lan,WANG Chan-juan.An improved VLAD coding method based on fusion feature in action recognition[J].Acta Electronica Sinica,2019,47(1):49-58.(in Chinese)
[15] 田国会,尹建芹,闫云章,李国栋.基于混合高斯模型和主成分分析的轨迹分析行为识别方法[J].电子学报,2016,44(1):143-149. TIAN Guo-hui,YIN Jian-qin,YAN Yun-zhang,LI Guo-dong.Gaussian mixture models and principal component analysis based human trajectory behavior recognition[J].Acta Electronica Sinica,2016,44(1):143-149.(in Chinese)
[16] Cao Z,Simon T,Wei S-E,et al.Realtime multi-person 2d pose estimation using part affinity fields[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2017.7291-7299.
[17] Qiu Z,Yao T,Mei T.Learning spatio-temporal representation with pseudo-3d residual networks[A].Proceedings of the IEEE International Conference on Computer Vision[C].USA:IEEE,2017.5533-5541.
[18] Choi W,Shahid K,Savarese S.What are they doing?:Collective activity classification using spatio-temporal relationship among people[A].IEEE 12th International Conference on Computer Vision(ICCV) Workshops[C].USA:IEEE,2009.1282-1289.
[19] Choi W,Shahid K,Savarese S.Learning context for collective activity recognition[A].Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern RecognitionJune(CVPR'11)[C].USA:IEEE,2011.3273-3280.
[20] Li X,Choo Chuah M.SBGAR:semantics based group activity recognition[A].Proceedings of the IEEE International Conference on Computer Vision[C].USA:IEEE,2017.2876-2885.
[21] Lan T,Wang Y,Yang W,et al.Discriminative latent models for recognizing contextual group activities[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2011,34(8):1549-1562.
[22] Choi W,Savarese S.Understanding collective activitiesof people from videos[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,36(6):1242-1257.
[23] Amer M R,Lei P,Todorovic S.Hirf:Hierarchical random field for collective activity recognition in videos[A].European Conference on Computer Vision[C].Cham:Springer,2014.572-585.

基于伪3D残差网络与交互关系建模的群组行为识别方法

Group Activity Recognition Method Based on Pseudo 3D Residual Network and Interaction Modeling

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	周登文, 李文斌, 李金新, 黄志勇. 一种轻量级的多尺度通道注意图像超分辨率重建网络[J]. 电子学报, 2022, (): 1-12.
[2]	孔玮, 刘云, 李辉, 崔雪红, 杨浩冉. 基于全局自适应有向图的行人轨迹预测[J]. 电子学报, 2022, (): 1-12.
[3]	肖斌, 陈嘉博, 毕秀丽, 张俊辉, 李伟生, 王国胤, 马旭. 基于一维卷积神经网络与循环神经网络串联的心音分析方法[J]. 电子学报, 2022, (): 1-8.
[4]	麻文刚, 张亚东, 郭进. 基于景深先验引导与环境光优化的图像去雾[J]. 电子学报, 2022, (): 1-14.
[5]	李超, 黄新宇, 王凯. 基于特征融合和自学习锚框的高分辨率图像小目标检测算法[J]. 电子学报, 2022, (): 1-12.
[6]	姜奇, 文悦, 张瑞杰, 魏福山, 马建峰. 面向智能手机的自适应触屏持续认证方案[J]. 电子学报, 2022, 50(5): 1131-1139.
[7]	卓亚琦, 魏家辉, 李志欣. 基于双注意模型的图像描述生成方法研究[J]. 电子学报, 2022, 50(5): 1123-1130.
[8]	胡振涛, 杨诗博, 胡玉梅, 周林, 金勇, 杨琳琳. 基于变分贝叶斯的分布式融合目标跟踪[J]. 电子学报, 2022, 50(5): 1058-1065.
[9]	王相海, 赵晓阳, 王鑫莹, 赵克云, 宋传鸣. 非抽取小波边缘学习深度残差网络的单幅图像超分辨率重建[J]. 电子学报, 2022, (): 1-13.
[10]	严莉, 张凯, 徐浩, 韩圣亚, 刘珅岐, 史玉良. 基于图注意力机制和Transformer的异常检测[J]. 电子学报, 2022, 50(4): 900-908.
[11]	庾骏, 黄伟, 张晓波, 尹贺峰. 基于松弛Hadamard矩阵的多模态融合哈希方法[J]. 电子学报, 2022, 50(4): 909-920.
[12]	袁冠, 邴睿, 刘肖, 代伟, 张艳梅, 蔡卓. 基于时空图神经网络的手势识别[J]. 电子学报, 2022, 50(4): 921-931.
[13]	张涛, 张文涛, 代凌, 陈婧怡, 王丽, 魏倩茹. 基于序贯博弈多智能体强化学习的综合模块化航空电子系统重构方法[J]. 电子学报, 2022, 50(4): 954-966.
[14]	刘微容, 米彦春, 杨帆, 张彦, 郭宏林, 刘仲民. 基于多级解码网络的图像修复[J]. 电子学报, 2022, 50(3): 625-636.
[15]	陆克中, 陈超凡, 蔡桓, 吴定明. 面向概念漂移和类不平衡数据流的在线分类算法[J]. 电子学报, 2022, 50(3): 585-597.