基于深度学习和智能规划的行为识别

doi:10.3969/j.issn.0372-2112.2019.08.008

电子学报 ›› 2019, Vol. 47 ›› Issue (8): 1661-1668.DOI: 10.3969/j.issn.0372-2112.2019.08.008

基于深度学习和智能规划的行为识别

郑兴华¹, 孙喜庆¹, 吕嘉欣¹, 鲜征征², 李磊¹

1. 中山大学数据科学与计算机学院, 广东广州 510006;
2. 广东金融学院互联网金融与信息工程学院, 广东广州 510521

收稿日期:2018-05-18 修回日期:2019-01-02 出版日期:2019-08-25
- 通讯作者:
- 鲜征征
- 作者简介:
- 郑兴华 男,1983年生于广东广州.中山大学数据科学与计算机学院博士.主要研究方向为智能规划、数据挖掘、模式识别、神经网络等.E-mail:zhengxh5@mail3.sysu.edu.cn
- 基金资助:
- 广东省自然科学基金 (No.2017A030313391）; 广东省科技厅国际合作项目 (No.2017A050501042）

Action Recognition Based on Deep Learning and Artificial Intelligence Planning

ZHENG Xing-hua¹, SUN Xi-qing¹, LU Jia-xin¹, XIAN Zheng-zheng², LI Lei¹

1. School of Data and Computer Science, Sun Yat-sen University, Guangzhou, Guangdong 510006, China;
2. School of Internet Finance and Information Engineering, Guangdong University of Finance, Guangzhou, Guangdong 510521, China

Received:2018-05-18 Revised:2019-01-02 Online:2019-08-25 Published:2019-08-25
- Supported by:
- National Natural Science Foundation of Guangdong Province, China (No.2017A030313391); International Coorperation Program of Department of Science and Technology of Guangdong Province (No.2017A050501042)

摘要/Abstract

摘要： 现有行为识别方法在未能持续覆盖造成视频监控盲区所引起行为数据缺失的情况，难以有效实施特征分析、行为分类补全，无法准确识别出智能体完整的行为动作序列.为此，本文提出一种基于深度学习和智能规划的行为识别方法.首先，利用深度残差网络对图像进行分类训练，然后使用递归神经网络对图像特征进行提取深度信息以增强分类效果；其次，运用智能规划的STRIPS （Stanford Research Institute Problem Solver）模型，将深度学习提取的图像特征命题信息转化为规划领域的模型描述文档，并使用前向状态空间搜索规划器推导出完整的行为动作序列.在HMDB51等行为识别公共数据集中，本方法与生成式对抗网络、深度卷积逆向图网络、深度信念网络、支持向量机等同类先进方法相比展现出更好的性能.

关键词: 行为识别, 深度学习, 智能规划, 深度残差网络, 递归神经网络, STRIPS规划模型, 前向状态空间搜索规划器

Abstract: Currently, action recognition methods can hardly carry out feature analysis,behavior classification,and action completion, and are incapable of accurately identifying the complete behavioral action sequence of intelligent agent for the discontinuous and incomplete motion capture, behavioral data missing or even broken in the time dimension, which are resulted from sensor device not being continuous coverage caused by the monitoring blind area. In this regard, we put forward a method of action recognition based on deep learning and artificial intelligence planning. Firstly, a deep learning network is constructed, by which the image is classified and trained using DRN(Deep Residual Network).After that,the extraction depth information of image frame feature for recurrent neural network is trained to enhance the classification effect. Secondly, the STRIPS (Stanford Research Institute Problem Solver) planning model is used to extract the image feature of deep learning, transforming into the description document for domain model, which facilitates deriving the optimal planning solution by means of forward state-space search planner. In the experiment, we exhibit that our method outperforms baselines in the public datasets, e.g., DCIGN(Deep Convolutional Inverse Graphics Networks), GAN(Generative Adversarial Networks), DBN(Deep Belief Networks), and SVM(Support Vector Machine).

Key words: action recognition, deep learning, artificial intelligence planning, deep residual network, recurrent neural network, STRIPS planning model, forward state-space search planner

中图分类号:

TP302

郑兴华, 孙喜庆, 吕嘉欣, 等. 基于深度学习和智能规划的行为识别[J]. 电子学报, 2019, 47(8): 1661-1668.

ZHENG Xing-hua, SUN Xi-qing, LU Jia-xin, et al. Action Recognition Based on Deep Learning and Artificial Intelligence Planning[J]. Acta Electronica Sinica, 2019, 47(8): 1661-1668.

参考文献

[1] Jo o Carreira,Andrew Zisserman.Quo Vadis.Action recognition? a new model and the kinetics dataset[A].IEEE Conference on Computer Vision and Pattern Recognition[C].Washington,DC,USA:IEEE,2017.4724-4733.
[2] 张友梅,常发亮,刘洪彬.基于3D人体骨架的动作识别[J].电子学报,2017,45(4):906-911. ZHANG You-mei,CHAN Fa-liang,LIU Hong-bin.Action recognition based on 3D skeleton[J].Acta Electronica Sinica,2017,45(4):906-911.(in Chinese)
[3] Christoph Feichtenhofer,Axel Pinz,RichardWildes,Andrew Zisserman.What have we learned from deep representations for action recognition?[A].IEEE Conference on Computer Vision and Pattern Recognition[C].Washington,DC,USA:IEEE,2018.7844-7853.
[4] Du Tran,et al.A closer look at spatiotemporal convolutions for action recognition[A].IEEE Conference on Computer Vision and Pattern Recognition[C].Washington,DC,USA:IEEE,2018.6450-6459.
[5] Chen B,Ting J A,De Freitas N.Deep learning of invariant spatio-temporal features from video[A].Conference and Workshop on Neural Information Processing Systems[C].Cambridge,Massachusetts,USA:MIT Press,2010.46-57.
[6] Jurgen Schmidhuber.Deep learning in neural networks:An overview[J].Neural Networks,2015,61:85-117.
[7] Hinton G E,Salakhutdinov R R.Reducing the dimensionality of data with neural networks[J].Science,2006,50:313-325.
[8] Shaoqing Ren,Kaiming He,Ross B.Girshick,Jian Sun.Faster R-CNN:Towards real-time object detection with region proposal networks[A].Conference and Workshop on Neural Information Processing Systems[C].Cambridge,Massachusetts,USA:MIT Press,2015.91-99.
[9] 李倩玉,蒋建国,齐美彬.基于改进深层网络的人脸识别算法[J].电子学报,2017,45(3):619-625. LI Qian-yu,JIANG Jian-guo,QI Mei-bin.Face recognition algorithm based on improved deep networks[J].Acta Electronica Sinica,2017,45(3):619-625.(in Chinese)
[10] 曾霖,卓汉逵,李磊.基于智能规划的工作流任务识别算法[J].2018,46(4):871-877. ZENG Lin,ZHUO Han-kui,LI Lei.Workflowtask recognition based on intelligent planning[J].Acta Electronica Sinica,2016,44(8):2025-2032.(in Chinese)
[11] 高洁,卓汉逵,刘亚松,李磊.基于众包模式的开放式规划问题研究[J].电子学报,2016,44(8):2025-2032. GAO Jie,ZHUO Han-kui,LIU Ya-seng,LI Lei.Research on crowdsourced open planning[J].Acta Electronica Sinica,2016,44(8):2025-2032.(in Chinese)
[12] Fikes,R.E.,Nilsson,N.I.STRIPS:A new approach to the application of theorem proving to problem solving[J].Artificial Intelligence,1972,2:189-208.
[13] Hankz Hankui Zhuo,Qiang Yang,Subbarao Kambhampati.Action-model based multi-agent plan recognition[A].Conference and Workshop on Neural Information Processing Systems[C].Cambridge,Massachusetts,USA:MIT Press,2012.377-385.
[14] Hankz Hankui Zhuo,Tuan Nguyen and Subbarao Kambhampati.Refiningincomplete planning domain models through plan traces[A].International Joint Conferences on Artificial Intelligence[C].San Jose,California,USA:Margan Kaufmann,2013.2451-2457.
[15] Thomas B.Moeslund,et al.A survey of advances in vision-based human motion capture and analysis[J].Computer Vision and Image Understanding,2006,104(2):90-126.
[16] E.Charniak and R.P.Goldman.A Bayesian model of plan recognition[J].Artificial Intelligence,1993,64:53-79.
[17] Tao Gu,et al.SICAR:An emerging patterns based approach to sequential,interleaved and concurrent activity r.ecognition[A].International Conference on Pervasive Computing and Communications[C].Washington,DC,USA:IEEE Computer Society,2009.142-151.
[18] Kautz H,Allen J F.Generalized plan recognition[A].AAAI Conference on Artificial Intelligence[C].Palo Alto,California,USA:AAAI,1986.167-178.
[19] Ji S,Xu W,Yang M,et al.3D convolutional neural networks for human action recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,35(1):221-231.
[20] Simonyan K,Zisserman A.Two-stream convolutional networks for action recognition in videos[A].Conference and Workshop on Neural Information Processing System[C].Cambridge,Massachusetts,USA:MIT Press,2014.568-576.
[21] Bengio Y,Delaalleau O.On the expressive power of deep architectures[A].International Conference on Algorithmic Learning Theory[C].Berlin,GER:Springer,2011.18-36.
[22] Taylor G W,Hinton G E.Factored conditional restricted Boltzmann machines for modeling motion style[A].International Conference on Machine Learning[C].San Jose,California,USA:ACM,2009.1025-1032.
[23] Shuai Zheng,et al.Conditional random fields as recurrent neural networks[A].IEEE International Conference on Computer Vision[C].Washington,DC,USA:IEEE,2015.1529-1537.
[24] Jamil Ahmad,Khan Muhammad,Muhammad Sajjad,Sung Wook Baik.Action recognition in video sequences using deep bi-directional LSTM with CNN features[J].IEEE Access,2018,6(1):1155-1166.
[25] A.Krizhevsky,I.Sutskever,G.Hinton.ImageNet classification with deep convolutional neural networks[A].Conference and Workshop on Neural Information Processing Systems[C].Cambridge,Massachusetts,USA:MIT Press,2012.1273-1291.
[26] LeCun Y,et al.Backpropagation applied to handwritten zip code recognition[J].Neural Computation,1989,1(4):541-551.
[27] Ghallab M,Aeronautiques C,Isi C K,et al.PDDL-The Planning Domain Definition Language[M].Boston:Auerbach Publications,1998.
[28] Henry A.Kautz,Bart Selman.Pushingthe envelope:planning,propositional logic and stochastic search[A].AAAI Conference on Artificial Intelligence[C].Palo Alto,California,USA:AAAI,1996.1194-1201.
[29] Javier Segovia Aguas,Sergio Jiménez Celorrio,Anders Jonsson.Hierarchical finite state controllers for generalized planning[A].International Joint Conference on Artificial Intelligence[C].San Jose,California,USA:Margan Kaufmann,2016.3235-3241.
[30] Chao Dong,Chen Change Loy,Kaiming He,Xiaoou Tang.Imagesuper-resolution using deep convolutional networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295-307.
[31] 王泽宇,吴艳霞,张国印,布树辉.基于空间结构化推理深度融合网络的RGB-D场景解析[J].电子学报,2018,46(5):1253-1258. WANG Ze-yu,WU Yan-xia,ZHANG Guo-yin,BU Shu-hui.RGB-D scene parsing based on spatial structured inference deep fusion networks[J].Acta Electronica Sinica,2018,46(5):1253-1258.(in Chinese)
[32] Xinhua Liu,Yao Zou,Chengjuan Xie,Xiaolin Ma.Bidirectional face aging synthesis based on improved deep convolutional generative adversarial networks[J].Information,2019,10(2):69-84.
[33] Joe Yue-Hei Ng,et al.Beyond short snippets:Deep networks for video classification[A].IEEE Conference on Computer Vision and Pattern Recognition[C].Washington,DC,USA:IEEE,2015.4694-4702.
[34] P.Shih,C.Liu.Face detection using discriminating feature analysis and support vector machine[J].Pattern Recognition,2006,39(11):260-276.

基于深度学习和智能规划的行为识别

Action Recognition Based on Deep Learning and Artificial Intelligence Planning

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	李钦, 刘伟, 牛朝阳, 宝音图, 惠周勃. 低信噪比下基于分裂EfficientNet网络的雷达信号调制方式识别[J]. 电子学报, 2023, 51(3): 675-686.
[2]	范兵兵, 何庭建, 张聪炫, 陈震, 黎明. 联合遮挡约束与残差补偿的特征金字塔光流计算方法[J]. 电子学报, 2023, 51(3): 648-657.
[3]	张聿远, 闫文君, 张立民. 基于多模态特征融合网络的空时分组码识别算法[J]. 电子学报, 2023, 51(2): 489-498.
[4]	许新征, 李杉. 基于特征膨胀卷积模块的轻量化技术研究[J]. 电子学报, 2023, 51(2): 355-364.
[5]	张永梅, 孙捷. 基于动静态特征双输入神经网络的咳嗽声诊断COVID-19算法[J]. 电子学报, 2023, 51(1): 202-212.
[6]	袁海英, 成君鹏, 曾智勇, 武延瑞. Mobile_BLNet：基于Big-Little Net的轻量级卷积神经网络优化设计[J]. 电子学报, 2023, 51(1): 180-191.
[7]	王神龙, 雍宇, 吴晨睿. 基于伪孪生神经网络的低纹理工业零件6D位姿估计[J]. 电子学报, 2023, 51(1): 192-201.
[8]	李滔, 董秀成, 林宏伟. 基于深监督跨尺度注意力网络的深度图像超分辨率重建[J]. 电子学报, 2023, 51(1): 128-138.
[9]	郭晓轩, 冯其波, 冀振燕, 郑发家, 杨燕燕. 多线激光光条图像缺陷分割模型研究[J]. 电子学报, 2023, 51(1): 172-179.
[10]	贾童瑶, 卓力, 李嘉锋, 张菁. 基于深度学习的单幅图像去雾研究进展[J]. 电子学报, 2023, 51(1): 231-245.
[11]	何滢婕, 刘月峰, 边浩东, 郭威, 张小燕. 基于Informer的电池荷电状态估算及其稀疏优化方法[J]. 电子学报, 2023, 51(1): 50-56.
[12]	吴靖, 叶晓晶, 黄峰, 陈丽琼, 王志锋, 刘文犀. 基于深度学习的单帧图像超分辨率重建综述[J]. 电子学报, 2022, 50(9): 2265-2294.
[13]	李雪莹, 王田路, 梁鹏, 王翀. 基于系统模型的用户评论中非功能需求的自动分类[J]. 电子学报, 2022, 50(9): 2079-2089.
[14]	琚长瑞, 秦晓燕, 袁广林, 李豪, 朱虹. 尺度敏感损失与特征融合的快速小目标检测方法[J]. 电子学报, 2022, 50(9): 2119-2126.
[15]	邓海刚, 王传旭, 李成伟, 林晓萌. 深度学习框架下群组行为识别算法综述[J]. 电子学报, 2022, 50(8): 2018-2036.