基于多模式时空交互的行人轨迹预测模型

doi:10.12263/DZXB.20210752

电子学报 ›› 2022, Vol. 50 ›› Issue (11): 2806-2812.DOI: 10.12263/DZXB.20210752

基于多模式时空交互的行人轨迹预测模型

桑海峰, 陈旺兴, 王海峰, 王金玉

沈阳工业大学信息科学与工程学院, 辽宁沈阳 110870

收稿日期:2021-06-14 修回日期:2021-08-23 出版日期:2022-11-25
- 作者简介:
- 桑海峰男, 1978年1月出生于辽宁省沈阳市.现为沈阳工业大学教授、博士生导师.主要研究方向为机器视觉检测技术和智能视频分析技术.E‑mail: sanghaif@163.com
  陈旺兴（通讯作者）男, 1998年3月出生于江西省抚州市.目前在沈阳工业大学攻读硕士学位，主要研究方向为行人轨迹预测.E‑mail: 1909703861@qq.com
  王海峰男, 1995年3月出生于吉林省吉林市.目前在沈阳工业大学攻读硕士学位，主要研究方向为行人轨迹预测.E‑mail: 798466420@qq.com
  王金玉女, 1996年5月出生于辽宁省盖州市.目前在沈阳工业大学攻读博士学位，主要研究方向为行人轨迹预测.E‑mail: 1911131982@qq.com
- 基金资助:
- 国家自然科学基金(61773105);辽宁省教育厅科研项目(LJGD2020006)

Pedestrian Trajectory Prediction Model Based on Multi-Model Space-Time Interaction

SANG Hai-feng, CHEN Wang-xing, WANG Hai-feng, WANG Jin-yu

School of Information Science and Engineering, Shenyang University of Technology, Shenyang, Liaoning 110870, China

Received:2021-06-14 Revised:2021-08-23 Online:2022-11-25 Published:2022-11-19
- Supported by:
- National Natural Science Foundation of China(61773105);Research Project of Education Department of Liaoning Province(LJGD2020006)

摘要/Abstract

摘要：

在正确地规划合理路径方面，行人轨迹预测具有重要的意义.大多数现有轨迹预测方法在考虑周围行人的影响时，都是简单地将周围行人全部考虑在内，这必然带来的冗余信息.本文提出了一种基于多模式时空交互的行人轨迹预测模型，该模型通过多模式行人空间交互模块对不同行人在不同情况下给予不同的权重，使得模型可以有效减小冗余信息带来的影响.并且本文的模型针对于输入轨迹信息的不同重要程度，设计了加权信息融合模块在原轨迹信息的基础上融合了赋予不同权重的历史轨迹信息，使得模型的轨迹信息更加有效.此外，该模型采用了时间卷积网络模块来捕获行人的时间交互.实验结果表明，在公开数据集ETH和UCY上，相比于Social-STGCNN平均位移误差(Average Displacement Error, ADE)和终点位移误差(Final Displacement Error, FDE)分别降低了15%和14%.

关键词: 行人轨迹预测, 多模式时空交互, 行人空间交互, 加权信息融合, 时间卷积网络, 时间交互

Abstract:

Pedestrian trajectory prediction plays an important role in correctly planning reasonable paths. Most of the existing trajectory prediction methods simply take all the pedestrians into account when considering the influence of the surrounding pedestrians, which inevitably brings redundant information. A pedestrian trajectory prediction model based on multi-mode space-time interaction is proposed. This model gives different weights to different pedestrians in different situations through multi-mode pedestrian space interaction module, which makes the model effectively reduce the impact of redundant information. Aiming at the different importance of the input trajectory information, the weighted information fusion module is designed to integrate the historical trajectory information with different weights on the basis of the original trajectory information, so as to make the trajectory information of the model more effective. In addition, the model uses time convolution network module to capture pedestrian time interaction. The experimental results show that compared with social-stgcnn, average displacement error(ADE) and final displacement error(FDE) is reduced by 15% and 14% respectively on the open data sets ETH and UCY.

Key words: pedestrian trajectory prediction, multi-mode space-time interaction, pedestrian space interaction, weighted information fusion, time convolution network, time interaction

中图分类号:

TP391

桑海峰, 陈旺兴, 王海峰, 等. 基于多模式时空交互的行人轨迹预测模型[J]. 电子学报, 2022, 50(11): 2806-2812.

Hai-feng SANG, Wang-xing CHEN, Hai-feng WANG, et al. Pedestrian Trajectory Prediction Model Based on Multi-Model Space-Time Interaction[J]. Acta Electronica Sinica, 2022, 50(11): 2806-2812.

图/表 11

参考文献 20

1	MOUSSAÏDM, PEROZON, GARNIERS, et al. The walking behaviour of pedestrian social groups and its impact on crowd dynamics[J]. Plos One, 2010, 5(4): e10047.
2	ZHAOD, CHENY, LEL. Deep reinforcement learning with visual attention for vehicle classification[J]. IEEE Transactions on Cognitive & Developmental Systems, 2017, 9(4): 356‐367.
3	ALAHIA, GOELK, RAMANATHANV, et al. Social LSTM: Human trajectory prediction in crowded spaces[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE CS, 2016: 961‐971.
4	HUANGY F, BIH K, LIZ X, et al. Stgat: Modeling spatial-temporal interactions for human trajectory prediction[C]//Proceedings of the IEEE International Conference on Computer Vision. Seoul: CV/IEEE, 2019: 6271‐6280.
5	SADEGHIANA, KOSARAJUV, SADEGHIANA, et al. Sophie: An attentive GAN for predicting paths compliant to social and physical constraints[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE CS, 2019: 1349‐1358.
6	GUPTAA, JOHNSONJ, FEI-FEILI, et al. Social GAN: Socially acceptable trajectories with generative adversarial networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE CS, 2018: 2255‐2264.
7	IVANOVICB, PAVONEM. The trajectron: Probabilistic multi-agent trajectory modeling with dynamic spatiotemporal graphs[C]//Proceedings of the IEEE International Conference on Computer Vision. Seoul: CV/IEEE, 2019: 2375‐2384.
8	ZHANGP, OUYANGW, ZHANGP, et al. SR-LSTM: State refinement for LSTM towards pedestrian trajectory prediction[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE CS, 2019: 12077‐12086.
9	XUEH, HUYNHD Q, REYNOLDSM. Bi-prediction: pedestrian trajectory prediction based on bidirectional LSTM classification[C]//Proceedings of the International Conference on Digital Image Computing: Techniques and Applications. Sydney: IEEE, 2017: 1‐8.
10	XUEH, HUYNHD Q, REYNOLDSM. SS-LSTM: A hierarchical LSTM model for pedestrian trajectory prediction[C]//Proceedings of the IEEE Winter Conference on Applications of Computer Vision. Lake Tahoe: IEEE, 2018: 1186‐1194.
11	XUEH, HUYNHD Q, REYNOLDSM. A location-velocity-temporal attention LSTM model for pedestrian trajectory prediction[J]. IEEE Access, 2020, 8: 44576‐44589.
12	LEEN, CHOIW, VERNAZAP, et al. Desire: Distant future prediction in dynamic scenes with interacting agents[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2165‐2174.
13	FERNANDOT, DENMANS, SRIDHARANS, et al. Soft+ hardwired attention: An LSTM framework for human trajectory prediction and abnormal event detection[J]. Neural Networks, 2018, 108: 466‐478.
14	SUNL, YANZ, MELLADOS M, et al. 3DOF pedestrian trajectory prediction learned from long-term autonomous mobile robot deployment data[C]//Proceedings of the IEEE International Conference on Robotics and Automation. Brisbane: IEEE, 2018: 5942‐5948.
15	MOHAMEDA, QIANK, ELHOSEINYM, et al. Social-stgcnn: A social spatio-temporal graph convolutional neural network for human trajectory prediction[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 14412‐14420.
16	ZHAOX, CHENY, GUOJ, et al. A spatial-temporal attention model for human trajectory prediction[J]. IEEE/CAA Journal of Automatica Sinica, 2020, 7(4): 965‐974.
17	WANGC X, CAIS F, TANG. Graphtcn: Spatio-temporal interaction modeling for human trajectory prediction[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Waikoloa: IEEE, 2021: 3450‐3459.
18	PELLEGRINIS, ESS A, SCHINDLERK, VAN GOOLL. You'll never walk alone: Modeling social behavior for multi-target tracking[C]//2009 IEEE 12th International Conference on Computer Vision. Kyoto: IEEE, 2009: 261‐268.
19	LERNERA, CHRYSANTHOUY, LISCHINSKID. Crowds by example[J]. Computer Graphics Forum, 2007, 26(3): 655‐664.
20	KOSARAJUV, SADEGHIANA, MARTÍN-MARTÍNR, et al. Social-bigat: Multimodal trajectory forecasting using bicycle-GAN and graph attention networks[C]//Proceedings of Annual Conference on Neural Information Processing Systems. Vancouver: NeurIPS, 2019: 1‐10.

数据集场景	对比的模型						本文模型方法
数据集场景	S-LSTM	S-GAN	SoPhie	Social-BiGAT	SR-LSTM	Social-STGCNN	本文模型方法
ETH	1.09/2.35	0.81/1.52	0.70/1.43	0.69/1.29	0.63/1.25	0.64/1.11	0.62/1.08
HOTEL	0.79/1.76	0.72/1.61	0.76/1.67	0.49/1.01	0.37/0.74	0.49/0.85	0.28/0.44
UNIV	0.67/1.40	0.60/1.26	0.54/1.24	0.55/1.32	0.51/1.10	0.44/0.79	0.39/0.74
ZARA1	0.47/1.00	0.34/0.69	0.30/0.63	0.30/0.62	0.41/0.90	0.34/0.53	0.28/0.50
ZARA2	0.56/1.17	0.42/0.84	0.38/0.78	0.36/0.75	0.32/0.70	0.30/0.48	0.25/0.43
AVG	0.72/1.54	0.58/1.18	0.54/1.15	0.48/1.00	0.45/0.94	0.44/0.75	0.37/0.64

数据集场景	对比的模型						本文模型方法
数据集场景	S-LSTM	S-GAN	SoPhie	Social-BiGAT	SR-LSTM	Social-STGCNN	本文模型方法
ETH	1.09/2.35	0.81/1.52	0.70/1.43	0.69/1.29	0.63/1.25	0.64/1.11	0.62/1.08
HOTEL	0.79/1.76	0.72/1.61	0.76/1.67	0.49/1.01	0.37/0.74	0.49/0.85	0.28/0.44
UNIV	0.67/1.40	0.60/1.26	0.54/1.24	0.55/1.32	0.51/1.10	0.44/0.79	0.39/0.74
ZARA1	0.47/1.00	0.34/0.69	0.30/0.63	0.30/0.62	0.41/0.90	0.34/0.53	0.28/0.50
ZARA2	0.56/1.17	0.42/0.84	0.38/0.78	0.36/0.75	0.32/0.70	0.30/0.48	0.25/0.43
AVG	0.72/1.54	0.58/1.18	0.54/1.15	0.48/1.00	0.45/0.94	0.44/0.75	0.37/0.64

	模型参数	推理时间/s
S-LSTM	264k	1.1789
SR-LSTM	64.9k	0.1578
S-GAN-P^［6］	46.3k	0.0968
Social-STGCNN	7.6k	0.0020
本文模型	2828	0.0008

	模型参数	推理时间/s
S-LSTM	264k	1.1789
SR-LSTM	64.9k	0.1578
S-GAN-P^［6］	46.3k	0.0968
Social-STGCNN	7.6k	0.0020
本文模型	2828	0.0008

	ETH	HOTEL	UNIV	ZARA1	ZARA2	AVG
无	0.68/1.17	0.36/0.57	0.41/0.76	0.32/0.55	0.27/0.45	0.41/0.70
有	0.62/1.08	0.28/0.44	0.39/0.74	0.28/0.50	0.25/0.43	0.37/0.64

基于多模式时空交互的行人轨迹预测模型

Pedestrian Trajectory Prediction Model Based on Multi-Model Space-Time Interaction

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献 20

相关文章 2

编辑推荐

Metrics

	单人交互	三人交互	五人交互	七人交互	ADE/FDE
情况1	√				0.33/0.57
情况2	√	√			0.34/0.61
情况3	√		√		0.30/0.47
情况4		√	√		0.31/0.51
情况5	√	√		√	0.29/0.48
情况6	√		√	√	0.33/0.55
情况7		√	√	√	0.30/0.50
情况8	√	√	√	√	0.33/0.58
情况9	√	√	√		0.28/0.44

	1	2	3	4	5	6
1	0.41/0.75	0.39/0.67	0.40/0.70	0.39/0.73	0.37/0.64	0.39/0.70
2	0.42/0.74	0.40/0.69	0.43/0.80	0.40/0.72	0.39/0.68	0.41/0.73
3	0.42/0.75	0.43/0.79	0.41/0.72	0.41/0.80	0.38/0.68	0.41/0.72
4	0.45/0.81	0.44/0.80	0.41/0.73	0.42/0.76	0.44/0.84	0.42/0.75

[1]	裴炤, 邱文涛, 王淼, 马苗, 张艳宁. 基于Transformer动态场景信息生成对抗网络的行人轨迹预测方法[J]. 电子学报, 2022, 50(7): 1537-1547.
[2]	苏兆品, 吴张倩, 岳峰, 武钦芳, 张国富. 自然环境背景噪声下基于低维深度特征的手机来源识别[J]. 电子学报, 2021, 49(4): 637-646.