基于双向准循环神经网络和注意力机制的业务流程剩余时间预测方法

doi:10.12263/DZXB.20211477

电子学报 ›› 2022, Vol. 50 ›› Issue (8): 1975-1984.DOI: 10.12263/DZXB.20211477

基于双向准循环神经网络和注意力机制的业务流程剩余时间预测方法

徐兴荣¹, 刘聪¹^,², 李婷¹, 郭娜¹(), 任崇广¹(), 曾庆田²()

^1.山东理工大学计算机科学与技术学院，山东淄博 255000
^2.山东科技大学计算机科学与工程学院，山东青岛 266590

收稿日期:2021-11-02 修回日期:2022-04-28 出版日期:2022-08-25
- 通讯作者:
- 刘聪
- 作者简介:
- 徐兴荣男，1995年5月出生于山东省济南市.现为山东理工大学计算机科学与技术学院硕士研究生.主要研究方向为流程挖掘.
  刘聪男，1990年8月出生于山东省淄博市 . 现为山东理工大学计算机科学与技术学院教授、博士生导师. 主要研究方向为流程挖掘. E-mail: liucongchina@sdust.edu.cn
  李婷女，1996年11月出生于山东省济宁市.现为山东理工大学计算机科学与技术学院硕士研究生.主要研究方向为流程挖掘.E-mail: Ltingoo@163.com
  郭娜女，1996年7月出生于山东省淄博市.现为山东理工大学计算机科学与技术学院博士研究生.主要研究方向为流程挖掘.E-mail: guona_7@163.com
  任崇广男，1982年7月出生于山东省淄博市.现为山东理工大学计算机科学与技术学院教授、硕士生导师.主要研究方向为智能装备.E-mail: renchg@sdut.edu.cn
  曾庆田男，1976年1月出生于山东省潍坊市.现为山东科技大学计算机科学与工程学院教授、博士生导师.主要研究方向为流程挖掘.E-mail: qtzeng@163.com
- 基金资助:
- 国家自然科学基金 (61902222); 山东省泰山学者工程专项基金 (ts20190936); 山东省自然科学基金优秀青年基金 (ZR2021YQ45); 山东省高等学校青创科技计划创新团队项目 (2021KJ031)

Business Process Remaining Time Prediction: An Approach Based on Bidirectional Quasi Recurrent Neural Network with Attention

XU Xing-rong¹, LIU Cong¹^,², LI Ting¹, GUO Na¹(), REN Chong-guang¹(), ZENG Qing-tian²()

^1.School of Computer Science and Technology，Shandong University of Technology，Zibo，Shandong 255000，China
^2.College of Computer Science and Engineering，Shandong University of Science and Technology，Qingdao，Shandong 266590，China

Received:2021-11-02 Revised:2022-04-28 Online:2022-08-25 Published:2022-09-08
- Corresponding author:
- LIU Cong

摘要/Abstract

摘要：

业务流程预测可以有效帮助企业进行流程控制和传递高质量服务，因此作为此类场景中的核心任务之一，业务流程剩余时间预测得到国内外学者的广泛关注.当前，在利用深度学习技术对业务流程剩余时间进行预测时，大都采用传统长短期记忆循环神经网络，然而，由于长短期记忆循环神经网络在处理序列数据的过程中缺乏并行性且建模能力有限，使得预测准确度还有进一步提升空间.因此，本文提出一种基于双向准循环神经网络和注意力机制的业务流程剩余时间预测方法.首先，该方法以双向准循环神经网络构建剩余时间预测模型，并在预测模型中融入注意力机制增强双向准循环神经网络输出的特征信息.其次，设计了一种基于不同长度轨迹前缀训练迭代策略，解决流程实例中不同长度轨迹前缀数量存在差异性的问题.最后，提出一种基于Word2vec的事件表示学习方法，实现对同一轨迹且经常出现事件的相似性向量表示，从而达到提高剩余时间预测准确度的目的.经在5个公开事件日志数据集上实验，本文方法与已有方法相比在预测准确度上平均提高近15%，模型训练时间平均缩短约26%.

关键词: 深度学习, 准循环神经网络, 业务流程, 剩余时间预测, 事件表示学习

Abstract:

Business process prediction can effectively facilitate enterprises to control processes and deliver high-quality services. As one of the core tasks of process prediction, remaining time prediction has been widely concerned by scholars. Currently, traditional long short-term memory(LSTM) neural networks have been used to predict the remaining time of business process instances. However, due to the lack of parallelism and limited modeling ability of LSTM in processing sequence data,the accuracy of prediction has further room to improve. In this paper, the remaining time prediction method based on bidirectional quasi-recurrent neural network with attention is proposed. Firstly, this method uses the bidirectional quasi-recurrent neural network to build the prediction model, and adds the attention mechanism to the model enhances the characteristic information of the bidirectional quasi-recurrent neural network output.Secondly, a training iteration strategy based on different length trace prefixes is designed, which solves the problem of the difference in the number of trace prefixes of different lengths. Finally, event representation learning method is proposed, to achieve vectors representation of similarity to the same traces and frequent events, improves the accuracy of the remaining time prediction. Experiments on five public event log datasets show this method has improved the accuracy of prediction by an average of nearly 15%, and the average training time is reduced by about 26%, compared with the existing methods.

Key words: deep learning, quasi-recurrent neural network, business process, remaining time prediction, event representation learning

中图分类号:

TP391

徐兴荣, 刘聪, 李婷, 郭娜, 任崇广, 曾庆田. 基于双向准循环神经网络和注意力机制的业务流程剩余时间预测方法[J]. 电子学报, 2022, 50(8): 1975-1984.

XU Xing-rong, LIU Cong, LI Ting, GUO Na, REN Chong-guang, ZENG Qing-tian. Business Process Remaining Time Prediction: An Approach Based on Bidirectional Quasi Recurrent Neural Network with Attention[J]. Acta Electronica Sinica, 2022, 50(8): 1975-1984.

图/表 10

参考文献 25

1	TEINEMAA I, DUMAS M, ROSA M L, et al. Outcome-oriented predictive process monitoring: Review and benchmark[J]. ACM Transactions on Knowledge Discovery from Data, 2019, 13(2): 17.
2	TAX N, VERENICH I, LA ROSA M, et al. Predictive business process monitoring with LSTM neural networks[C]//International Conference on Advanced Information Systems Engineering. Aachen: Springer, 2017: 477-492.
3	VAN DER AALST W P M, SCHONENBERG M H, SONG M. Time prediction based on process mining[J]. Information Systems, 2011, 36(2): 450-475.
4	赵海燕, 李帅标, 陈庆奎, 等. 面向业务过程的时间预测方法[J]. 小型微型计算机系统, 2019, 40(2): 42-48.
	ZHAO Hai-yan, LI Shuai-biao, CHEN Qing-kui, et al. Method of time prediction for business process[J]. Journal of Chinese Computer Systems, 2019, 40(2): 42-48. (in Chinese)
5	ROGGE-SOLTI A, WESKE M. Prediction of business process durations using non-Markovian stochastic petri nets[J]. Information Systems, 2015, 54: 1-14.
6	NAVARIN N, VINCENZI B, POLATO M, et al. LSTM networks for data-aware remaining time prediction of business process instances[C]//IEEE Symposium Series on Computational Intelligence. Hawaiian: IEEE, 2017: 1-7.
7	LEONTJEVA A, CONFORTI R, DI FRANCESCOMARINO C, et al. Complex symbolic sequence encodings for predictive monitoring of business processes[C]//International Conference on Business Process Management. Dubai: Springer, 2016: 297-313.
8	FOLINO F, GUARASCIO M, PONTIERI L. Mining predictive process models out of low-level multidimensional logs[C]//International Conference on Advanced Information Systems Engineering. Luxembourg: Springer, 2014: 533-547.
9	崔亮. 基于机器学习的业务流程系统的预测[D]. 北京: 北京邮电大学, 2019.
10	POLATO M, SPERDUTI A, BURATTIN A, et al. Time and activity sequence prediction of business process instances[J]. Computing, 2018, 100(9): 1005-1031.
11	VERENICH I, NGUYEN H, LA ROSA M, et al. White-box prediction of process performance indicators via flow analysis[C]//Proceedings of the 2017 International Conference on Software and System Process. Paris: ACM, 2017: 85-94.
12	EVERMANN J, REHSE J, FETTKE P. A deep learning approach for predicting process behaviour at runtime[C]//Business Process Management Workshops. Rio de Janeiro: Springer, 2016: 327-388.
13	NGUYEN A, CHATTERJEE S, WEINZIERL S, et al. Time matters: Time-aware LSTMs for predictive business process monitoring[C]//International Workshop on Leveraging Machine Learning in Process Mining. Padua: ICPM, 2020: 1-12.
14	倪维健, 孙宇健, 刘彤, 等. 基于注意力双向循环神经网络的业务流程剩余时间预测方法[J]. 计算机集成制造系统, 2020, 26(6): 1564-1572.
	NI Wei-jian, SUN Yu-jian, LIU Tong, et al. Business process remaining time prediction using bidirectional recurrent neural networks with attention[J]. Computer Integrated Manufacturing Systems, 2020, 26(6): 1564-1572. (in Chinese)
15	KHAN A, LE H, DO K, et al. DeepProcess: Supporting business process execution using a MANN-based recommender system[EB/OL]. (2018)[2021]. .
16	PASQUADIBISCEGLIE V, APPICE A, CASTELLANO G, et al. Using convolutional neural networks for predictive process analytics[C]//International Conference on Process Mining, Aachen: ICPM, 2019: 29-136.
17	TAYMOURI F, ROSA M L. Encoder-decoder generative adversarial nets for suffix generation and remaining time predication of business process models[EB/OL].(2020)[2021]. .
18	BUKHSH Z A, SAEED A, DIJKMAN R M. ProcessTransformer: Predictive business process monitoring with transformer network[EB/OL]. (2021)[2021]. .
19	MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space[EB/OL]. (2013)[2021]. .
20	BRADBURY J, MERITY S. Quasi-recurrent neural networks[EB/OL]. (2019)[2021]. .
21	王栋, 李业刚, 张晓, 等. 基于准循环神经网络的中文命名实体识别[J]. 计算机工程与设计, 2020, 41(7): 2038-2043.
	WANG Dong, LI Ye-gang, ZHANG Xiao, et al. Chinese name entity recognition based on quasi-recurrent neural networks[J]. Computer Engineering and Design, 2020, 41(7): 2038-2043. (in Chinese)
22	时云龙, 袁文浩, 胡少东, 等. 一种用于实时语音增强的卷积准循环网络[J]. 西安电子科技大学学报, 2022, 49(3): 1-8.
	Shi Yun-long, Yuan Wen-hao, Hu Shao-dong, et al. Convolutional quasi-recurrent network for real-time speechenhancement[J]. Journal of Xidian University, 2022, 49(3):1-8. (in Chinese)
23	DONGEN VAN. Bpi2012[OL]. .
24	VERENICH IlYA. Helpdesk[OL]. .
25	MANNHARDT FELIX. Hospital Billing[OL]. .

数据集	轨迹数量	事件数量	活动数量	轨迹最大长度	轨迹最小长度
BPIC_2012_A^[23]	13 087	73 022	10	10	3
BPIC_2012_O^[23]	5 015	41 728	7	39	4
BPIC_2012_W^[23]	9 658	147 450	6	153	1
Helpdesk^[24]	3 804	13 710	9	14	1
Hospital_Billing^[25]	100 000	451 359	18	217	1

数据集	轨迹数量	事件数量	活动数量	轨迹最大长度	轨迹最小长度
BPIC_2012_A^[23]	13 087	73 022	10	10	3
BPIC_2012_O^[23]	5 015	41 728	7	39	4
BPIC_2012_W^[23]	9 658	147 450	6	153	1
Helpdesk^[24]	3 804	13 710	9	14	1
Hospital_Billing^[25]	100 000	451 359	18	217	1

方法	BPIC_2012_A	BPIC_2012_O	BPIC_2012_W	Helpdesk	Hospital_Billing
LSTM^[14]	3.588	7.993	8.021	3.542	42.050
Bi-LSTM^[14]	4.600	8.407	7.221	2.952	41.515
Att-Bi-LSTM^[14]	3.895	7.324	6.153	2.677	36.691
Trans-Att-Bi-LSTM^[14]	3.489	5.858	5.826	3.357	33.201
GRU^[14]	4.888	7.394	7.753	3.425	47.400
Bi-GRU^[14]	4.461	8.144	7.334	3.222	37.157
Att-Bi-GRU^[14]	3.512	7.306	6.338	3.303	33.201
Trans-Att-Bi-GRU^[14]	3.438	5.863	5.821	3.299	32.187
QRNN	3.543	6.779	7.516	3.199	36.611
Bi-QRNN	3.992	6.757	7.128	2.884	35.248
Att-Bi-QRNN	3.200	6.276	6.007	2.626	32.101
Trans-Att-Bi-QRNN	2.373	5.158	5.275	2.423	31.436

方法	BPIC_2012_A	BPIC_2012_O	BPIC_2012_W	Helpdesk	Hospital_Billing
LSTM^[14]	3.588	7.993	8.021	3.542	42.050
Bi-LSTM^[14]	4.600	8.407	7.221	2.952	41.515
Att-Bi-LSTM^[14]	3.895	7.324	6.153	2.677	36.691
Trans-Att-Bi-LSTM^[14]	3.489	5.858	5.826	3.357	33.201
GRU^[14]	4.888	7.394	7.753	3.425	47.400
Bi-GRU^[14]	4.461	8.144	7.334	3.222	37.157
Att-Bi-GRU^[14]	3.512	7.306	6.338	3.303	33.201
Trans-Att-Bi-GRU^[14]	3.438	5.863	5.821	3.299	32.187
QRNN	3.543	6.779	7.516	3.199	36.611
Bi-QRNN	3.992	6.757	7.128	2.884	35.248
Att-Bi-QRNN	3.200	6.276	6.007	2.626	32.101
Trans-Att-Bi-QRNN	2.373	5.158	5.275	2.423	31.436

	Diff- Att-Bi-QRNN	Trans- Att-Bi-QRNN
BPIC_2012_A	3.880	3.739
BPIC_2012_O	5.838	5.219
BPIC_2012_W	7.475	6.844
Helpdesk	2.108	1.511
Hospital_Billing	36.488	30.878

基于双向准循环神经网络和注意力机制的业务流程剩余时间预测方法

Business Process Remaining Time Prediction: An Approach Based on Bidirectional Quasi Recurrent Neural Network with Attention

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 25

相关文章 15

编辑推荐

Metrics

预测模型	参数量/M
LSTM	0.007 73
Bi-LSTM	0.013 83
Att-Bi-LSTM	0.014 03
GRU	0.006 23
Bi-GRU	0.010 83
Att-Bi-GRU	0.011 03
QRNN	0.003 83
Bi-QRNN	0.004 23
Att-Bi-QRNN	0.004 43

[1]	吴靖, 叶晓晶, 黄峰, 陈丽琼, 王志锋, 刘文犀. 基于深度学习的单帧图像超分辨率重建综述[J]. 电子学报, 2022, 50(9): 2265-2294.
[2]	李雪莹, 王田路, 梁鹏, 王翀. 基于系统模型的用户评论中非功能需求的自动分类[J]. 电子学报, 2022, 50(9): 2079-2089.
[3]	琚长瑞, 秦晓燕, 袁广林, 李豪, 朱虹. 尺度敏感损失与特征融合的快速小目标检测方法[J]. 电子学报, 2022, 50(9): 2119-2126.
[4]	张志昌, 于沛霖, 庞雅丽, 朱林, 曾扬扬. SMGN：用于对话状态跟踪的状态记忆图网络[J]. 电子学报, 2022, 50(8): 1851-1858.
[5]	张亚洲, 俞洋, 朱少林, 陈锐, 戎璐, 梁辉. 一种量子概率启发的对话讽刺识别网络模型[J]. 电子学报, 2022, 50(8): 1885-1893.
[6]	王飞扬, 冀鹏欣, 孙笠, 危倩, 李根, 张忠宝. 一种基于深度学习的动态社交网络用户对齐方法[J]. 电子学报, 2022, 50(8): 1925-1936.
[7]	裴炤, 邱文涛, 王淼, 马苗, 张艳宁. 基于Transformer动态场景信息生成对抗网络的行人轨迹预测方法[J]. 电子学报, 2022, 50(7): 1537-1547.
[8]	欧阳与点, 谢鲲, 谢高岗, 文吉刚. 面向大规模网络测量的数据恢复算法：基于关联学习的张量填充[J]. 电子学报, 2022, 50(7): 1653-1663.
[9]	杨伟超, 杜宇, 文伟, 侯舒维, 徐常志, 张建华. 基于多重分形谱智能分析的卫星信号调制识别研究[J]. 电子学报, 2022, 50(6): 1336-1343.
[10]	彭闯, 王伦文, 胡炜林. 融合深度特征的电磁频谱异常检测算法[J]. 电子学报, 2022, 50(6): 1359-1369.
[11]	李政伟, 李佳树, 尤著宏, 聂茹, 赵欢, 钟堂波. 基于异质图注意力网络的miRNA与疾病关联预测算法[J]. 电子学报, 2022, 50(6): 1428-1435.
[12]	张波, 陆云杰, 秦东明, 邹国建. 一种卷积自编码深度学习的空气污染多站点联合预测模型[J]. 电子学报, 2022, 50(6): 1410-1427.
[13]	冀振燕, 韩梦豪, 宋晓军, 冯其波. 面向激光光条图像修复的循环相似度映射网络[J]. 电子学报, 2022, 50(5): 1234-1242.
[14]	廖勇, 李玉杰. 一种轻量化低复杂度的FDD大规模MIMO系统CSI反馈方法[J]. 电子学报, 2022, 50(5): 1211-1217.
[15]	潘敏婷, 王韫博, 朱祥明, 高思宇, 龙明盛, 杨小康. 基于无标签视频数据的深度预测学习方法综述[J]. 电子学报, 2022, 50(4): 869-886.