SMGN：用于对话状态跟踪的状态记忆图网络

doi:10.12263/DZXB.20201463

摘要/Abstract

摘要：

对话状态跟踪是任务型对话系统的重要模块.已有研究使用注意力机制模拟图结构来引入历史信息，但这种方法无法显式利用对话状态的结构性.此外，如何生成复杂格式的对话状态也为研究带来了挑战.针对以上问题，本文提出一种状态记忆图网络SMGN（State Memory Graph Network）.该网络通过状态记忆图保存历史对话信息，并使用图结构与当前对话进行特征交互.本文还设计了一种基于状态记忆图的复杂对话状态生成方法.实验结果表明，本文提出的方法在CrossWOZ数据集上联合正确率提高1.39%，在MultiWOZ数据集上提高1.86%.

关键词: 任务型对话系统, 对话状态跟踪, 注意力机制, 自然语言处理, 深度学习

Abstract:

Dialogue state tracking is an important module of task-oriented dialogue system. Previous studies exploited the historical dialogue information by attention-based graph structure simulation, but these methods cannot explicitly take advantage of the structure of the dialogue state. In addition, how to generate complex format dialogue states also brings challenges to research. In this paper, we propose a state memory graph network(SMGN). The network saves historical information through the state memory graph, and uses the graph to interact with the current dialogue. We also implement a complex dialogue state generation method based on state memory graph. Experimental results show that the proposed method improves the joint accuracy by 1.39% on the CrossWOZ dataset and 1.86% on the MultiWOZ dataset.

Key words: task-oriented dialogue system, dialogue state tracking, attention mechanism, natural language processing, deep learning

中图分类号:

TP391.1

张志昌, 于沛霖, 庞雅丽, 朱林, 曾扬扬. SMGN：用于对话状态跟踪的状态记忆图网络[J]. 电子学报, 2022, 50(8): 1851-1858.

ZHANG Zhi-chang, YU Pei-lin, PANG Ya-li, ZHU Lin, ZENG Yang-yang. SMGN: A State Memory Graph Network for Dialogue State Tracking[J]. Acta Electronica Sinica, 2022, 50(8): 1851-1858.

图/表 9

参考文献 26

1	HENDERSON M, THOMSON B, YOUNG S. Word-based dialog state tracking with recurrent neural networks[C]//Proceedings of the 15th Annual Meeting of the SIGDIAL. Stroudsburg, PA, USA: Association for Computational Linguistics, 2014: 292-299.
2	MRKŠIĆ N, SÉAGHDHA D Ó, WEN T H, et al. Neural belief tracker: Data-driven dialogue state tracking[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Vancouver, Canada: Association for Computational Linguistics, 2017: 1777-1788.
3	ZHONG V, XIONG C M, SOCHER R. Global-locally self-attentive encoder for dialogue state tracking[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics(Volume 1: Long Papers). Stroudsburg, PA, USA: Association for Computational Linguistics, 2018: 1458-1467.
4	CHAO G L, LANE I. BERT-DST: Scalable end-to-end dialogue state tracking with bidirectional encoder representations from transformer[C]//Proceedings of the 20th Annual Conference of the International Speech Communication Association. Graz, Austria: ISCA, 2019: 1468-1472.
5	WU C S, MADOTTO A, HOSSEINI-ASL E, et al. Transferable multi-domain state generator for task-oriented dialogue systems[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy: Association for Computational Linguistics, 2019: 808-819.
6	ZHOU L, SMALL K. Multi-domain dialogue state tracking as dynamic knowledge graph enhanced question answering[EB/OL]. (2020-06-20)[2020-12]. .
7	GAO S Y, SETHI A, AGARWAL S, et al. Dialog state tracking: A neural reading comprehension approach[C]//Proceedings of the SIGDial 2019 Conference. Stockholm, Sweden: Association for Computational Linguistics, 2019: 264-273.
8	ZHANG J G, HASHIMOTO K, WU C S, et al. Find or classify? Dual strategy for slot-value predictions on multi-domain dialog state tracking[C]//Proceedings of the 9th Joint Conference on Lexical and Computational Semantics. Barcelona, Spain: Association for Computational Linguistics, 2020: 154-167.
9	CHEN L, LV B E, WANG C, et al. Schema-guided multi-domain dialogue state tracking with graph attention neural networks[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(5): 7521-7528.
10	ZHU S, LI J Y, CHEN L, et al. Efficient context and schema fusion networks for multi-domain dialogue state tracking[C]//Findings of the Association for Computational Linguistics: EMNLP 2020. Virtual Conference: Association for Computational Linguistics, 2020: 766-781.
11	LEE H, LEE J, KIM T Y. SUMBT: Slot-utterance matching for universal and scalable belief tracking[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy: Association for Computational Linguistics, 2019: 5478-5483.
12	HUANG Z H, XU W, YU K. Bidirectional LSTM-CRF models for sequence tagging[EB/OL]. (2015-08-09)[2020-12].
13	尹存燕, 黄书剑, 戴新宇, 等. 中英命名实体识别及对齐中的中文分词优化[J]. 电子学报, 2015, 43(8): 1481-1487.
	YIN C Y, HUANG S J, DAI X Y, et al. Optimization of Chinese word segmentation in named entity recognition and word alignment[J]. Acta Electronica Sinica, 2015, 43(8): 1481-1487. (in Chinese)
14	郜成胜, 张君福, 李伟平, 等. 一种基于混合神经网络的命名实体识别与共指消解联合模型[J]. 电子学报, 2020, 48(3): 442-448.
	GAO C S, ZHANG J F, LI W P, et al. A joint model of named entity recognition and coreference resolution based on hybrid neural network[J]. Acta Electronica Sinica, 2020, 48(3): 442-448. (in Chinese)
15	FENG Y, WANG Y, LI H. A sequence-to-sequence approach to dialogue state tracking[C]//Proceedings of the 59th ACL and the 11th IJCNLP. Virtual Conference: Association for Computational Linguistics, 2021:1714-1725
16	VELIČKOVIĆ P, CUCURULL G, CASANOVA A, et al. Graph attention networks[EB/OL]. (2018-02-04)[2020-12]. .
17	ZHU Q, HUANG K L, ZHANG Z, et al. CrossWOZ: A large-scale Chinese cross-domain task-oriented dialogue dataset[J]. Transactions of the Association for Computational Linguistics, 2020, (8): 281-295.
18	BUDZIANOWSKI P, WEN T H, TSENG B H, et al. MultiWOZ: A large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling[C]//Proceedings of the 2018 Conference on EMNLP. Brussels: Association for Computational Linguistics, 2018: 5016-5026.
19	ERIC M, GOEL R, PAUL S, et al. MultiWOZ 2.1: A consolidated multi-domain dialogue dataset with state corrections and state tracking baselines[C]//Proceedings of the 12th Language Resources and Evaluation Conference(LREC 2020). Marseille, France: European Language Resources Association, 2020: 422-428
20	DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of NAACL-HLT 2019. Minneapolis, Minnesota, USA: Association for Computational Linguistics, 2019: 4171-4186.
21	张志昌, 曾扬扬, 庞雅丽. 融合语义角色和自注意力机制的中文文本蕴含识别[J]. 电子学报, 2020, 48(11): 2162-2169.
	ZHANG Z C, ZENG Y Y, PANG Y L. A Chinese textual entailment recognition method incorporating semantic role and self-Attention. Acta Electronica Sinica, 2020, 48(11): 2162-2169. (in Chinese)
22	DE BOER P T, KROESE D P, MANNOR S, et al. A tutorial on the cross-entropy method[J]. Annals of Operations Research, 2005, 134(1): 19-67.
23	CUI Y M, CHE W X, LIU T, et al. Pre-training with whole word masking for Chinese BERT[J]. IEEE/ACM Transactions on Audio, Speech and Language Processing, 2021, 29: 3504-3514.
24	KINGMA D P, BA J. Adam: A method for stochastic optimization[C]//Proceedings of the 3rd International Conference on Learning Representations(ICLR 2015). San Diego: arXiv.org, 2015: 1-13.
25	GOEL R, PAUL S, HAKKANI-TÜR D. HyST: A hybrid approach for flexible and accurate dialogue state tracking[C]//Proceedings of the 20th Annual Conference of the International Speech Communication Association. Graz, Austria: ISCA, 2019: 1458-1462.
26	KIM S, YANG S, KIM G, et al. Efficient dialogue state tracking by selectively overwriting memory[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Virtual Conference: Association for Computational Linguistics, 2020: 567-582.

统计信息	CrossWOZ	MultiWOZ 2.0	MultiWOZ 2.1
语言	中文	英文	英文
领域	5	7	5
槽位	72	25	25
槽值	7 871	4 510	4 510
对话数	5 012	8 438	8 133
轮次总数	84 692	115 424	113 556

统计信息	CrossWOZ	MultiWOZ 2.0	MultiWOZ 2.1
语言	中文	英文	英文
领域	5	7	5
槽位	72	25	25
槽值	7 871	4 510	4 510
对话数	5 012	8 438	8 133
轮次总数	84 692	115 424	113 556

任务分类	TRADE联合正确率/%	SMGN联合正确率/%
Single-domain (S)	71.67	72.63
Independent multi-domain (M)	45.29	46.76
multi-domain+traffic (M+T)	37.98	39.25
Cross multi-domain (CM)	30.77	32.92
Cross multi-domain+traffic (CM+T)	25.65	27.74
Overall	36.08	37.47

任务分类	TRADE联合正确率/%	SMGN联合正确率/%
Single-domain (S)	71.67	72.63
Independent multi-domain (M)	45.29	46.76
multi-domain+traffic (M+T)	37.98	39.25
Cross multi-domain (CM)	30.77	32.92
Cross multi-domain+traffic (CM+T)	25.65	27.74
Overall	36.08	37.47

模型	Ontology	BERT	MultiWOZ 2.0 联合正确率/%	MultiWOZ 2.1 联合正确率/%
DSTreader			39.41	36.40
HyST	√		42.33	38.10
TRADE			48.60	45.60
DSTQA	√		51.44	51.17
SOM-DST		√	51.38	52.57
SUMBT	√	√	48.81	52.75
SST	√		51.17	55.23
SMGN		√	53.03	54.88