基于短语向量和主题加权的关键词抽取方法

doi:10.12263/DZXB.20200014

电子学报 ›› 2021, Vol. 49 ›› Issue (9): 1682-1690.DOI: 10.12263/DZXB.20200014

所属专题：自然语言处理：技术与应用

基于短语向量和主题加权的关键词抽取方法

孙新¹^,², 盖晨¹, 申长虹¹, 张颖捷¹

^1.北京理工大学计算机学院北京市海量语言信息处理与云计算应用工程技术研究中心，北京 100081
^2.北京理工大学东南信息技术研究院，福建莆田 351100

收稿日期:2019-12-23 修回日期:2020-11-24 出版日期:2021-10-21
- 作者简介:
- 孙新　女， 1975年4月生于吉林省长春市.现为北京理工大学计算机学院副教授，硕士生导师. 主要研究方向为人工智能，机器学习.E-mail: sunxin@bit.edu.cn
  盖晨　男，1996年5月生于河北省石家庄市.现为北京理工大学计算机学院硕士生研究生.主要研究方向为人工智能.E-mail: 851774342@qq.com
  申长虹　女，1995年3月生于河南省.北京理工大学计算机学院硕士生.主要研究方向为人工智能. E-mail: suishelly@foxmail.com
- 基金资助:
- 国家重点研发计划项目 (2017YFB0803300)

The Theme-Weighted Keyphrase Extraction Algorithm Based on Phrase Embedding

SUN Xin¹^,², GE Chen¹, SHEN Chang-hong¹, ZHANG Ying-jie¹

^1.Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications, School of Computer Science and Technology, Beijing Institute of Technology, Beijing 100081, China
^2.Beijing Institute of Technology Southeast Academy of Information Technology, Putian, Fujian 351100, China

Received:2019-12-23 Revised:2020-11-24 Online:2021-10-21 Published:2021-09-25
- Supported by:
- Program of National Key Research and Development Program of China (2017YFB0803300)

摘要/Abstract

摘要：

现有关键词抽取算法缺乏对短语的有效表示，为抽取出更能反映文本主题的关键短语，本文提出一种基于短语向量的关键词抽取方法PhraseVecRank.首先设计基于LSTM（Long Short-Term Memory）和CNN（Convolutional Neural Network）自编码器的短语向量构建模型，解决复杂短语的语义表示问题.然后，利用短语向量对每个候选短语计算主题权重，通过主题加权排序提高关键词抽取的效果.在公共数据集和学术论文数据上的实验表明，本文提出的方法能够有效提取与文本主题信息相关的关键短语，同时利用自编码器构造的短语向量可以更好地表示短语的语义信息.

关键词: 短语向量, 自编码器, 主题加权, 关键词抽取

Abstract:

Keyword extraction is a key basic problem in the field of natural language processing. The keyphrase extraction algorithms(PhraseVecRank) is proposed based on phrase embedding. Firstly, a phrase vector construction model based on LSTM（Long Short-Term Memory） and CNN（Convolutional Neural Network） is designed to solve the semantic representation of complex phrases. Then, PhraseVecRank uses phrase embedding to calculate theme weight for each candidate phrase, and uses semantic similarity between candidate phrase embedding and co-occurrence information to calculate edge weight together, which can improve the extraction effect of keyphrases through topic weighted ranking. The experimental results verify that PhraseVecRank can effectively extract keyphrases covering the topic information of text, and the phrase embedding models we proposed can better represent the semantic information of phrases.

Key words: phrase embedding, auto-encoder, theme-weighted, keyphrases extraction

中图分类号:

TP181

孙新, 盖晨, 申长虹, 等. 基于短语向量和主题加权的关键词抽取方法[J]. 电子学报, 2021, 49(9): 1682-1690.

Xin SUN, Chen GE, Chang-hong SHEN, et al. The Theme-Weighted Keyphrase Extraction Algorithm Based on Phrase Embedding[J]. Acta Electronica Sinica, 2021, 49(9): 1682-1690.

图/表 10

参考文献 15

1	PapagiannopoulouE, TsoumakasG. A review of keyphrase extraction[J]. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2020, 10(2): e1339.
2	刘慧婷, 刘志中, 王利利, 等. 一般间隙序列模式挖掘的关键词抽取[J]. 电子学报, 2019, 47(5): 1121 - 1128.
	LiuH T, LiuZ Z, WangL L, et al. Keyphrase extraction using sequential patterns mining algorithm with one-off and general gaps condition[J]. Acta Electronica Sinica, 2019, 47(5): 1121 - 1128.(in Chinese)
3	MihalceaR, TarauP. TextRank: Bringing order into texts[A]. Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing[C]. Barcelona, Spain: ACL, 2004. 404 - 411.
4	WanX J, XiaoJ G. Single document keyphrase extraction using neighborhood knowledge[A]. Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence[C]. Seattle, Washington: AAAI Press,2008. 855 - 860.
5	LiuZ, HuangW, ZhengY, SunM. Automatic keyphrase extraction via topic decomposition[A]. Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing. Massachusetts[C]. Cambridge, MA: ACL, 2010. 366 - 376.
6	FlorescuC, CarageaC. PositionRank: An unsupervised approach to keyphrase extraction from scholarly documents[A]. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics[C]. Vancouver, Canada: ACL, 2017. 1105 - 1115.
7	马慧芳, 刘芳, 夏琴, 等. 基于加权超图随机游走的文献关键词提取算法[J]. 电子学报, 2018, 46(6): 1410 - 1414.
	MaH F, LiuF, XiaQ, et al. Keywords extraction algorithm based on weighted hypergraph random walk[J]. Acta Electronica Sinica, 2018, 46(6): 1410 - 1414.(in Chinese)
8	BojanowskiP, GraveE, JoulinA, et al. Enriching word vectors with subword information[J]. Transactions of the Association for Computational Linguistics, 2017, 5: 135 - 146.
9	SunY, QiuH P, ZhengY, et al. SIFRank: A new baseline for unsupervised keyphrase extraction based on pre-trained language model[J]. IEEE Access, 2020, 8: 10896 - 10906.
10	PetersM, NeumannM, IyyerM, et al. Deep contextualized word representations[A]. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies[C]. New Orleans, Louisiana: ACL, 2018. 2227 - 2237.
11	SaltonG, BuckleyC. Term-weighting approaches in automatic text retrieval[J]. Information Processing & Management, 1988, 24(5): 513 - 523.
12	LiP, LiuY, SunM. Recursive autoencoders for ITG-based translation[A]. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing[C]. Seattle: ACL, 2013. 567 - 577.
13	WittenI H, PaynterG W, FrankE, et al. KEA: practical automatic keyphrase extraction[A]. Proceedings of the fourth ACM conference on Digital Libraries[C]. Berkeley:ACM, 1999. 254 - 255.
14	MedelyanO, FrankE, WittenI H. Human-competitive tagging using automatic keyphrase extraction[A]. Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing[C]. Singapore: ACL, 2009. 1318 - 1327.
15	MengR, ZhaoS Q, HanS G, et al. Deep keyphrase generation[A]. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics[C]. Vancouver, Canada: ACL, 2017. 582 - 592.

参数类型	LSTMAE	ConvAE
epoch	500	5000
batch_size	128	1000
LSTM_size	200	-
encoder_embedding_size	100	-
decoder_embedding_size	100	-
filter_len	-	2
filter_num	-	200
learning rate	0.1	1.0

参数类型	LSTMAE	ConvAE
epoch	500	5000
batch_size	128	1000
LSTM_size	200	-
encoder_embedding_size	100	-
decoder_embedding_size	100	-
filter_len	-	2
filter_num	-	200
learning rate	0.1	1.0

关键词长度	数目	占比
1	90161	35.4%
2	115322	45.3%
3	33550	13.2%
4	9175	3.6%
5	3042	1.2%
其它	3126	1.2%

关键词长度	数目	占比
1	90161	35.4%
2	115322	45.3%
3	33550	13.2%
4	9175	3.6%
5	3042	1.2%
其它	3126	1.2%

词性	数目	占比
n	252862	50.3%
v	84527	16.8%
vn	30302	6.0%
eng	19338	4.0%
x	16666	3.3%
nz	16272	3.2%
a	13667	2.7%
其它	69218	13.7%

基于短语向量和主题加权的关键词抽取方法

The Theme-Weighted Keyphrase Extraction Algorithm Based on Phrase Embedding

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 15

相关文章 6

编辑推荐

Metrics

关键词抽取算法	准确率P（%）	召回率R（%）	F1值（%）
TFIDF	32.7	38.6	33.4
TextRank	22.99	11.44	15.28
SingleRank	34.8	40.4	35.2
Maui	-	-	4.2
KEA	-	-	12.6
CopyRNN	-	-	34.2
PhraseVecRank-WA	34.6	41.6	37.8
PhraseVecRank-ITG-RAE	34.9	42.1	38.2
PhraseVecRank-LSTMAE	36.0	45.7	40.3
PhraseVecRank-ConvAE	35.7	44.5	39.6

关键词抽取算法	准确率P（%）			召回率R（%）			F1值（%）
关键词抽取算法	Top5	Top10	Top15	Top5	Top10	Top15	Top5	Top10	Top15
TFIDF	10.89	7.17	5.38	12.99	17.08	19.10	11.85	10.10	8.39
TextRank	9.01	6.08	4.67	10.74	14.48	16.59	9.80	8.56	7.28
SingleRank	11.04	7.79	5.69	13.16	18.54	20.32	12.00	10.97	8.89
TopicalPageRank	11.88	8.25	6.23	14.14	19.64	22.25	12.91	11.61	9.73
PhraseVecRank-WA	12.60	9.69	7.57	15.01	23.11	27.08	13.69	13.65	11.83
PhraseVecRank-ITG-RAE	12.69	9.73	7.59	15.14	23.20	27.16	13.81	13.71	11.86
PhraseVecRank-LSTMAE	13.59	10.32	8.16	16.84	25.41	29.53	15.04	14.68	12.79
PhraseVecRank-ConvAE	14.02	10.39	8.30	17.26	25.57	29.91	15.47	14.78	12.99

关键词抽取算法	Top2	Top3	Top5	Top10	Top15
TFIDF	8.12	9.54	11.85	10.10	8.39
TextRank	8.03	9.24	9.80	8.56	7.28
SingleRank	8.22	10.14	12.00	10.97	8.89
TopicalPageRank	8.41	10.53	12.91	11.61	9.73
PhraseVecRank-WA	8.54	11.14	13.69	13.65	11.83
PhraseVecRank-ITG-RAE	8.85	11.45	13.81	13.71	11.86
PhraseVecRank-LSTMAE	10.17	12.74	15.04	14.68	12.79
PhraseVecRank-ConvAE	10.53	12.98	15.47	14.78	12.99

[1]	王雪松, 张翰林, 程玉虎. 基于自编码器和超图的半监督宽度学习系统[J]. 电子学报, 2022, 50(3): 533-539.
[2]	席亮, 刘涵, 樊好义, 张凤斌. 基于深度对抗学习潜在表示分布的异常检测模型[J]. 电子学报, 2021, 49(7): 1257-1265.
[3]	刘慧婷, 刘志中, 王利利, 吴信东. 一般间隙序列模式挖掘的关键词抽取[J]. 电子学报, 2019, 47(5): 1121-1128.
[4]	黄健航, 雷迎科. 基于半监督矩形网络的通信电台个体识别[J]. 电子学报, 2019, 47(1): 1-8.
[5]	付晓, 沈远彤, 付丽华, 杨迪威. 基于特征聚类的稀疏自编码快速算法[J]. 电子学报, 2018, 46(5): 1041-1046.
[6]	罗畅, 王洁, 王鹏飞, 肖军, 肖红. 卷积自编码器中粗粒度池化特征提取研究[J]. 电子学报, 2017, 45(10): 2390-2401.