面向开放域问答的问题分类技术研究进展

doi:10.3969/j.issn.0372-2112.2015.08.024

摘要
图/表
参考文献(0)
相关文章 (15)

全文: PDF (385 KB) HTML (1 KB)
输出: BibTeX | EndNote (RIS)

摘要

开放域问答是当前自然语言处理和信息检索领域的研究热点,作为开放域问答系统的重要组成部分,问题分类可以缩小答案的搜索空间并决定答案的选择策略.近年来,基于机器学习的问题分类技术受到广泛的关注,相关研究表明问题分类的准确性直接影响问答系统的整体性能.本文从分类体系与数据集、特征提取、分类器设计、性能评测等层面,总结了问题分类技术近年的主要研究成果.重点分析了各种基于监督学习的问题分类方法的特点和不足,讨论了核方法、半监督学习、主动学习、迁移学习等在问题分类中的应用,同时对问题分类技术未来研究动向进行了展望.

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	杨思春
	戴新宇
	陈家骏

关键词 ：开放域问答, 问题分类, 机器学习, 特征提取, 分类器设计

Abstract：

Open-domain question answering is becoming a hot topic in the fields of natural language processing and information retrieval.Question classification, as an important component of question answering, has shown its significant influence on the overall performance of question answering systems.It can help reduce the search space and choose the exact search strategy to find answers.In this paper, we present a through overview of the state-of-the-art approaches to question classification, in terms of category/dataset, feature extraction, classification methods and performance metrics.Firstly, we give a detailed analysis of the supervised learning based question classification approaches.Then, we introduce some related work on question classification, such as kernel methods, semi-supervised learning methods, active learning and transfer learning methods, and so on.Finally, we give some possible research directions on question classification.

Key words： open-domain question answering question classification machine learning feature extraction classifier design

收稿日期: 2014-07-21

ZTFLH:

TP391

基金资助:

国家自然科学基金(No.61472183,No.61170181)

通讯作者: 戴新宇 E-mail: daixinyu@nlp.nju.edu.cn

作者简介: 杨思春 男,1970年生于安徽六安.博士,副教授.研究方向为自然语言处理、自动问答. E-mail:yangsc@nlp.nju.edu.cn

引用本文:

杨思春, 戴新宇, 陈家骏. 面向开放域问答的问题分类技术研究进展[J]. 电子学报, 2015, 43(8): 1627-1636. YANG Si-chun, DAI Xin-yu, CHEN Jia-jun. Advances in Question Classification for Open-Domain Question Answering. Chinese Journal of Electronics, 2015, 43(8): 1627-1636.

链接本文:

http://www.ejournal.org.cn/CN/10.3969/j.issn.0372-2112.2015.08.024 或 http://www.ejournal.org.cn/CN/Y2015/V43/I8/1627

[1] 张志昌,张宇,等.开放域问答技术研究进展[J].电子学报,2009,37(5):1058-1069. Zhang Zhichang,Zhang Yu,et al.Advances in open-domain question answering[J].Acta Electronica Sinica,2009,37(5):1058-1069.(in Chinese)
[2] 范士喜,王晓龙.面向真实环境的问句分析方法[J].电子学报,2010,38(5):113-1135. Fan Shixi,Wang Xiaolong.Real environment oriented question analyzing[J].Acta Electronica Sinica,2010,38(5):1131-1135.(in Chinese)
[3] 高明霞,刘椿年.基于约束的自然语言问题到OWL的语义映射方法研究[J].电子学报,2007,35(8):1598-1602. Gao Mingxia,Liu Chunnian.A constraints-based semantic mapping method from natural language questions to OWL[J].Acta Electronica Sinica,2007,35(8):1598-1602.(in Chinese)
[4] Moldovan D,et al.Performance issues and error analysis in an open-domain question answering system[J].ACM Transactions on Information Systems,2003,21(2):133-154
[5] Li X,et al.Learning question classifiers[A].Proc of the 19th International Conference on Computational Linguistics[C].Taipei,China:Association for Computational Linguistics,2002.1-7.
[6] Li X,et al.Learning question classifiers:the role of seman-tic information[J].Natural Language Engineering,2006,12 (3):229-249.
[7] Zhang D,et al.Question classification using support vector machines[A].Proc of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval[C].Toronto,Canada:ACM,2003.26-32.
[8] Huang Z,et al.Question classification using head words and their hypernyms[A].Proc of the 2008 Conference on Empirical Methods on Natural Language Processing[C].Honolulu,USA:Association for Computational Linguistics,2008.927-936.
[9] Huang Z,et al.Investigation of question classifier in ques-tion answering[A].Proc of the 2009 Conference on Empirical Methods in Natural Language Processing[C].Singapore:Association for Computational Linguistics,2009.543-550.
[10] Bu F,et al.Function-based question classification for general QA[A].Proc of the 2010 Conference on Empirical Methods in Natural Language Processing[C].Massachusetts,USA:Association for Computational Linguistics,2010.1119-1128.
[11] 邱锡鹏,缪有栋,等.基于主动学习的中文问题分类数据集构建[J].哈尔滨工业大学学报,2012,44(5):125-128. Qiu Xipeng,Miao Youdong,et al.Constructing Chinese question classification dataset with active learning[J].Journal of Harbin Institute of Technology,2012,44(5):125-128.(in Chinese)
[12] 刘小明,樊孝忠,等.一种结合本体和焦点的问题分类方法[J].北京理工大学学报,2012,32(5):498-502. LiuXiaoming,Fan Xiaozhong,et al.A question classification method combining domain ontology and question focus[J].Transactions of Beijing Institute of Technology,2012,32 (5):498-502.(in Chinese)
[13] Suzuki J,et al.Hierarchical directed acyclic graph kernel:methods for structured natural language data[A].Proc of the 41th Annual Meeting of the Association for Computational Linguistics[C].Sapporo,Japan:Association for Computation-al Linguistics,2003.32-39.
[14] Wu Y Z,et al.Chinese question classification from approach and semantic views[A].Proc of the Second Asia Information Retrieval Symposium[C].Jeju Island,Korea:Asia Information Retrieval Society,2005.485-490.
[15] Pinchak C,et al.A probabilistic answer type model[A].Proc of the 11th Conference of the European Chapter of the Association for Computational Linguistics[C].Trento,Italy:Association for Computational Linguistics,2006.393-400.
[16] Mikhailian A,et al.Learning foci for question answering over topic maps[A].Proc of the ACL-IJCNLP 2009 Conference[C].Suntec,Singapore:Association for Computational Linguistics,Asian Federation of Natural Language Processing,2009.325-328.
[17] 张宇,刘挺,等.基于改进贝叶斯模型的问题分类[J].中文信息学报,2005,19(2):100-105. Zhang Yu,Liu Ting,et al.Modified bayesian model based question classification[J].Journal of Chinese Information Processing,2005,19(2):100-105.(in Chinese)
[18] 文勖,张宇,等.基于句法结构分析的中文问题分类[J].中文信息学报,2006,20(2):33-39. Wen Xu,Zhang Yu,et al.Syntactic structure parsing based Chinese question classification[J].Journal of Chinese Infor-mation Processing,2006,20(2):33-39.(in Chinese)
[19] 孙景广,蔡东风,等.基于《知网》的中文问题自动分类[J].中文信息学报,2007,21(1):90-96. SunJingguang,Cai Dongfeng,et al.HowNet Based Chinese Question Automatic Classification[J].Journal of Chinese Information Processing,2007,21(1):90-96.(in Chinese)
[20] Blunsom P,et al.Question classification with loglinear models[A].Proc of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval[C].New York,USA:ACM,2006.615-616.
[21] Zhang Z,et al.Automatic recognition of focus and interro-gative word in Chinese question for classification[J].Comp-uter and Information Science,2010,3(1):168-174.
[22] Silva J,et al.From symbolic to sub-symbolic information in question classification[J].Artificial Intelligence Review,2011,35(2):137-154.
[23] Loni B,et al.Question classification with weighted combination of lexical,syntactical and semantic features[A].Proc of the 14th International Conference of Text,Speech and Dialog[C].Pilsen,Czech Republic:Springer,2011.243-250.
[24] Li F T,et al.Classifying what-type questions by head noun tagging[A].Proc of the 22th International Conference on Computational Linguistics[C].Manchester,UK:Association for Computational Ling uistics,2008.481-488.
[25] Bunescu R,et al.Towards a general model of answer typing:question focus identification[J].Special issue:Natural Language Processing and its Applications.Research in Com-puting Science,2010,46:231-242.
[26] Nguyen M L,et al.Subtree mining for question classification problem[A].Proc of the 20th International Joint Conference on Artificial Intelligence[C].Hyderabad,India:AA AI Press,2007:1695-1700.
[27] 李鑫,黄萱菁,等.基于错误驱动算法组合分类器及其在问题分类中的应用[J].计算机研究与发展,2008,45(3):535-541. Li Xin,Huang Xuanjing,et al.Combined multiple classifiers based on TBL algorithm and their application in question classification[J].Journal of Computer Research and Develop-ment,2008,45(3):535-541.(in Chinese)
[28] Fellbaum C.WordNet:an electronic lexical database[M].Cambridge:MIT Press,1998.
[29] 董振东,董强.《知网》[EB/OL].http://www.keenage.com /html/ c_index.html,2014-07-16.
[30] 余正涛,樊孝忠,等.基于支持向量机的汉语问题分类[J].华南理工大学学报(自然科学版),2005,33(9):25-29. Yu Zhengtao,Fan Xiaozhong,et al.Chinese question classifier based on support vector machine[J].Journal of South China University of Technology(Natural Science Edition),2005,33(9):25-29.(in Chinese)
[31] Metzler D,et al.Analysis of statistical question classification for factbased questions[J].Information Retrieval,20 05,8:481-504.
[32] Ray S K,et al.A semantic approach for question classifica-tion using WordNet and Wikipedia[J].Pattern Recognition Letters,2010,31(13):1935-1943.
[33] 李方涛,张显,等.一种新的层次化结构问题分类器[J].中文信息学报,2008,22(1):93-98. Li Fangtao,Zhang Xian,et al.A novel hierarchical structure question classifier[J].Journal of Chinese Information Proce-ssing,2008,22(1):93-98.(in Chinese)
[34] 张志昌,张宇,等.基于线索词识别和训练集扩展的中文问题分类[J].高技术通讯,2009,19(2):111-118. Zhang Zhichang,Zhang Yu,et al.Chinese question classifi-cation based on identification of cue words and extension of training set[J].Chinese High Technology Letters,2009,19(2):111-118.(in Chinese)
[35] Banerjee S,et al.An empirical study of combining multiple models in Bengali question classification[A].Proc of the International Joint Conference on Natural Language Processing[C].Nagoya,Japan:Asian Federation of Natural Language Processing,2013.892-896.
[36] Manning C D,et al.Optimization,maxent models,and conditional estimation without magic[A].Proc of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology[C].Tutorial,Edmonton,Canada:Association for Computational Linguistics,2003.8-8.
[37] Vapnik V.The nature of statistical learning theory[M].New York:Springer-Verlag,1995.
[38] 张学工.关于统计学习理论与支持向量机[J].自动化学报,2000,26(1):32-42. Zhang Xuegong.Introduction to statistical learning theory and support vector machines[J].Acta Automatica Sinica,2000,26(1):32-42.(in Chinese)
[39] 车万翔.基于核方法的语义角色标注研究[D].哈尔滨:哈尔滨工业大学,2008.
[40] Haussler D.Convolution kernels on discrete structures[R].Santa Cruz:Department of Computer Science,University of California in Santa Cruz,1999.
[41] Moschitti A,et al.Exploiting syntactic and shallow semantic kernels for question answer classification[A].Proc of the 45th Annual Meeting on Association for Computational Linguistics[C].Prague,Czech Republic:Association for Computational Linguistics,2007.776-783.
[42] Liu L,et al.Chinese question classification based on ques-tion property kernel[J].International Journal of Machine Learning and Cybernetics,2013,11:1-8.
[43] Moschitti A,et al.Using syntactic and semantic structural kernels for classifying definition questions in Jeopardy![A].Proc of the 2011 Conference on Empirical Methods in Natural Language Processing[C].Edinburgh,Scotland,UK:Asso-ciation for Computational Linguistics,2011.712-724.
[44] Blum A,et al.Combining labeled and unlabeled data with cotraining[A].Proc of the 11th Annual Conference on Computational Learning Theory[C].Wisconsin,USA:ACM,1998.92-100.
[45] Zhou Z H,et al.Tri-training:exploiting unlabeled data using three classifiers[J].IEEE Transactions on Knowledge and Data Engineering,2005,17(11):1529-1541.
[46] Yu Z T,et al.Question classification based on cotraining style semi-supervised learning[J].Pattern Recognition Letters,2010,31:1975-1980.
[47] Tri T N,et al.Using semi-supervised learning for question classification[J].Information and Media Technologies.2008,3(1):112-130.
[48] Mishra T,et al.Qme!:A speech-based question-answering system on mobile devices[A].Proc of the 2010 Annual Con ference of the North American Chapter of the Association for Computational Linguistics[C].Los Angeles,USA:Association for Computational Linguistics,2010.55–63.
[49] Marin-i-D,et al.Question classification with active learning[A].Proc of 15th International Conference on Text,Speech and Dialogue[C].LNCS 7499,Springer,2012.673-680.
[50] Su L,et al.Domain adaptation for question classification[J].Journal of Computational Information Systems,2011,7(9):32 61-3267.
[51] 李鹏,王晓龙,等.一种基于粗糙集增量式规则学习的问题分类方法研究[J].电子信息学报,2008,30(5):1127-1130. Li Peng,Wang Xiaolong,et al.Question classification with incremental rule learning algorithm based on rough set[J].Journal of Electronics and Information Technology,2008,30 (5):1127-1130.(in Chinese)
[52] Loni B.A survey of state-of-the-art methods on question classification[R].Delft:TU Delft Repository,2011.
[53] 李茹,宋小香,等.基于汉语框架网的中文问题分类[J].计算机工程与应用,2009,45(31):111-115. Li Ru,Song Xiaoxiang,et al.Chinese question classification based on Chinese FrameNet[J].Computer Engineering and Applications,2009,45(31):111-115.(in Chinese)
[54] 杨思春,高超,等.基于差异性和重要性的问句特征组合[J].电子学报,2014,42(5):918-924. Yang Sichun,Gao Chao,et al.Combining Features of Question Based on Diversity and Importance[J].Acta Electronica Sinica,2014,42(5):918-924.(in Chinese)
[55] Collobert R,et al.Natural language processing(almost) from scratch[J].The Jounal of Machine Learning Research,2011,12:2493-2537.
[56] Zeng D,et al.Relation classification via convolutional deep neural network[A].Proc of the 25th International Con ference on Computational Linguistics[C].Dubin,Ireland:Association for Computational Linguistics,2014.2335-2344.
[57] Zhou G,et al.Exploiting bilingual translation for question retrieval in community-based question answering[A].Proc of the 23th International Conference on Computational Ling-uistics[C].Mumbai,India:Association for Computational Linguistics,2012.3153-3170.
[58] Zhou G,et al.Group non-negative matrix factorization with natural categories for question retrieval in community question answer archives[A].Proc of the 25th International Conference on Computational Linguistics[C].Dublin,Ireland:Association for Computational Linguistics,2014.89-98.
[59] Singh A,et al.CQC:Classifying questions in CQA websites[A].Proc of the 20th ACM International Conference on Information and Knowledge Management[C].Glasgow,Scotland,UK:ACM,2011.2033-2036.
[60] Chan W,et al.Community answer summarization for multi-sentence question with group L1 regularization[A].Proc of the 50th Annual Meeting of the Association for Computational Linguistics[C].Jeju,Korea:Association for Computational Linguistics,2012.582-591.
[61] Bao J,et al.Knowledge-based question answering as machine translation[A].Proc of the 52th Annual Meeting of the Association for Computational Linguistics[C].Baltimore,Maryland:Association for Computational Linguistics,2014.967-976.
[62] Zhao S,et al.Tailor knowledge graph for query under-standing:linking intent topics by propagation[A].Proc of the 2014 Conference on Empirical Methods in Natural Language Processing[C].Doha,Qatar:Association for Computational Linguistics,2014.1070-1080.