基于系统模型的用户评论中非功能需求的自动分类

doi:10.12263/DZXB.20210454

摘要/Abstract

摘要：

移动应用程序中的用户评论是获取用户需求的重要来源. 从用户评论中获取的用户需求，不仅可以帮助开发人员维护现有系统，还可以快速、准确地定位新的用户需求. 本文主要关注移动应用用户评论中的非功能需求，并基于系统模型、采用机器学习和深度学习算法将其自动分类为行为型需求和表示型需求. 在使用机器学习方法分类时，将2种特征提取技术与5种机器学习算法进行组合. 在使用深度学习方法分类时，使用了2种基于词嵌入的深度学习算法和1种基于字符嵌入的深度学习算法. 从性能和时间消耗2个维度比较了机器学习模型和深度学习模型，结果表明，机器学习模型比深度学习模型表现更好. 此外，支持向量机（Support Vector Machine，SVM）与词频-逆文档频率（Term Frequency?Inverse Document Frequency，TF?IDF）组合获得了最好的分类性能，精确率为0.941，召回率为0.990，F1-score为0.965.

关键词: 用户评论, 系统模型, 非功能需求, 自动分类, 机器学习, 深度学习

Abstract:

App user reviews are an important source of user requirements. The requirements obtained from user reviews can not only help developers maintain the existing systems, but also quickly and accurately locate new user requirements. This work focuses on non-functional requirements in App user reviews, and further classifies them into behavioral requirements and representational requirements based on system model with machine learning and deep learning algorithms. When using machine learning to classify non-functional requirements, we combined two feature extraction techniques with five machine learning algorithms. When applying deep learning to classify non-functional requirements, we used two deep learning algorithms based on word embedding and one deep learning algorithm based on character embedding. We compared machine learning models and deep learning models from the performance and time consumption perspectives. The results show that, machine learning models perform better than the deep learning models. In addition, the combination of SVM(Support Vector Machine) and TF-IDF(Term Frequency?Inverse Document Frequency) achieves the best performance of classification, with a precision of 0.941, a recall of 0.990, and an F1-score of 0.965.

Key words: user review, system model, non-functional requirement, automatic classification, machine learning, deep learning

中图分类号:

TP311.5

李雪莹, 王田路, 梁鹏, 王翀. 基于系统模型的用户评论中非功能需求的自动分类[J]. 电子学报, 2022, 50(9): 2079-2089.

LI Xue-ying, WANG Tian-lu, LIANG Peng, WANG Chong. Automatic Classification of Non-Functional Requirements in App User Reviews Based on System Model[J]. Acta Electronica Sinica, 2022, 50(9): 2079-2089.

图/表 19

参考文献 20

1	FERNÁNDEZ D M. Supporting requirements-engineering research that industry needs: The NaPiRE initiative[J]. IEEE Software, 2018, 35(1): 112-116.
2	BROY M. Rethinking Functional Requirements: A Novel Approach Categorizing System and Software Requirements[M]//Software Technology: 10 Years of Innovation in IEEE Computer. Hoboken, NJ, USA: John Wiley & Sons, Inc, 2018: 155-187.
3	BROY M. Rethinking nonfunctional software requirements[J]. Computer, 2015, 48(5): 96-99.
4	ECKHARDT J. Categorizations of Product-related Requirements in Practice[D]. München: Technische Universität München, 2017.
5	LU M M, LIANG P. Automatic classification of non-functional requirements from augmented app user reviews[C]//EASE'17: Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering. New York: ACM, 2017: 344-353.
6	Joint Technical Committee ISO/IEC [S/OL].[2021-04-07]..
7	李雪莹,王田路,梁鹏,王翀. 基于系统模型的用户评论中非功能需求的自动分类[EB/OL]. [2021-04-07] .
8	GLINZ M. On non-functional requirements[C]//15th IEEE International Requirements Engineering Conference. Piscataway: IEEE, 2007: 21-26.
9	ABAD Z S H, KARRAS O, GHAZI P, et al. What works better? A study of classifying requirements[C]//2017 IEEE 25th International Requirements Engineering Conference. Piscataway: IEEE, 2017: 496-501.
10	LI C Y, HUANG L G, GE J D, et al. Automatically classifying user requests in crowdsourcing requirements engineering[J]. Journal of Systems and Software, 2018, 138: 108-123.
11	STANIK C, HAERING M, MAALEJ W. Classifying multilingual user feedback using traditional machine learning and deep learning[C]//2019 IEEE 27th International Requirements Engineering Conference Workshops. Piscataway: IEEE, 2019: 220-226.
12	JHA N, MAHMOUD A. Mining non-functional requirements from App store reviews[J].Empirical Software Engineering, 2019, 24(6): 3659-3695.
13	WANG T L, LIANG P, LU M M. What aspects do non-functional requirements in app user reviews describe? an exploratory and comparative study[C]//2018 25th Asia-Pacific Software Engineering Conference(APSEC). Piscataway: IEEE, 2018: 494-503.
14	MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space[EB/OL]. (2013-01-16). .
15	BOJANOWSKI P, GRAVE E, JOULIN A, et al. Enriching word vectors with subword information[J]. Transactions of the Association for Computational Linguistics, 2017, 5: 135-146.
16	KIM Y. Convolutional neural networks for sentence classification[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg, PA, USA: Association for Computational Linguistics, 2014: 1746-1751.
17	LAI S W, XU L H, LIU K, et al. Recurrent convolutional neural networks for text classification[C]//AAAI'15: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence. Menlo Park, CA: AAAI, 2015: 2267-2273.
18	ZHANG X, ZHAO J, LECUN Y. Character-level convolutional networks for text classification[C]// Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1. Cambridge, MA: MIT Press, 2015: 649－657.
19	COHEN J. A coefficient of agreement for nominal scales[J]. Educational and Psychological Measurement, 1960, 20(1): 37-46.
20	SHULL F, SINGER J, SJØBERG D. I. Guide to Advanced Empirical Software Engineering[M]. Berlin, German: Springer, 2008.

Hyper-Parameter	Value
Dimension of word vector	200
Length of word sequence	200
Number of convolution kernel	128
Size of convolution kernel	[3~5]
Learning rate	0.001
Batch size	64
Epoch	10

Hyper-Parameter	Value
Dimension of word vector	200
Length of word sequence	200
Number of convolution kernel	128
Size of convolution kernel	[3~5]
Learning rate	0.001
Batch size	64
Epoch	10

Hyper-parameter	Value
Dimension of word vector	200
Length of word sequence	200
Size of hidden layer	128
Learning rate	0.001
Batch size	64
Epoch	10

Hyper-parameter	Value
Dimension of word vector	200
Length of word sequence	200
Size of hidden layer	128
Learning rate	0.001
Batch size	64
Epoch	10

Hyper-Parameter	Value
Dimension of character vector	70
Length of character sequence	1014
Size of convolution and pooling layers	([256,7,3], [256,7,3], [256,3,None], [256,3,None], [256,3,None], [256,3,3])
Size of fully-connected	[1024,1024,2]
Learning rate	0.001
Batch size	64
Epoch	10