基于关键用例获取的测试用例排序方法

doi:10.12263/DZXB.20201284

电子学报 ›› 2022, Vol. 50 ›› Issue (1): 149-156.DOI: 10.12263/DZXB.20201284

基于关键用例获取的测试用例排序方法

范书平¹, 万里², 姚念民³, 张岩⁴, 马宝英⁵

^1.牡丹江师范学院计算机与信息技术学院，黑龙江牡丹江 157012
^2.天津大学智能与计算学部，天津 300350
^3.大连理工大学计算机科学与技术学院，辽宁大连 116024
^4.宿迁学院信息工程学院，江苏宿迁 223800
^5.牡丹江医学院卫生管理学院，黑龙江牡丹江 157011

收稿日期:2020-11-17 修回日期:2021-02-24 出版日期:2022-01-25
- 通讯作者:
- 马宝英
- 作者简介:
- 范书平男，1977年生，黑龙江牡丹江人.副教授，主要从事复杂软件的测试数据生成和进化计算方面的研究工作. E-mail:f8259@163.com
  万里男，1994年生，湖北荆州人.毕业于天津大学，主要从事软件测试、语音信号处理方面的研究工作.
  姚念民男，1974年生，黑龙江大庆人.教授，主要从事基于搜索的软件工程方面的研究工作.
  张岩女，1972年生，辽宁本溪人.教授，CCF会员，主要从事基于搜索的软件工程、进化计算方面的研究工作.
  马宝英（通信作者）女，1985年生，黑龙江鸡西人.讲师，主要从事复杂软件的测试数据生成方面的研究工作.
- 基金资助:
- 国家自然科学基金 (2018AAA0100300); 黑龙江省自然科学基金 (LH2021F055); 牡丹江市应用技术研究与开发计划 (HT2020JG049); 大连市科技创新项目 (2018J12GX045)

Test Case Sorting Method Based on Key Use Cases Extracted

FAN Shu-ping¹, WAN Li², YAO Nian-min³, ZHANG Yan⁴, MA Bao-ying⁵

^1.School of Computer and Information Technology, Mudanjiang Normal University, Mudanjiang, Heilongjiang 157012, China
^2.Department of Intelligence and Computing, Tianjin University, Tianjin 300350, China
^3.School of Computer Science and Technology, Dalian University of Technology, Dalian, Liaoning 116024, China
^4.School of Information Engineering, Suqian University, Suqian, Jiangshu 223800, China
^5.School of Health Management, Mudanjiang Medical University, Mudanjiang, Heilongjiang 157011, China

Received:2020-11-17 Revised:2021-02-24 Online:2022-01-25 Published:2022-01-25
- Corresponding author:
- MA Bao-ying
- Supported by:
- National Natural Science Foundation of China (2018AAA0100300); Natural Science Foundation of Heilongjiang Province, China (LH2021F055); Mudanjiang Applied Technology Research and Development Project (HT2020JG049); Dalian Science and Technology Innovation Project (2018J12GX045)

摘要/Abstract

摘要：

测试用例排序技术通过在测试过程中确定用例执行的先后次序来增加早期揭示缺陷的可能性.本文在用例排序过程中动态提取关键用例，在测试用例运行被测程序后，得到各候选用例（待排序用例）覆盖程序新分支情况以及改善用例覆盖程序分支的均衡程度，进而计算候选用例的权重使得关键用例的权重增加，优先进行排序.实验结果表明，与现有方法比较，所提方法在用例排序时间与缺陷检测方面体现出较好的性能.

关键词: 回归测试, 测试用例排序, 用例偏离度, 缺陷检测, 分支覆盖

Abstract:

Test case sorting increases the possibility of early discovery of defects by determining the execution order of test cases in the testing process. In the paper, the key test cases are dynamically extracted during the test case sorting process. After test cases running the program under test, the candidate test cases (test cases to be sorted) covering the new branches of the program and improving the balance of test case covering program branches are obtained, and then the weight of the candidate test cases are calculated so that the weight of key test cases is increased, and prioritize them. Comparing with existing methods, experimental results show that the proposed method shows better performance in the time consumed and fault detection of test case sorting.

Key words: regression testing, test case sorting, use case deviation, fault detection, branches coverage

中图分类号:

TP311

范书平, 万里, 姚念民, 张岩, 马宝英. 基于关键用例获取的测试用例排序方法[J]. 电子学报, 2022, 50(1): 149-156.

FAN Shu-ping, WAN Li, YAO Nian-min, ZHANG Yan, MA Bao-ying. Test Case Sorting Method Based on Key Use Cases Extracted[J]. Acta Electronica Sinica, 2022, 50(1): 149-156.

图/表 5

参考文献 26

1	MUKHERJEER, PATNAIKK S. A survey on different approaches for software test case prioritization[J]. Journal of King Saud University-Computer and Information Sciences, 2018, 33(9): 1041-1054.
2	MARCHETTOA, SCANNIELLOG, SUSIA. Combining code and requirements coverage with execution cost for test suite reduction[J]. IEEE Transactions on Software Engineering, 2019, 45(4): 363-390.
3	HENARDC, PAPADAKISM, HARMANM, et al. Comparing white-box and black-box test prioritization[C]//IEEE/ACM International Conference on Software Engineering. New York: IEEE, 2016: 523-534.
4	李英玲, 王青. 持续集成测试用例集优化综述研究[J]. 软件学报, 2018, 29(10): 3021-3050.
	LIY L, WANGQ. Test set optimization in continuous integration: A systematic literature review[J]. Journal of Software, 2018, 29(10): 3021-3050. (in Chinese)
5	TAHATL, KORELB, KOUTSOGIANNAKISG, et al. State-based models in regression test suite prioritization[J]. Software Quality Journal, 2017, 25: 1-40.
6	FANGC R, CHENZ Y, XUB W. Comparing logic coverage criteria on test case prioritization[J]. Science China Information Sciences, 2012, (12): 2826-2840.
7	CHENZ Y, DUANY W, ZHAOZ H, et al. Using program slicing to improve the efficiency and effectiveness of cluster test selection[J]. International Journal of Software Engineering and Knowledge Engineering, 2011, 21(6), 759-777.
8	NOORT B, HEMMATIH. A similarity-based approach for test case prioritization using historical failure data[C]//IEEE 26th International Symposium on Software Reliability Engineering(ISSRE). New York: IEEE, 2015: 58-68.
9	潘伟丰, 李兵, 马于涛, 等. 基于复杂软件网络的回归测试用例优先级排序[J]. 电子学报, 2012, 40(12), 2456-2465.
	PANW F, LIB, MAY T, et al. Test case prioritization based on complex software networks[J]. Acta Electronica Sinica, 2012, 40(12): 2456-2465. (in Chinese)
10	张卫祥, 魏波, 杜会森. 一种基于遗传算法的测试用例优先排序方法[J]. 小型微型计算机系统, 2015, 36(9): 1998-2002.
	ZHANGW X, WEIB, DUH S. Test case prioritization method based on genetic algorithm[J]. Journal of Chinese Computer Systems, 2015, 36(9): 1998-2002. (in Chinese)
11	石宇楠, 李征, 龚沛. 基于多目标协同进化的测试用例优先排序[J]. 计算机科学, 2015, 42(12): 124-129.
	SHIY N, LIZ, GONGP. Multi-objective coevolutionary test case prioritization[J]. Computer Science, 2015, 42(12): 124-129. (in Chinese)
12	NAYAKG, RAY M. Modified condition decision coverage criteria for test suite prioritization using particle swarm optimization[J]. International Journal of Intelligent Computing and Cybernetics, 2019, 12(4): 425-443.
13	HAOD, ZHANGL M, ZHANGL, et al. A unified test case prioritization approach[J]. ACM Transactions on Software Engineering and Methodology, 2014, 24(2): 1-31.
14	KAVITHA, SURESHKUMARN. Test case prioritization for regression testing based on severity of fault[J]. International Journal on Computer Science & Engineering, 2010, 2(5): 1462-1466.
15	ROTHERMELG, UNTCHR H, CHUC Y, et al. Prioritizing test cases for regression testing[J]. IEEE Transactions on Software Engineering, 2001, 27(10): 929-948.
16	ELBAUMS, MALISHEVSKYA G, ROTHERMELG. Test case prioritization: A family of empirical studies[J]. IEEE Transactions on Software Engineering, 2002, 28(2): 159-182.
17	HAOD, ZHANGL, MEIH. Test-case prioritization: Achievements and challenges[J]. Frontiers of Computer Science, 2016, 10: 769-777.
18	张娜, 姚澜, 包晓安, 等. 多目标优化的测试用例优先级在线调整策略[J]. 软件学报, 2015, 26(10): 2451-2464.
	ZHANGN, YAOL, BAOX A, et al. Multi-objective optimization based on-line adjustment strategy of test case prioritization[J]. Journal of Software, 2015, 26(10): 2451-2464. (in Chinese)
19	JIANGB, ZHANGZ Y, CHANW K, et al. Adaptive random test case prioritization[C]//Proceedings of the 24th International Conference on Automated Software Engineering(ASE'09). Auckland: IEEE, 2009: 233-244.
20	陈梦云, 高建华. 基于圈复杂度的静态测试用例排序方法 [J]. 计算机应用与软件, 2016, 33(1): 1-3, 15.
	CHENM Y, GAOJ H. Static test cases sorting method based on cyclomatic complexity[J]. Computer Applications and Software, 2016, 3(1): 1-3, 15. (in Chinese)
21	FANGC R, CHENZ Y, WuK, et al. Similarity-based test case prioritization using ordered sequences of program entities[J]. Software Quality Control, 2014, 22(2): 335-361.
22	JEFFREYD, GUPTAN. Experiments with test case prioritization using relevant slices[J]. Journal of Systems & Software, 2008, 81(2): 196-221.
23	MAHDIEHM, MIRIAN-HOSSEINABADIS H, ETEM-ADI K, et al. Incorporating fault-proneness estimations into coverage-based test case prioritization methods[J]. Information and Software Technology, 2020, 121: 106269.
24	HUANGR B, ZHANGQ J, TOWEYD, et al. Regression test case prioritization by code combinations coverage[J]. Journal of Systems and Software, 2020, 169: 110712.
25	CHIJ L, QUY, ZHENGQ H, et al. Relation-based test case prioritization for regression testing[J]. Journal of Systems and Software, 2020, 163: 110539.
26	范书平, 张岩, 马宝英, 等. 基于均衡优化理论的路径覆盖测试数据进化生成[J]. 电子学报, 2020, 48(7): 1303-1310.
	FANS P, ZHANGY, MAB Y, et al. Evolutionary generation of path coverage test data based on equilibrium optimization theory[J]. Acta Electronica Sinica, 2020, 48(7): 1303-1310. (in Chinese)

Programs	LOC	Faults	Test cases
space（fixgramp）	95	12	100
space（fixsgrid）	126	11	41
tot_info	406	20	450
replace	564	25	640
flex	1103	300	3000
sed	1762	200	2000

Programs	LOC	Faults	Test cases
space（fixgramp）	95	12	100
space（fixsgrid）	126	11	41
tot_info	406	20	450
replace	564	25	640
flex	1103	300	3000
sed	1762	200	2000

Programs	本文方法		BED		SED		AGA
Programs	运行时间/ms	用例/%	运行时间/ms	用例/%	运行时间/ms	用例/%	运行时间/ms	用例/%
space（fixgramp）	3.40	28.71	56.14	50.27	102.51	49.57	1.12	61.94
space（fixsgrid）	2.30	21.95	92.00	40.98	94.35	42.07	0.75	77.31
tot_info	26.14	28.67	330.52	48.52	382.51	50.76	10.52	69.42
replace	38.41	30.25	548.24	52.36	486.32	51.39	16.38	72.15
sed	98.64	45.15	531.25	55.36	541.68	66.52	419.61	69.25
flex	79.83	38.26	466.81	49.72	529.32	53.64	327.12	67.33

Programs	本文方法		BED		SED		AGA
Programs	运行时间/ms	用例/%	运行时间/ms	用例/%	运行时间/ms	用例/%	运行时间/ms	用例/%
space（fixgramp）	3.40	28.71	56.14	50.27	102.51	49.57	1.12	61.94
space（fixsgrid）	2.30	21.95	92.00	40.98	94.35	42.07	0.75	77.31
tot_info	26.14	28.67	330.52	48.52	382.51	50.76	10.52	69.42
replace	38.41	30.25	548.24	52.36	486.32	51.39	16.38	72.15
sed	98.64	45.15	531.25	55.36	541.68	66.52	419.61	69.25
flex	79.83	38.26	466.81	49.72	529.32	53.64	327.12	67.33

Programs	均衡性/%
Programs	本文方法	BED	SED	AGA
space（fixgramp）	20.15	8.54	9.63	16.72
space（fixsgrid）	22.33	13.33	19.21	15.54
tot_info	34.16	22.57	19.38	26.31
replace	24.89	9.64	12.18	16.71
sed	28.34	11.35	14.28	20.83
flex	31.20	16.32	19.11	23.95

基于关键用例获取的测试用例排序方法

Test Case Sorting Method Based on Key Use Cases Extracted

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 5

参考文献 26

相关文章 7

编辑推荐

Metrics

[1]	钱忠胜, 宋佳, 俞情媛, 成轶伟, 孙志旺. 利用函数影响力的相似程序间测试用例重用与生成[J]. 电子学报, 2022, 50(7): 1696-1707.
[2]	师奕兵, 罗清旺, 王志刚, 张伟, 马东. 基于多元接收线圈的管道局部缺陷检测方法研究[J]. 电子学报, 2018, 46(1): 197-202.
[3]	马森, 赵文, 习翔宇, 王栋伟. 基于值依赖分析的空指针解引用检测[J]. 电子学报, 2015, 43(4): 647-651.
[4]	董玉坤, 宫云战, 金大海. 基于区域内存模型的空指针引用缺陷检测[J]. 电子学报, 2014, 42(9): 1744-1752.
[5]	禹振, 苏小红, 王甜甜, 马培军. C程序隐式规则自动提取与反例检测[J]. 电子学报, 2013, 41(2): 248-254.
[6]	潘伟丰, 李兵, 马于涛, 刘婧. 基于复杂软件网络的回归测试用例优先级排序[J]. 电子学报, 2012, 40(12): 2456-2465.
[7]	张志华;牟永敏. 基于函数调用的路径覆盖生成技术研究[J]. 电子学报, 2010, 38(8): 1808-1811.