基于复杂软件网络的回归测试用例优先级排序

PDF(2989 KB)

电子学报 ›› 2012, Vol. 40 ›› Issue (12) : 2456-2465.

学术论文

基于复杂软件网络的回归测试用例优先级排序

潘伟丰¹, 李兵^2,3, 马于涛^2,3, 刘婧^2,3

作者信息 +

Test Case Prioritization Based on Complex Software Networks

PAN Wei-feng¹, LI Bing^2,3, MA Yu-tao^2,3, LIU Jing^2,3

Author information +

文章历史 +

摘要

测试用例优先级排序技术是一种高效实用的回归测试技术.针对现有排序方法未能有效利用软件结构复杂性信息的不足,提出了一种基于复杂软件网络的回归测试用例优先级排序方法.该方法用加权类依赖网络模型抽象类粒度软件系统,从结构角度测度类引入错误的可能性及错误的严重性,并据此评价类的测试重要性,同时结合测试用例的覆盖信息,对测试用例进行排序.实例研究表明,本文的方法能够显著提高严重错误的发现速率,并且具有较好的稳定性.同时,揭示了方法有效性与软件错误类型和结构度量指标两者间的关系.

Abstract

Test case prioritization is one of the effective and practical regression testing techniques.This paper proposed a new test case prioritization technique which takes the software structure complexity that has usually been neglected in previous work into consideration.It used weighted class dependency networks to represent software at the class level.It defined fault proneness index of classes to describe the fault-introduced probability.It defined fault propagation ratio of classes to describe the fault severity.Then test attention of classes was introduced to depict the testing importance of classes,and based on which,together with the coverage information of each test case,the priority of each test case was determined.The empirical results on several software systems suggest that the proposed approach is more effective than the other test case prioritization techniques and has relatively high stability.And the relationships between the effectiveness of the proposed approach and two factors (i.e.fault types and structural complexity metrics) were also uncovered.

导出引用

潘伟丰, 李兵, 马于涛, 刘婧. 基于复杂软件网络的回归测试用例优先级排序[J]. 电子学报, 2012, 40(12): 2456-2465.

PAN Wei-feng, LI Bing, MA Yu-tao, LIU Jing. Test Case Prioritization Based on Complex Software Networks[J]. Acta Electronica Sinica, 2012, 40(12): 2456-2465.

中图分类号： TP311.5

参考文献

[1] Rothermel G,Untch R H,Chu C Y,et al.Prioritizing test cases for regression testing [J].IEEE Transactions on Software Engineering,2001,27(10):929-948.
[2] Kim J M,Porter A.A history-based test prioritization technique for regression testing in resource constrained environments .Proceedings of the 24th International Conference on Software Engineering .New York,USA:ACM,2002.119-129.
[3] Srikanth H,Williams L,Osborne J.System test case prioritization of new and regression test cases .Proceedings of the 2005 International Symposium on Empirical Software Engineering .Washington D C,USA:IEEE,2005.62-71.
[4] Li Z,Harman M,Hierons R M.Search algorithm for regression test case prioritization [J].IEEE Transactions on Software Engineering,2007,33(4):225-237.
[5] Elbaum S,Malishevsky A G,Rothermel G.Test case prioritization:A family of empirical studies [J].IEEE Transactions on Software Engineering,2002,28(2):159-182.
[6] Do H,Mirarab S,Tahvildari L,et al.The effects of time constraints on test case prioritization:A series of controlled experiments [J].IEEE Transactions on Software Engineering,2010,36(5):593-617.
[7] Yau S S,Collofello J S.Some stability measures for software maintenance [J].IEEE Transactions on Software Engineering,1980,SE-6(6):545-552.
[8] Pan W F,Li B,Ma Y T,et al.Multi-granularity evolution analysis of software using complex network theory [J].Journal of Systems Science and Complexity,2011,24(6):1068-1082.
[9] 李兵,王浩,李增扬,等.基于复杂网络的软件复杂性度量研究 [J].电子学报,2006,34(12A):2371-2375. Li B,Wang H,Li Z Y,et al.Software complexity metrics based on complex networks [J].Acta Electronica Sinica,2006,34(12A):2371-2375.(in Chinese)
[10] Potanin A,Noble J,Frean M,et al.Scale-free geometry in object-oriented programs [J].Communications of the ACM,2005,48(5):99-103.
[11] Concas G,Marchesi M,Pinna S,et al.Power-laws in a large object-oriented software system [J].IEEE Transactions on Software Engineering,2007,33(10):687-708.
[12] 李辉,赵海,徐久强,等.基于k-核的大规模软件宏观拓扑结构层次性研究 [J].电子学报,2010,38(11):2635-2643. Li H,Zhao H,Xu J Q,et al.Research on hierarchy of large-scale software macro-topology based on k-core [J].Acta Electronica Sinica,2010,38(11):2635-2643.(in Chinese)
[13] Albert R,Jeong H,Barabási A L.Error and attack tolerance in complex networks .Nature,2000,406(6794):378-382.
[14] Zhou Y M,Leung H.Empirical analysis of object-oriented design metrics for predicting high and low severity faults [J].IEEE Transactions on Software Engineering,2006,32(10):771-789.
[15] Xml-security .http://santuario.apache.org/,2012-8-20.
[16] JTopas .http://sourceforge.net/projects/jtopas/,2012-8-20.
[17] 屈波,聂长海,徐宝文.回归测试中测试用例优先级技术研究综述 [J].计算机科学与探索,2009,3(3):225-233. Qu B,Nie C H,Xu B W.Survey of test case prioritization for regression testing .Journal of Frontier of Computer Science and Technology,2009,3(3):225-233.(in Chinese)
[18] Elbaum S,Malishevsky A G,Rothermel G.Incorporating varying test costs and fault severities into test case prioritization .Proceedings of the International Conference on Software Engineering .Washington D C,USA:IEEE,2001.329-338.
[19] Jones J,Harrold M J.Test-suite reduction and prioritization for modified condition/decision coverage .Proceedings of the International Conference on Software Maintenance .New York,USA:ACM,2001.92-101.
[20] Jeffrey D,Gupta N.Test case prioritization using relevant slices .Proceedings of the International Computer Software Applications Conference .Washington D C,USA:IEEE,2006.411-420.
[21] Zhang X F,Nie C H,Xu B W,et al.Test case prioritization based on varying testing requirement priorities and test case costs .Proceedings of the International Conference on Quality Software .Washington D C,USA:IEEE,2007.15-24.
[22] Do H,Rothermel G,Kinneer A.Empirical studies of test case prioritization in a JUnit testing environment .Proceedings of the 15th International Symposium on Software Reliability Engineering .Washington D C,USA:IEEE,2004.113-124.
[23] Do H,Rothermel G,Kinneer A.Prioritizing JUnit test cases:An empirical assessment and cost-benefits analysis [J].Empirical Software Engineering:An International Journal,2006,11(1):33-70.
[24] Srikanth H.Requirements-based test case prioritization .Proceedings of Student Research Forum in 12th ACM SIGSOFT International Symposium on the Foundations of Software Engineering .New York,USA:ACM,2004.
[25] Tonella P,Avesani P,Susi A.Using the case-based ranking methodology for test case prioritization .Proceedings of the International Conference on Software Maintenance .New York,USA:ACM,2006.123-133.
[26] Korel B,Koutsogiannakis G,Tahat L.Model-based test prioritization heuristic methods and their evaluation .Proceedings of the 3rd International Workshop on Advances in Model-based Testing .New York,USA:ACM,2007.34-43.
[27] Walcott K R,Soffa M L,Kapfhammer G M,et al.Time-aware test suit prioritization .Proceedings of the International Symposium on Software Testing and Analysis .New York,USA:ACM,2006.1-11.
[28] Zhang L,Hou S,Guo C,et al.Time-aware test-case prioritization using integer linear programming .Proceedings of the International Symposium on Software Testing and Analysis .New York,USA:ACM,2009.213-224.
[29] Zhang L,Zhou J,Hao D,et al.Prioritizing JUnit test cases in absence of coverage information .Proceedings of the 25th International Conference on Software Maintenance .New York,USA:ACM,2009.19-28.
[30] Jiang B,Zhang Z Y,Chan W K,et al.How well does test case prioritization integrate with statistical fault localization? [J].Information and Software Technology,2012,54(7):739-758.
[31] Subramanyan R,Krishnan M S.Empirical analysis of CK metrics for object-oriented design complexity:Implication for software defects [J].IEEE Transactions on Software Engineering,2003,29(10):297-310.
[32] Basili V R,Briand L C,melo W L.Validation of object-oriented design metrics as quality indicators [J].IEEE Transactions on Software Engineering,1996,22(10):751-761.
[33] Emam K EI,Benlarbi S,Goel N.The confounding effect of class size on the validity of object-oriented metrics [J].IEEE Transactions on Software Engineering,2001,27(6):630-650.
[34] Gyimóthy T,Ferenc R,Siket I.Empirical validation of object-oriented metrics on open source software for fault prediction [J].IEEE Transactions on Software Engineering,2003,31(10):897-910.
[35] Liu J,Lü J H,He K Q,et al.Characterizing the structural quality of general complex software networks [J].International Journal of Bifurcation and Chaos,2008,18(2):605-613.
[36] Pan W F,Li B,Ma Y T,et al.Measuring structural quality of object-oriented softwares via bug propagation analysis on weighted software networks [J].Journal of Computer Science and Technology,2010,25(6):1202-1213.
[37] JUnit .http://junit.sourceforge.net/,2012-8-21.
[38] djUnit .http://works.dgic.co.jp/djunit/,2012-8-21.
[39] McClave J T,Benson P G,Sincich T.Statistics for Business and Economics (Tenth Edition) [M].New Jersey:Prentice Hall,2008.
[40] Chidamber S R,Kemerer C F.A metrics suit for object-oriented design [J].IEEE Transactions on Software Engineering,1994,20(6):476-493.
[41] Xu J,Ho D,Capretz L F.An empirical validation object-oriented design metrics for fault prediction [J].Journal of Computer Science,2008,4(7):571-577.
[42] 张莉,钱冠群,李琳.基于变更传播仿真的软件稳定性分析 [J].计算机学报,2010,33(3):440-451. Zhang L,Qian G Q,Li L.Software stability analysis based on change impact simulation [J].Chinese Journal of Computers,2010,33(3):440-451.(in Chinese)
[43] Noh J D,Rieger H.Random walks on complex network [J].Physical Review Letters,92(11):118701.
[44] Liu X F,Tse C K,Small M.Composing music with complex network .Proceedings of the 1st International Conference on Complex Sciences:Theory and Applications .Aarhus,Denmark:Springer,2009.2196-2205.
[45] Do H,Elbaum S,Rothermel G.Supporting controlled experimentation with testing techniques:An Infrastructure and its potential impact [J].Empirical Software Engineering:An International Journal,2005,10(4):405-435.
[46] Chillarege R,Bhandari I S,Chaar J K,et al.Orthogonal defect classification:A concept for in-process measurements [J].IEEE Transactions on Software Engineering,1992,18(11):943-956.
[47] Ma Y T,He K Q,Li B,et al.A hybrid set of complexity metrics for large-scale object-oriented software systems [J].Journal of Computer Science and Technology,2010,25(6):1184-1201.