Effectiveness Evaluation of Mutation Operators for Distributed Memory Parallel Programs

TIAN Tian; WANG Miao-miao; LI Cheng-long; GONG Dun-wei

doi:10.12263/DZXB.20240741

您当前的位置：

首页 >

文章列表页 >

Effectiveness Evaluation of Mutation Operators for Distributed Memory Parallel Programs

PAPERS | 更新时间：2025-12-08

- Effectiveness Evaluation of Mutation Operators for Distributed Memory Parallel Programs
- ACTA ELECTRONICA SINICA Vol. 53, Issue 3, Pages: 864-877(2025)
- 作者机构：
  
  1.山东建筑大学计算机科学与技术学院，山东济南 250101
  2.青岛科技大学自动化与电子工程学院，山东青岛 266061
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China(62102235);Natural Science Foundation of Shandong Province(ZR2020MF084)
- DOI：10.12263/DZXB.20240741
  CLC： TP311;
- Received：13 August 2024，
  
  Revised：2024-12-27，
  
  Published：25 March 2025
- 稿件说明：
移动端阅览
田甜, 王苗苗, 李成龙, 等. 分布存储并行程序变异算子有效性评估[J]. 电子学报, 2025, 53(03): 864-877.

TIAN Tian, WANG Miao-miao, LI Cheng-long, et al. Effectiveness Evaluation of Mutation Operators for Distributed Memory Parallel Programs[J]. Acta Electronica Sinica, 2025, 53(03): 864-877.
田甜, 王苗苗, 李成龙, 等. 分布存储并行程序变异算子有效性评估[J]. 电子学报, 2025, 53(03): 864-877. DOI：10.12263/DZXB.20240741

TIAN Tian, WANG Miao-miao, LI Cheng-long, et al. Effectiveness Evaluation of Mutation Operators for Distributed Memory Parallel Programs[J]. Acta Electronica Sinica, 2025, 53(03): 864-877. DOI：10.12263/DZXB.20240741

摘要

变异测试通过注入变异算子生成变异体模拟软件中的潜在缺陷，是提高软件质量的关键技术.大量变异体及其执行成本制约着变异测试的研究进展和在工业领域的应用.选择有效的变异算子是减少变异体数量的主要途径.针对分布存储并行程序，本文提出变异算子有效性评价准则.从顽固变异体、崩溃变异体和等价变异体3个方面对变异体进行分类；基于不同变异体对测试数据质量的影响，定义变异算子有效性评价准则，分析不同变异算子的有效性.实验结果表明，使用本文提出的评价准则能够选择合理的变异算子，基于这些变异算子能够生成更多的有效变异体和尽可能少的无效变异体，在保持变异测试有效性的前提下，平均减少了22.61%的变异体，提高了变异测试效率.

Abstract

Mutation testing is a pivotal technology for enhancing software quality by injecting mutation operators to generate mutant programs that mimic potential defects in software. The substantial number of mutants and the associated execution costs

however

limit the advancement and industrial application of mutation testing. The selection of effective mutation operators is a primary strategy for reducing the volume of mutants. Addressing distributed memory parallel programs

this paper introduces an evaluation criterion for the effectiveness of mutation operators. Mutants are categorized into three types: stubborn mutants

crash mutants

and equivalent mutants. Drawing on the influence of various mutants on the quality of test data

we establish a criterion for evaluating the effectiveness of mutation operators and analyze the performance of different mutation operators. Experimental results demonstrate that the proposed evaluation criterion enables the selection of suitable mutation operators

leading to the generation of a higher number of valid mutants and a minimized number of invalid mutants. Consequently

while preserving the effectiveness of mutation testing

the average number of mutants is decreased by 22.61%

thereby enhancing the efficiency of mutation testing.

关键词

Keywords

references

GU H F , ZHANG J N , CHEN M S , et al . Specification-driven conformance checking for virtual/silicon devices using mutation testing [J ] . IEEE Transactions on Computers , 2021 , 70 ( 3 ): 400 - 413 .

SOUZA S R S , BRITO M A S , SILVA R A , et al . Research in concurrent software testing: A systematic review [C ] // Proceedings of the Workshop on Parallel and Distributed Systems: Testing, Analysis, and Debugging . New York : ACM , 2011 : 1 - 5 .

JIN K X , LANO K . Design and classification of mutation operators for OCL specification [C ] // Proceedings of the 25th International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings . New York : ACM , 2022 : 852 - 861 .

BASILE D , TER BEEK M H , LAZREG S , et al . Static detection of equivalent mutants in real-time model-based mutation testing [J ] . Empirical Software Engineering , 2022 , 27 ( 7 ): 160 .

AHMED Z , SCHWASS E , HERBOLD S , et al . A new perspective on the competent programmer hypothesis through the reproduction of real faults with repeated mutations [J ] . Software Testing, Verification and Reliability , 2024 , 34 ( 3 ): e1874 .

HUMBATOVA N , JAHANGIROVA G , TONELLA P . DeepCrime: From real faults to mutation testing tool for deep learning [C ] // 2023 IEEE/ACM 45th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) . Piscataway : IEEE , 2023 : 68 - 72 .

田甜 , 巩敦卫 . 并发程序变异测试研究综述 [J ] . 电子学报 , 2020 , 48 ( 11 ): 2267 - 2277 .

TIAN T , GONG D W . Survey on mutation testing of concurrent programs [J ] . Acta Electronica Sinica , 2020 , 48 ( 11 ): 2267 - 2277 . (in Chinese)

KAUFMAN S J , FEATHERMAN R , ALVIN J , et al . Prioritizing mutants to guide mutation testing [C ] // 2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE) . Piscataway : IEEE , 2022 : 1743 - 1754 .

GLIGORIC M , ZHANG L M , PEREIRA C , et al . Selective mutation testing for concurrent code [C ] // Proceedings of the 2013 International Symposium on Software Testing and Analysis . New York : ACM , 2013 : 224 - 234 .

DU H , PALEPU V K , JONES J A . Ripples of a mutation: An empirical study of propagation effects in mutation testing [C ] // Proceedings of the IEEE/ACM 46th International Conference on Software Engineering . New York : ACM , 2024 : 1 - 13 .

OFFUTT A J , LEE A , ROTHERMEL G , et al . An experimental determination of sufficient mutant operators [J ] . ACM Transactions on Software Engineering and Methodology , 1996 , 5 ( 2 ): 99 - 118 .

DU H , PALEPU V K , JONES J A , et al . To kill a mutant: An empirical study of mutation testing kills [C ] // Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis . New York : ACM , 2023 : 715 - 726 .

LI J X , ZHANG Y M , LU S , et al . Performance bug analysis and detection for distributed storage and computing systems [J ] . ACM Transactions on Storage , 2023 , 19 ( 3 ): 1 - 33 .

CHEKAM T T , PAPADAKIS M , CORDY M , et al . Killing stubborn mutants with symbolic execution [J ] . ACM Transactions on Software Engineering and Methodology , 2021 , 30 ( 2 ): 1 - 23 .

GHOSH S . Towards measurement of testability of concurrent object-oriented programs using fault insertion: A preliminary investigation [C ] // Proceedings of Second IEEE International Workshop on Source Code Analysis and Manipulation . Piscataway : IEEE , 2002 : 17 - 25 .

DELAMARO M , PEZZÈ M , VINCENZI A M R . Mutant operators for testing concurrent Java programs [C ] // Anais do XV Simpósio Brasileiro de Engenharia de Software (SBES 2001) . Raleigh : Sociedade Brasileira de Computação , 2001 : 272 - 285 .

BRADBURY J S , CORDY J R , DINGEL J . Mutation operators for concurrent Java (J2SE 5.0 )[C ] // Second Workshop on Mutation Analysis (Mutation 2006 - ISSRE Workshops 2006) . Piscataway : IEEE , 2006 : 11 .

WU X X , ZHENG W , SHI Z , et al . Concurrency bug-oriented mutation operators design for Java [C ] // 2018 IEEE International Conference on Progress in Informatics and Computing (PIC) . Piscataway : IEEE , 2018 : 364 - 369 .

JAGANNATH V , GLIGORIC M , LAUTERBURG S , et al . Mutation operators for actor systems [C ] // 2010 Third International Conference on Software Testing, Verification, and Validation Workshops . Piscataway : IEEE , 2010 : 157 - 162 .

MORADI MOGHADAM M , BAGHERZADEH M , KHATCHADOURIAN R , et al . Akka: Mutation testing for actor concurrency in akka using real-world bugs [C ] // Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering . New York : ACM , 2023 : 262 - 274 .

CAÑIZARES P C , NÚÑEZ A , DE LARA J . OUTRIDER: Optimizing the mUtation Testing pRocess in Distributed EnviRonments [J ] . Procedia Computer Science , 2017 , 108 : 505 - 514 .

ESTERO-BOTARO A , PALOMO-LOZANO F , MEDINA-BULO I . Mutation operators for WS-BPEL 2.0 [C ] // Proceedings of the 21th International Conference on Software & Systems Engineering and Their Applications . Paris : Genie Lo giciel , 2008 : 1 - 7 .

SNIR M , OTTO S , STEVEN H , et al . MPI-the Complete Reference: The MPI Core [M ] . Cambridge : MIT Press , 1998 .

SILVA R A , DO ROCIO SENGER DE SOUZA S , DE SOUZA P S L . Mutation operators for concurrent programs in MPI [C ] // 2012 13th Latin American Test Workshop (LATW) . Piscataway : IEEE , 2012 : 1 - 6 .

KURTZ B , AMMANN P , OFFUTT J , et al . Analyzing the validity of selective mutation with dominator mutants [C ] // Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering . New York : ACM , 2016 : 571 - 582 .

MATHUR A P . Performance, effectiveness, and reliability issues in software testing [C ] // Proceedings The Fifteenth Annual International Computer Software & Applications Conference . Piscataway : IEEE , 2002 : 604 - 605 .

OFFUTT A J , ROTHERMEL G , ZAPF C . An experimental evaluation of selective mutation [C ] // Proceedings of 1993 15th International Conference on Software Engineering . Piscataway : IEEE , 1993 : 100 - 107 .

KAZEROUNI A M , DAVIS J C , BASAK A , et al . Fast and accurate incremental feedback for students’ software tests using selective mutation analysis [J ] . Journal of Systems and Software , 2021 , 175 : 1 - 18 .

ZHANG S Y , WANG X Y , FENG L C , et al . Mutation operator reduction for deep learning system [C ] // 2021 7th International Symposium on System and Software Reliability (ISSSR) . Piscataway : IEEE , 2021 : 32 - 37 .

KIM Y , HONG S . Learning-based mutant reduction using fine-grained mutation operators [C ] // 2022 IEEE Conference on Software Testing, Verification and Validation (ICST) . Piscataway : IEEE , 2022 : 464 .

宋利 , 刘靖 . 基于SOM神经网络的二阶变异体约简方法 [J ] . 软件学报 , 2019 , 30 ( 5 ): 1464 - 1480 .

SONG L , LIU J . Second-order mutant reduction based on SOM neural network [J ] . Journal of Software , 2019 , 30 ( 5 ): 1464 - 1480 . (in Chinese)

王子健 . 基于多目标差分进化算法的二阶变异体约简方法研究 [D ] . 呼和浩特 : 内蒙古大学 , 2022 .

WANG Z J . Research on Second-Order Variation Reduction Method Based on Multi-Objective Differential Evolution Algorithm [D ] . Hohhot : Inner Mongolia University , 2022 . (in Chinese)

FAN L X , LI Z , LIU H Y , et al . SGS: Mutant reduction for higher-order mutation-based fault localization [C ] // 2023 IEEE 47th Annual Computers, Software, and Applications Conference (COMPSAC) . Piscataway : IEEE , 2023 : 870 - 875 .

OJDANIC M , SOREMEKUN E , DEGIOVANNI R , et al . Mutation testing in evolving systems: Studying the relevance of mutants to code evolution [J ] . ACM Transactions on Software Engineering and Methodology , 2023 , 32 ( 1 ): 1 - 39 .

WANG X Y , ZHANG S Y , LIU F X , et al . MQP: Mutants quality prediction for cost-effective mutation testing [C ] // 2021 IEEE 21st International Conference on Software Quality, Reliability and Security Companion (QRS-C) . Piscataway : IEEE , 2021 : 45 - 50 .

SUN C G , FU A , GUO X L , et al . ReMuSSE: A redundant mutant identification technique based on selective symbolic execution [J ] . IEEE Transactions on Reliability , 2022 , 71 ( 1 ): 415 - 428 .

GARG A , OJDANIC M , DEGIOVANNI R , et al . Cerebro: Static subsuming mutant selection [J ] . IEEE Transactions on Software Engineering , 2023 , 49 ( 1 ): 24 - 43 .

TIAN Z , CHEN J J , ZHU Q H , et al . Learning to construct better mutation faults [C ] // Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering . New York : ACM , 2022 : 1 - 13 .

BOSCH J , CARLOS Á , JIMÉNEZ-GONZÁLEZ D , et al . Asynchronous runtime with distributed manager for task-based programming models [J ] . Parallel Computing , 2020 , 97 : 102664 .

FU X J , CHEN Z B , ZHANG Y F , et al . MPISE: Symbolic execution of MPI programs [C ] // 2015 IEEE 16th International Symposium on High Assurance Systems Engineering . Piscataway : IEEE , 2015 : 181 - 188 .

CHEN Z B , YU H B , FU X J , et al . MPI-SV: A symbolic verifier for MPI programs [C ] // 2020 IEEE/ACM 42nd International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) . Piscataway : IEEE , 2020 : 93 - 96 .

SIEGEL S F , ZIRKEL T K . TASS: The toolkit for accurate scientific software [J ] . Mathematics in Computer Science , 2011 , 5 ( 4 ): 395 - 426 .

SIEGEL S F , ZIRKEL T K . FEVS: A functional equivalence verification suite for high-performance scientific computing [J ] . Mathematics in Computer Science , 2011 , 5 ( 4 ): 427 - 435 .

Headquarters ASC . CORAL benchmarks [EB/OL ] . ( 2017-12-01 )[ 2024-8-13 ] . https://asc.llnl.gov/coral-2-benchmarks https://asc.llnl.gov/coral-2-benchmarks .

CHEN G , HONG A , SHAN J . The Parallel Algorithm [M ] . Beijing : Higher Educaion Press , 2004 : 353 - 355 .

DUNBAR J . NAS [CP/OL ] . ( 2023-02-24 )[ 2024-8-13 ] . https://www.nas.nasa.gov/publications/npb.html https://www.nas.nasa.gov/publications/npb.html .

DANG X Y , YAO X J , GONG D W , et al . Efficiently generating test data to kill stubborn mutants by dynamically reducing the search domain [J ] . IEEE Transactions on Reliability , 2020 , 69 ( 1 ): 334 - 348 .

KIM J , JEON J , HONG S , et al . Predictive mutation analysis via the natural language channel in source code [J ] . ACM Transactions on Software Engineering and Methodology , 2022 , 31 ( 4 ): 1 - 27 .

田甜 , 邵阳阳 , 王苗苗 , 等 . 基于程序依赖关系的变异体生成策略 [J ] . 计算机应用 , 2024 , 44 ( 9 ): 2863 - 2870 .

TIAN T , SHAO Y Y , WANG M M , et al . Mutant generation strategy based on program dependencies [J ] . Journal of Computer Applications , 2024 , 44 ( 9 ): 2863 - 2870 . (in Chinese)

LALLCHANDANI J T , MALL R . Computation of dynamic slices for object-oriented concurrent programs [C ] // 12th Asia-Pacific Software Engineering Conference (APSEC’05) . Piscataway : IEEE , 2005 : 8 - 15 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

A Method for Finding Redundant Mutants in Mutation Testing

Selecting Objects to be Mutated Based on Statement Importance

Mutation Testing Based on Comparison of Paths

Survey on Mutation Testing of Concurrent Programs

Related Author

MENG Fan-rong

GONG Yun-zhan

QIAN Gen-nan

WANG Ya-wen

TIAN Tian

QIN Bei

GONG Dun-wei

YAO Xiang-juan

Related Institution

Changchun Automobile Industry Institute Changchun Jilin China

Guangxi Colleges and Universities Key Laboratory of Cloud Computing and Complex Systems Guilin University of Electronic Technology Guilin Guangxi China

Guangxi Cooperative Innovation Center of Cloud Computing and Big Data Guilin University of Electronic Technology Guilin Guangxi China

State Key Laboratory of Networking and Switching Technology Beijing University of Posts and Telecommunications Beijing China

Changchun Automobile Industry Institute

⁰