Journal of Information Processing Systems

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Test Set Generation for Pairwise Testing Using Genetic Algorithms

  • Sabharwal, Sangeeta (Dept. of Computer Science and IT, Netaji Subhas Institute of Technology) ;
  • Aggarwal, Manuj (Dept. of Computer Science and IT, Netaji Subhas Institute of Technology)
  • Received : 2014.05.23
  • Accepted : 2014.10.28
  • Published : 2017.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

In software systems, it has been observed that a fault is often caused by an interaction between a small number of input parameters. Even for moderately sized software systems, exhaustive testing is practically impossible to achieve. This is either due to time or cost constraints. Combinatorial (t-way) testing provides a technique to select a subset of exhaustive test cases covering all of the t-way interactions, without much of a loss to the fault detection capability. In this paper, an approach is proposed to generate 2-way (pairwise) test sets using genetic algorithms. The performance of the algorithm is improved by creating an initial solution using the overlap coefficient (a similarity matrix). Two mutation strategies have also been modified to improve their efficiency. Furthermore, the mutation operator is improved by using a combination of three mutation strategies. A comparative survey of the techniques to generate t-way test sets using genetic algorithms was also conducted. It has been shown experimentally that the proposed approach generates faster results by achieving higher percentage coverage in a fewer number of generations. Additionally, the size of the mixed covering arrays was reduced in one of the six benchmark problems examined.

Keywords

References

  1. A.P. Mathur, Foundations of Software Testing, 1st ed. Delhi, India: Pearson Education, 2008, pp. 309-315.
  2. D. M. Cohen, S. R. Dalal, M. L. Fredman, and G. C. Patton, "The AETG system: an approach to testing based on combinatorial design," IEEE Transactions on Software Engineering, vol. 23, no. 7, pp. 437-443, 1997. https://doi.org/10.1109/32.605761
  3. Combinatorial Testing, http://www.combinatorialtesting.com/.
  4. A. Hartman, "Software and hardware testing using combinatorial covering suites," in Graph Theory, Combinatorics and Algorithms. New York, NY: Springer, 2005, pp. 237-266.
  5. G. Sherwood, "On the construction of orthogonal arrays and covering arrays using permutation groups," 2004; http://testcover.com/pub/background/cover.htm.
  6. L. Gonzalez-Hernandez, N. Rangel-Valdez, and J. Torres-Jimenez, "Construction of mixed covering arrays of strengths 2 through 6 using a tabu search approach," Discrete Mathematics, Algorithms and Applications, vol. 4, no. 3, 2012.
  7. D. E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, 1st ed. Reading, MA: Addison-Wesley, 1989, pp. 1-22.
  8. K. A. Bush, "Orthogonal arrays of index unity," Annals of Mathematical Statistics, vol. 23, no. 3, pp. 426-434, 1952. https://doi.org/10.1214/aoms/1177729387
  9. D. M. Cohen, S. R. Dalal, J. Parelius, and G. C. Patton, "The combinatorial design approach to automatic test generation," IEEE Software, vol. 13, no. 5, pp. 83-88, 1996. https://doi.org/10.1109/52.536462
  10. R. C. Bryce, C.J. Colbourn, and M.B. Cohen, "A framework of greedy methods for constructing interaction test suites," in Proceedings of the 27th International Conference on Software Engineerin, New York, 2005, pp. 146-155.
  11. J. D. McCaffrey, "Generation of pairwise test sets using a genetic algorithm," in Proceedings of 33rd Annual IEEE International Computer Software and Applications Conference (COMPSAC '99), Seattle, WA, 2009, pp. 626-631.
  12. Y. Lei and K. C. Tai, "In-parameter-order: a test generation strategy for pairwise testing," in Proceedings of 3rd IEEE International High-Assurance Systems Engineering Symposium, Washington, DC, 1998, pp. 254-261.
  13. B. J. Garvin, M. B. Cohen, and M. B. Dwyer, "An improved meta-heuristic search for constrained interaction testing," in Proceedings of the 1st International Symposium on Search Based Software Engineering, Windsor, UK, 2009, pp. 13-22.
  14. X. Chen, Q. Gu, X. Zhang, and D. Chen, "Building prioritized pairwise interaction test suites with ant colony optimization," in Proceedings of IEEE 9th International Conference on Quality Software (QSIC'09), Jeju, Korea, 2009, pp. 347-352.
  15. X. Chen, Q. Gu, J. Qi, and D. Chen, "Applying particle swarm optimization to pairwise testing," in Proceedings of IEEE 34th Annual Computer Software and Applications Conference (COMPSAC), Seoul, Korea, 2010, pp. 107-116.
  16. B. S. Ahmed, K. Z. Zamli, and C. P. Lim, "Application of particle swarm optimization to uniform and variable strength covering array construction," Applied Soft Computing, vol. 12, no. 4, pp. 1330-1347, 2012. https://doi.org/10.1016/j.asoc.2011.11.029
  17. S. A. Ghazi and M. A. Ahmed, "Pair-wise test coverage using genetic algorithms," in Proceedings of the 2003 Congress on Evolutionary Computation (CEC'03), Canberra, Australia, 2003, pp. 1420-1424.
  18. J. D. McCaffrey, "An empirical study of pairwise test set generation using a genetic algorithm," in Proceedings of 7th International Conference on Information Technology: New Generations (ITNG), Las Vegas, NV, 2010, pp. 992-997.
  19. P. Flores and Y. Cheon, "PWiseGen: generating test cases for pairwise testing using genetic algorithms," in Proceedings of 2011 IEEE International Conference on Computer Science and Automation Engineering (CSAE), Shanghai, China, 2011, pp. 747-752.
  20. L. Yalan, C. Nie, J. M. Kauffman, G. M. Kapfhammer, and H. Leung, "Empirically identifying the best genetic algorithm for covering array generation," presented at 3rd International Symposium on Search Based Software Engineering, Szeged, Hungary, September 10-12, 2011.
  21. T. Shiba, T. Tsuchiya, and T. Kikuno, "Using artificial life techniques to generate test cases for combinatorial testing," in Proceedings of the 28th Annual International Computer Software and Applications Conference (COMPSAC'04), Hong Kong, 2004, pp. 72-77.
  22. P. Bansal, S. Sabharwal, S. Malik, V. Arora, and V. Kumar, "An approach to test set generation for pair-wise testing using genetic algorithms," in Proceedings 5th International Symposium on Search Based Software Engineering (SSBSE2013), St. Petersburg, Russia, 2013, pp. 294-299.
  23. J. D. McCaffrey, "Generation of pairwise test sets using a genetic algorithm," in Proceedings of 33rd Annual IEEE International Computer Software and Applications Conference (COMPSAC'99), Seattle, WA, 2009, pp. 626-631.
  24. A. R. A. Alsewari and K. Z. Zamli, "Design and implementation of a harmony-search-based variable-strength t-way testing strategy with constraints support," Journal of Information and Software Technology, vol. 54, no. 6, pp. 553-568, 2012. https://doi.org/10.1016/j.infsof.2012.01.002
  25. A. R. A. Alsewari and K. Z. Zamli, "A harmony search based pairwise sampling strategy for combinatorial testing," International Journal of the Physical Sciences, vol. 7, no. 7, pp. 1062-1072, 2012.
  26. M. B. Cohen, C. J. Colbourn, and A. C. Ling, "Constructing strength three covering arrays with augmented annealing," Discrete Mathematics, vol. 308, no. 13, pp. 2709-2722, 2008. https://doi.org/10.1016/j.disc.2006.06.036
  27. Overlap coefficient, http://en.wikipedia.org/wiki/Overlap_coefficient.

Cited by

  1. Combinatorial Test Set Prioritization Using Data Flow Techniques pp.2191-4281, 2018, https://doi.org/10.1007/s13369-017-2631-y