Applying a Genetic Algorithm for Test Suite Reduction in Industry

Time and cost of test execution increases when regression test suites grow over time. Techniques for test suite reduction have been proposed to streamline frequent test execution in continuous integration and to optimize the set of tests without sacrificing coverage and fault detection. In this paper we report on the design of a genetic algorithm to tackle the underlying optimization problem in context of an industry project from a software company developing tools for test automation.
The prototypical implementation of the algorithm has been applied to the project’s test suite containing several hundred test cases. We achieved an optimal solution with a 28 % reduction of test cases. The evaluation of the reduced test suite using higher-level coverage and mutation analyses showed a minimal loss of coverage. The results demonstrated that the genetic algorithm can be successfully applied in industry and the achieved results are able to satisfy the requirements of the studied project. Nevertheless, major challenges have been identified by applying the approach in industry. They are related to the reliable collection of test execution data from previous test runs and dealing with test suites containing tests exhibiting unpredictable side-effects and flakiness.

Authors: Philipp Stadler, Reinhold Ploesch and Rudolf Ramler

Vortragssprache: Englisch
Level: Experte
Zielgruppe: Tester, Developer

Unternehmen:
Software Competence Center Hagenberg GmbH (SCCH)

Rudolf Ramler