Using model-based testing to reduce test automation technical debt: An industrial experience report

Abstract

Technical debt is the metaphor used to describe the effect of incomplete or immature software artifacts that bring short-term benefits to projects, but may have to be paid later with interest. Software testing cost is proven to be high due to the time (and resource)-consuming activities involved. Test automation is a strategy that can potentially reduce this cost and provide savings to the software development process. The lack or poor implementation of a test automation approach derives in test automation debt. The goal of this paper is to report our experience using a model-based testing (MBT) approach on two industrial legacy applications and assess its impact on test automation debt reduction. We selected two legacy systems exhibiting high test automation debt, then used a MBT tool to model the systems and automatically generate test cases. We finally assessed the impact of this approach on the test automation technical debt by analyzing the code coverage attained by the tests and by surveying development team perceptions. Our results show that test automation debt was reduced by adding a suite of auto- mated tests and reaching more than 75% of code coverage. Moreover, the development team agrees in that MBT could help reduce other types of technical debt present in legacy systems, such as documentation debt and design debt. Although our results are promising, more studies are needed to validate our findings.