Comparison of end-to-end testing tools for microservices: A case study

Abstract

Microservices has emerged as a architectural style that pro- vides several benefits but also poses some challenges. One such challenge is testability, since an application may have hundreds or thousands of services operating together, and each of them needs to be tested as they evolve. To overcome this challenge, test automation is key, and together with it, the use of effective and efficient testing tools. Hence, we aim to contribute to this area by evaluating two tools that support end- to-end (E2E) testing of microservices. E2E tests allow to verify if the system works well as a whole (particularly relevant for systems made up of microservices). In this work, we first surveyed E2E testing tools reported in academic literature and by industry practitioners. Then, we applied the IEEE 14102-2010 standard to evaluate those tools. The two top-rated tools, Jaeger and Zipkin, were selected for further evaluation of their effectiveness and efficiency. Results from our case study reveal that Jaeger is more efficient and effective than Zipkinin terms of execution and failure detection times, as well as information provided to detect faults, severity and coverage.