End-to-end (E2E) testing, a technique employed to assure quality of web applications, is cost-effective only if the test suite is not flaky. Flaky test scripts produce non-deterministic results that undermine testers’ trust and thus the usefulness of the entire test suite. Recently, we were involved in the refactoring of an existing automated flaky E2E test suite for a large Web application. In this paper, we report on our experience. During the refactoring, we have computed the effort made and formalized the procedure we followed in algorithmic way so that our experience can also be of help to other developers/testers. Our procedure allowed to reduce the flakiness to virtually zero w.r.t. the original flaky test suite. Moreover, as a positive side effect, the execution time of the test suite has been reduced by of the 57%.

Reducing Flakiness in End-to-End Test Suites: An Experience Report

Olianas D.;Leotta M.;Ricca F.;
2021-01-01

Abstract

End-to-end (E2E) testing, a technique employed to assure quality of web applications, is cost-effective only if the test suite is not flaky. Flaky test scripts produce non-deterministic results that undermine testers’ trust and thus the usefulness of the entire test suite. Recently, we were involved in the refactoring of an existing automated flaky E2E test suite for a large Web application. In this paper, we report on our experience. During the refactoring, we have computed the effort made and formalized the procedure we followed in algorithmic way so that our experience can also be of help to other developers/testers. Our procedure allowed to reduce the flakiness to virtually zero w.r.t. the original flaky test suite. Moreover, as a positive side effect, the execution time of the test suite has been reduced by of the 57%.
2021
978-3-030-85346-4
978-3-030-85347-1
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11567/1071089
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 6
  • ???jsp.display-item.citation.isi??? ND
social impact