Context. Tests are becoming more and more central to the development process, so that their comprehensibility is of paramount importance. In particular, assertions, which express the test expected results, must be immediately understandable. Thus, recently several libraries emerged for making assertions "fluent", i.e., more comprehensible and easy to develop. However, there is no empirical evidence for the claimed advantages and benefits, that could convince SQA Managers and Testers for their adoption.Objective. The aim of this work is gauging one of the claimed benefits of fluent assertions, namely improvement in comprehensibility, with respect to basic assertions.Method. We conducted a controlled experiment involving 51 Bachelor students. AssertJ - a library supporting fluent assertions - is compared with JUnit Basic assertions, in a test comprehension scenario. We analysed the level of comprehension of the assertions, the time required to answer questions, and the overall efficiency in completing the assignments.Results. The results show that adopting AssertJ has no significant effect on the level of comprehension of the assertions, though it significantly reduces the time required to understand assertions, so that it significantly improves the overall efficiency during comprehension of assertions.Conclusions. From our study it emerges that fluent assertions are a better choice though they do not provide the expected improvements on understandability. Indeed they could be significantly improved by choosing methods names that better capture the assertion intended meaning.

Fluent vs Basic Assertions in Java: An Empirical Study

Leotta, Maurizio;Cerioli, Maura;Ricca, Filippo
2018

Abstract

Context. Tests are becoming more and more central to the development process, so that their comprehensibility is of paramount importance. In particular, assertions, which express the test expected results, must be immediately understandable. Thus, recently several libraries emerged for making assertions "fluent", i.e., more comprehensible and easy to develop. However, there is no empirical evidence for the claimed advantages and benefits, that could convince SQA Managers and Testers for their adoption.Objective. The aim of this work is gauging one of the claimed benefits of fluent assertions, namely improvement in comprehensibility, with respect to basic assertions.Method. We conducted a controlled experiment involving 51 Bachelor students. AssertJ - a library supporting fluent assertions - is compared with JUnit Basic assertions, in a test comprehension scenario. We analysed the level of comprehension of the assertions, the time required to answer questions, and the overall efficiency in completing the assignments.Results. The results show that adopting AssertJ has no significant effect on the level of comprehension of the assertions, though it significantly reduces the time required to understand assertions, so that it significantly improves the overall efficiency during comprehension of assertions.Conclusions. From our study it emerges that fluent assertions are a better choice though they do not provide the expected improvements on understandability. Indeed they could be significantly improved by choosing methods names that better capture the assertion intended meaning.
File in questo prodotto:
File Dimensione Formato  
08590189.pdf

accesso chiuso

Tipologia: Documento in versione editoriale
Dimensione 909.81 kB
Formato Adobe PDF
909.81 kB Adobe PDF   Visualizza/Apri   Richiedi una copia

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: http://hdl.handle.net/11567/946378
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 2
  • ???jsp.display-item.citation.isi??? 1
social impact