Fluent vs Basic Assertions in Java: An Empirical Study

Context. Tests are becoming more and more central to the development process, so that their comprehensibility is of paramount importance. In particular, assertions, which express the test expected results, must be immediately understandable. Thus, recently several libraries emerged for making assertions "fluent", i.e., more comprehensible and easy to develop. However, there is no empirical evidence for the claimed advantages and benefits, that could convince SQA Managers and Testers for their adoption.Objective. The aim of this work is gauging one of the claimed benefits of fluent assertions, namely improvement in comprehensibility, with respect to basic assertions.Method. We conducted a controlled experiment involving 51 Bachelor students. AssertJ - a library supporting fluent assertions - is compared with JUnit Basic assertions, in a test comprehension scenario. We analysed the level of comprehension of the assertions, the time required to answer questions, and the overall efficiency in completing the assignments.Results. The results show that adopting AssertJ has no significant effect on the level of comprehension of the assertions, though it significantly reduces the time required to understand assertions, so that it significantly improves the overall efficiency during comprehension of assertions.Conclusions. From our study it emerges that fluent assertions are a better choice though they do not provide the expected improvements on understandability. Indeed they could be significantly improved by choosing methods names that better capture the assertion intended meaning.