AI vs. AI: The Detection Game

In this paper, we present the results achieved by performing a test addressed to evaluate the capabilities of the so-called AI content Detectors or AI Detection Systems. Such systems promise to find out whether a given text was written by a human or created by an artificial intelligence (AI). The ability to identify the 'synthetic' nature of a text is paramount in many workplaces as well as in the educational field, where we focused our research. More specifically, we considered the academic environment and the frequent request by faculty to students to write term papers on a wide variety of topics. Then, it is essential for teachers to be able to evaluate content quality as well as their authenticity. Until now, anti-plagiarism tools have been widely used to verify that homework assignments were not copied, but the recent advent of generative artificial intelligence-based tools changed the game, moving the focus to automatic generation. In both cases, we are facing a violation of the educational pact by a student, versus a teacher, if not outright an offence. Conversely, an unfounded accusation of fraud by a teacher toward a student would be equally serious. Therefore, to use automatic detection tools one must have a guarantee on the reliability of their output.