AI-Generated Test Scripts for Web E2E Testing with ChatGPT and Copilot: A Preliminary Study

Automated testing is vital for ensuring the reliability of web applications. This paper presents a preliminary study on leveraging artificial intelligence (AI) models, specifically ChatGPT and Github Copilot, to generate test scripts for web end-to-end testing. Through experimentation, we evaluated the feasibility and effectiveness of AI language models in generating test scripts based on natural language descriptions of user interactions with web applications. Our preliminary results show that AI-based generation generally provides an advantage over fully manual test scripts development. Starting from test cases clearly defined in Gherkin, a reduction in development time is always observable. In some cases, this reduction is statistically significant (e.g., Manual vs. a particular use of ChatGPT). These results are valid provided that the tester has some skills in manual test script development and is therefore able to modify the code produced by the AI-generation tools. This study contributes to the exploration of AI-driven solutions in web test scripts generation and lays the foundation for future research in this domain.