LLM Agents for Autonomous System Testing: A Semi-Structured Literature Review

System-level software testing is essential to ensure that complex systems function correctly when all components interact, yet it remains labor-intensive and error-prone. Recent advancements in Large Language Models (LLMs) offer the potential to automate and enhance system-level testing by generating test cases, reasoning about system behavior, and supporting adaptive exploration. This paper presents a systematic review of literature on LLM-based autonomous testing agents, focusing on their architectures, interactions with the tested systems, and testing objectives. We identify common limitations in current approaches, like hallucinations, limited contextual understanding, incomplete test oracles, and challenges in navigating complex system states. Based on these findings, we discuss future research opportunities.

Talk language: English
Level: Scientific
Target group:

Company:
Software Competence Center Hagenberg GmbH (SCCH)

Dr. Stefan Fischer