Quality Assurance for AI-based Systems: Overview and Challanges

Authors: Michael Felderer, Rudolf Ramler

The number and importance of AI-based systems in all domains is growing. With the pervasive use and the dependence on AI-based systems, the quality of these systems becomes essential for their practical usage. However, quality assurance for AI-based systems is an emerging area that has not been well explored and requires collaboration between the SE and AI research communities. This paper discusses terminology and challenges on quality assurance for AI-based systems to set a baseline for that purpose. Therefore, we define basic concepts and characterize AIbased systems along the three dimensions of artefact type, process, and quality characteristics. Furthermore, we elaborate on the key challenges of (1) understandability and interpretability of AI models, (2) lack of specifications and defined requirements, (3) need for validation data and test input generation, (4) defining expected outcomes as test oracles, (5) accuracy and correctness measures, (6) non-functional properties of AI-based systems, (7) self-adaptive and self-learning characteristics, and (8) dynamic and frequently changing environments.

Vorgetragen von: Michael Felderer, Rudolf Ramler
Unternehmen: Universität Innsbruck

Vortragssprache: Englisch
Level: Fortgeschrittene

Partner der Konferenz 2020

ASQF e.V ATB - Austrian Testing Board Blekinge Institute of Technology CON.ECT Eventmanagement dpunkt.verlag GmbH Fortiss GmbH Heise Medien GmbH & Co. KG IREB GmbH iSQI GmbH IT Verlag für Informationstechnik GmbH IT-Visions.de Österreichische Computer Gesellschaft (OCG) Software Quality Lab GmbH TU Wien, Inst f. Information Systems Eng., CDL-SQ Verband Österreichischer Software Industrie (VÖSI)