Towards a Driving Exam for ADAS software

We present practical results of testing the OpenPilot ADAS software (the core of the Comma3 [https://comma.ai/]). OpenPilot consists of C/C++ and Python code to handle most of the logic involved in driving a car. Also, Machine Learning models are employed for various detection tasks, such as road line detection.

To test the functional behavior (“Is OpenPilot able to handle certain traffic situations?”), we are implementing a framework that generates test scenarios from traffic laws. For the non-functional aspect, we have looked at the robustness of the implemented ML models, as part of analysis for Trustworthy AI (“Can OpenPilot be fooled by modifying the input images?”).

We will present the outcomes, discuss the results, and give pointers for further research on ADAS testing.

Nutzen für den Teilnehmer:
The audience will obtain understanding of how validation processes for complex software (possible containing AI/ML) can be automated. We will perform this based on ADAS modules used in self-driving cars (in particular OpenPilot). However, the methodology and framework will be rich enough to be relevant for other domains as well. Also, the testing of different aspects for Trustworthy AI will be explained.

Behandelte Problemstellungen:
How to test ADAS software from a functional perspective?

How to test ADAS software from a non-functional perspective, in particular Trustworthy AI?

How to automate this ADAS type-approval process in a highly automated fashion?

Vortragssprache: Englisch
Level: Fortgeschrittene
Zielgruppe: SW quality engineers, ML specialists, ADAS software developers

Unternehmen:
Hanzehogeschool Groningen

Dr Rix Groenboom