Effective Black Box Testing of Sentiment Analysis Classification Networks

Transformer-based neural networks have demonstrated remarkable performance in natural language processing tasks such as sentiment analysis. Nevertheless, the issue of ensuring the dependability of these complicated architectures through comprehensive testing is still open. This paper presents a collection of coverage criteria specifically designed to assess test suites created for transformer-based sentiment analysis networks. Our approach utilizes input space partitioning, a black-box method, by considering emotionally relevant linguistic features such as verbs, adjectives, adverbs, and nouns. To effectively produce test cases that encompass a wide range of emotional elements, we utilize the k-projection coverage metric. This metric minimizes the complexity of the problem by examining subsets of k features at the same time, hence reducing dimensionality. Large language models are employed to generate sentences that display specific combinations of emotional features. Experiments demonstrate that test generation guided by our criteria increases test coverage by 16%, while reducing accuracy by 6.5%, thus exposing vulnerabilities.

Vortragssprache: Englisch
Level: Wissenschaftlich
Zielgruppe:

Unternehmen:
fortiss research institute for software-intensive systems

Dr Fathiyeh Faghih