- Title
- The Limitations of Current Similarity-Based Objective Metrics in the Context of Human-Agent Interaction Applications
- Creator
- Deffrennes, Armand; Vincent, Lucile; Pivette, Marie; El Haddad, Kevin; Bailey, Jacqueline Deanna; Perusquia-Hernandez, Monica; Alarcão, Soraia M.; Dutoit, Thierry
- Relation
- ICMI '23 Companion: Companion Publication of the 25th International Conference on Multimodal Interaction. Proceedings of The 25th International Conference on Multimodal Interaction (ICMI 2023) (Paris, France 09-13 October, 2023) p. 81-85
- Publisher Link
- http://dx.doi.org/10.1145/3610661.3617155
- Publisher
- Association for Computing Machinery
- Resource Type
- conference paper
- Date
- 2023
- Description
- There are two main ways of evaluating a model generating an interactive virtual agent's expressions. The first is through subjective perception tests, and the second is through objective metrics, which usually compare the model's generated expressions to a test set of expressions considered the ground truth. In this work, we argue that using such objective metrics comparing generated expressions, to expressions contained in a test set limits the accuracy of the evaluation by failing to consider expressions that are different from the test set, but are still valid and well-perceived by users. We support this argument through experiments showing that different expression sequences are well perceived as listening responses to the same speaker's utterance.
- Subject
- evaluation metrics; human-agent interactions; nonverbal expressions; avatar; virtual agent; embodied conversational agent
- Identifier
- http://hdl.handle.net/1959.13/1496807
- Identifier
- uon:54238
- Identifier
- ISBN:9798400703218
- Language
- eng
- Reviewed
- Hits: 1546
- Visitors: 1507
- Downloads: 0
| Thumbnail | File | Description | Size | Format |
|---|