Research Area:  Machine Learning

Abstract:

Conversational recommender systems aim to interactively support online users in their information search and decision-making processes in an intuitive way. With the latest advances in voice-controlled devices, natural language processing, and AI in general, such systems received increased attention in recent years. Technically, conversational recommenders are usually complex multi-component applications and often consist of multiple machine learning models and a natural language user interface. Evaluating such a complex system in a holistic way can therefore be challenging, as it requires (i) the assessment of the quality of the different learning components, and (ii) the quality perception of the system as a whole by users. Thus, a mixed methods approach is often required, which may combine objective (computational) and subjective (perception-oriented) evaluation techniques. In this paper, we review common evaluation approaches for conversational recommender systems, identify possible limitations, and outline future directions towards more holistic evaluation practices.

Keywords:  
Conversational recommender systems
Dialogue systems
Interactive systems
Natural language processing
Machine Learning
Deep Learning

Author(s) Name:  Dietmar Jannach

Publisher name:  Springer

DOI:  10.1007/s10462-022-10229-x

Volume Information:  

Paper Link:   https://link.springer.com/article/10.1007/s10462-022-10229-x