Research Area:  Machine Learning

Abstract:

In real-world applications, inferring the intentions of expert agents (e.g., human operators) can be fundamental to understand how possibly conflicting objectives are managed, helping to interpret the demonstrated behavior. In this paper, we discuss how inverse reinforcement learning (IRL) can be employed to retrieve the reward function implicitly optimized by expert agents acting in real applications. Scaling IRL to real-world cases has proved challenging as typically only a fixed dataset of demonstrations is available and further interactions with the environment are not allowed. For this reason, we resort to a class of truly batch model-free IRL algorithms and we present three application scenarios: (1) the high-level decision-making problem in the highway driving scenario, and (2) inferring the user preferences in a social network (Twitter), and (3) the management of the water release in the Como Lake. For each of these scenarios, we provide formalization, experiments and a discussion to interpret the obtained results.

Keywords:  
Inverse reinforcement learning
Model-free IRL
Truly batch IRL
IRL for real life
Multiple experts IRL
Non-stationary IRL

Author(s) Name:   Amarildo Likmeta, Alberto Maria Metelli, Giorgia Ramponi, Andrea Tirinzoni, Matteo Giuliani & Marcello Restelli

Publisher name:  Springer

DOI:  10.1007/s10994-020-05939-8

Volume Information:  volume 110, pages 2541–2576 (2021)

Paper Link:   https://link.springer.com/article/10.1007/s10994-020-05939-8