Research Breakthrough Possible @S-Logix pro@slogix.in

Office Address

Social List

Meta-reinforcement learning in nonstationary and nonparametric environments - 2023

meta-reinforcement-learning-in-nonstationary-and-nonparametric-environments.jpg

Research Topics On Meta-reinforcement learning in nonstationary and nonparametric environments

Research Area:  Machine Learning

Abstract:

Recent state-of-the-art artificial agents lack the ability to adapt rapidly to new tasks, as they are trained exclusively for specific objectives and require massive amounts of interaction to learn new skills. Meta-reinforcement learning (meta-RL) addresses this challenge by leveraging knowledge learned from training tasks to perform well in previously unseen tasks. However, current meta-RL approaches limit themselves to narrow parametric and stationary task distributions, ignoring qualitative differences and nonstationary changes between tasks that occur in the real world. In this article, we introduce a T ask- I nference-based meta-RL algorithm using explicitly parameterized G aussian variational autoencoders (VAEs) and gated R ecurrent units (TIGR), designed for nonparametric and nonstationary environments. We employ a generative model involving a VAE to capture the multimodality of the tasks. We decouple the policy training from the task-inference learning and efficiently train the inference mechanism on the basis of an unsupervised reconstruction objective. We establish a zero-shot adaptation procedure to enable the agent to adapt to nonstationary task changes. We provide a benchmark with qualitatively distinct tasks based on the half-cheetah environment and demonstrate the superior performance of TIGR compared with state-of-the-art meta-RL approaches in terms of sample efficiency (three to ten times faster), asymptotic performance, and applicability in nonparametric and nonstationary environments with zero-shot adaptation. Videos can be viewed at https://videoviewsite.wixsite.com/tigr.

Keywords:  
nonparametric environments
nonstationary
Meta-reinforcement learning

Author(s) Name:  Zhenshan Bing,Alois Knoll,Kai Huang,Long Cheng,Xiaojie Su

Journal name:  IEEE Transactions on Neural Networks and Learning Systems

Conferrence name:  

Publisher name:  IEEE

DOI:  10.1109/TNNLS.2023.3270298

Volume Information:  Volume 12,Pages 1-15,(2023)