Model-Based Transfer Learning for Contextual Reinforcement Learning
AI Papers Podcast Daily · AIPPD
Audio is streamed directly from the publisher (media.rss.com) as published in their RSS feed. Play Podcasts does not host this file. Rights-holders can request removal through the copyright & takedown page.
Show Notes
This research introduces Model-Based Transfer Learning (MBTL), a novel framework for improving the efficiency and robustness of deep reinforcement learning (RL) in contextual Markov Decision Processes (CMDPs). MBTL strategically selects training tasks to maximize generalization performance across a range of tasks by modeling both the performance set point using Gaussian processes and the generalization gap as a function of contextual similarity. The method uses Bayesian optimization to guide task selection, achieving theoretically sublinear regret and experimentally demonstrating up to a 50x improvement in sample efficiency compared to traditional training methods. The effectiveness of MBTL is validated across various continuous control and urban traffic benchmarks. Further analysis shows the method's insensitivity to the underlying RL algorithm and hyperparameters.