Reference-model based adaptive learning
A new theory of learning in which risk and surprise are central
OVERVIEW
We have been working on a new theory of learning, referred to as reference-model based learning (RMBL), where risk and surprise are central, while the usual prediction errors from TD learning play a secondary, though still crucial, role. Learning is modulated by how large the prediction errors are relative to model anticipation, i.e., by surprise. In a location prediction task where the target moved under a leptokurtic law, we found choices to be closer to RMBL predictions than to Bayesian learning. There are links with other theories, such as Active Inference, Actor-Critic Models and Reference-Model Based Adaptive Control in engineering.