Using Inverse Reinforcement Learning to Understand What is Being Learned in Motor Learning

Motor learning is an important part of our lives – when we learn a new skill, during childhood as we learn to control our bodies, and in rehabilitation when we re-learn how to perform day-to-day tasks. It is assumed that we perform these tasks in an optimal way, but it is not obvious what it is that we are optimizing.

In this research project, we tackle this problem by using computational techniques, specifically inverse reinforcement learning and inverse optimization, to try to understand what people are learning.

We explore this problem at two levels - using inverse reinforcement learning paradigms to determine what reward produces human-like movements in a simple point-to-point reaching task, and using inverse optimization, where we study a complex task (playing the piano) and try to understand what experts are optimizing in terms of selecting posture.

These findings can benefit our understanding of how and why we perform particular types of movements, and be used as a tool to accelerate motor learning.

Summary of the inverse optimization procedure in piano playing comparing experts and novices (a) Visualization of the arms and hands from a motion capture system (b) index finger joint angles when pressing a piano key (c) Slopes of the regression lines (loudness in Sone as a function of position).

Positive values correspond to performing a crescendo as required. The expert group performed the task with a higher slope - i.e., took better advantage of the dynamic range. (d) The least comfortable postures selected by the experts were more comfortable than the novices (p=0.05), suggesting that comfort is one factor optimized by experts.