We have our fourth seminar for the TC seminar series on Friday (March 24th 2023) at 13:00 EDT (GMT -4:00). Please find the flyer with all the details at this link.
Speaker: Nathan Ratliff
Title: On the Geometric Foundations of Continuous Control
Abstract: In 2022, Bertsekas made a deep observation in Lessons from AlphaZero. He showed that finite-horizon optimization over an actor-critic network is a powerful Newton step on the Bellman equation. In continuous control, this observation is salient because it provides a unifying perspective on optimal control and reinforcement learning, two workhorses which have separately been extremely successful in the field. In this talk, we argue that these observations will shape decision systems into the future, and rather than optimizing over standard physical dynamics, optimal control methods will move toward optimizing more generally over increasingly sophisticated models of behavioral dynamics that capture not only physics but also the statistics (a memory) of past optimal control solutions. We present a class of behavioral dynamics functions called geometric fabrics which build off the geometric properties of classical mechanics to create expressive path priors for statistically efficient behavior synthesis. We show both how to manually design these fabrics to capture relevant engineered behaviors such as obstacle avoidance, end-effector attraction, posturing, etc., and how to learn the fabrics from demonstration, achieving striking sample efficiencies from their geometric inductive bias. Bertsekas’s results are observations on the structure of abstract dynamic programming and apply equally well to discrete systems like AlphaZero and continuous systems like those of robotics. But to leverage them effectively in robotics, we need to better understand how to efficiently encode information into the continuous time differential equations that govern robotic behavior. Geometric fabrics are a step in that direction.
Date: Friday, March 24th, 2023
Time: 13:00 - 14:00 EDT (GMT -4:00)
More details on upcoming seminars (and video links for past ones) can be found here.