Session: MSNDC-08/MR-05-01: Motion Planning, Dynamics, and Control of Robots

Paper Number: 117007

117007 - Fixed-Time Learning for Optimal Feedback Control 

Exploiting the benefits of reinforcement learning, the control systems community has invested considerable effort towards designing control mechanisms that run in real-time and adapt to changes in the environment. To ensure the effective operation of a control mechanism without human intervention, it is necessary for the decision-making mechanism to generate optimal policies in fixed time rather than in an infinite or finite time. In this paper, we introduce the problem of fixed-time optimal stabilization to construct feedback controllers that guarantee closed-loop system fixed-time stability while optimizing a given performance measure. Specifically, fixed-time stability of the closed-loop system is established via a Lyapunov function satisfying a differential inequality while simultaneously serving as a solution to the steady-state Hamilton-Jacobi-Bellman equation ensuring optimality. Given that the Hamilton-Jacobi-Bellman equation is generally difficult to solve, we develop a critic-only reinforcement learning-based algorithm for learning the solution to the steady-state Hamilton-Jacobi-Bellman equation in fixed-time. In particular, a non-Lipschitz experience replay-based learning law utilizing recorded and current data is introduced for updating the critic weights to learn the value function. The non-Lipschitz property of the dynamics gives rise to fixed-time convergence and stability, while the experience replay-based approach eliminates the need of satisfying the persistence of excitation condition as long as the recorded data is sufficiently rich.  Simulation results demonstrate the efficacy of the proposed approach.

Presenting Author: NICK-MARIOS KOKOLAKIS Georgia Institute of Technology

Presenting Author Biography: Nick-Marios T. Kokolakis obtained a Diploma (a 5-year degree, equivalent to an M.Sc.) in electrical and computer engineering with a specialization in systems and control from the University of Patras, Greece, in 2018. He joined the Georgia Institute of Technology in 2019, where he is currently pursuing a Ph.D. degree in aerospace engineering. His research interests include control theory, game theory, probabilistic machine learning, and reinforcement learning, as well as their applications to safety-critical control, motion planning, coordinated target tracking, and cyber-physical security.

Authors:

NICK-MARIOS KOKOLAKIS Georgia Institute of Technology
Kyriakos Vamvoudakis Georgia Institute of Technology
Wassim Haddad Georgia Institute of Technology