ecms_neu_mini.png

Digital Library

of the European Council for Modelling and Simulation

 

Title:

How to run a world record? A Reinforcement Learning approach

Authors:

Sajad Shahsavari, Eero Immonen, Masoomeh Karami, Hashem Haghbayan, Juha Plosila

Published in:

 

 

(2022). ECMS 2022, 36th Proceedings
Edited by: Ibrahim A. Hameed, Agus Hasan, Saleh Abdel-Afou Alaliyat, European Council for Modelling and Simulation.

 

DOI: http://doi.org/10.7148/2022

ISSN: 2522-2422 (ONLINE)

ISSN: 2522-2414 (PRINT)

ISSN: 2522-2430 (CD-ROM)

 

ISBN: 978-3-937436-77-7
ISBN: 978-3-937436-76-0(CD)

 

Communications of the ECMS , Volume 36, Issue 1, June 2022,

Ålesund, Norway May 30th - June 3rd, 2022

 

Citation format:

Sajad Shahsavari, Eero Immonen, Masoomeh Karami, Hashem Haghbayan, Juha Plosila (2022). How to run a world record? A Reinforcement Learning approach, ECMS 2022 Proceedings Edited By: Ibrahim A. Hameed, Agus Hasan, Saleh Abdel-Afou Alaliyat, European Council for Modeling and Simulation.

doi:10.7148/2022-0159

DOI:

https://doi.org/10.7148/2022-0159

Abstract:

Finding the optimal distribution of exerted effort by an athlete in competitive sports has been widely investigated in the fields of sport science, applied mathematics and optimal control. In this article, we propose a reinforcement learning-based solution to the optimal control problem in the running race application. Well-known mathematical model of Keller is used for numerically simulating the dynamics in runner's energy storage and motion. A feed-forward neural network is employed as the probabilistic controller model in continuous action space which transforms the current state (position, velocity and available energy) of the runner to the predicted optimal propulsive force that the runner should apply in the next time step. A logarithmic barrier reward function is designed to evaluate performance of simulated races as a continuous smooth function of runner's position and time. The neural network parameters, then, are identified by maximizing the expected reward using on-policy actor-critic policy-gradient RL algorithm. We trained the controller model for three race lengths: 400, 1500 and 10000 meters and found the force and velocity profiles that produce a near-optimal solution for the runner's problem. Results conform with Keller's theoretical findings with relative percent error of 0.59% and are comparable to real world records with relative percent error of 2.38%, while the same error for Keller's findings is 2.82%.

Full text: