Autonomous Overtaking in Gran Turismo Sport Using Curriculum Reinforcement Learning (ICRA 2021)

Professional race car drivers can execute extreme overtaking maneuvers. However, conventional systems for autonomous overtaking rely on either simplified assumptions about the vehicle dynamics or solving expensive trajectory optimization problems online. When the vehicle is approaching its physical limits, existing model-based controllers struggled to handle highly nonlinear dynamics and cannot leverage the large volume of data generated by simulation or real-world driving. To circumvent these limitations, this work proposes a new learning-based method to tackle the autonomous overtaking problem. We evaluate our approach using Gran Turismo Sport---a world-leading car racing simulator known for its detailed dynamic modeling of various cars and tracks. By leveraging curriculum learning, our approach leads to faster convergence as well as increased performance compared to vanilla reinforcement learning. As a result, the trained controller outperforms the built-in model-based game AI and achieves comparable overtaking performance with an experienced human driver. Reference: Y. Song*, H. Lin*, E. Kaufmann, P. Duerr, D. Scaramuzza Autonomous Overtaking in GTS Using Curriculum Reinforcement Learning International Conference on Robotics and Automation (ICRA), 2021. PDF: More about our research on Deep Learning: Affiliations: Y. Song, E. Kaufmann and D. Scaramuzza are with the Robotics and Perception Group, Dep. of Informatics, University of Zurich, and Dep. of Neuroinformatics, University of Zurich and ETH Zurich, Switzerland H. Lin and P. Dürr are with Sony, Switzerland. Voiceover: Pete Edmunds | British Voiceover