Learning Transferable Policies for Autonomous Planetary Landing via Deep Reinforcement Learning

Giulia Ciabatti*

Shreyansh Daftry*

Roberto Capobianco

* External authors

Ascend 2021 by AIAA

2021

Abstract

In this work, we develop an application for autonomous landing, exploiting the properties of Deep Reinforcement Learning and Transfer Learning in order to tackle the problem of planetary landing on unknown or barely-known extra-terrestrial environments by learning good-performing policies, which are transferable from the training environment to other, new environments, without losing optimality. To this end, we model a real-physics simulator, by means of the Bullet/PyBullet library, composed by a lander, defined through the standard ROS/URDF framework and realistic 3D terrain models, for which we adapt official NASA 3D meshes, reconstructed from the data retrieved during missions. Where such models are not available, we reconstruct the terrain from mission imagery - generally SAR imagery. In this setup, we train a Deep Reinforcement Learning model - using DDPG and SAC, then comparing the outcomes - to autonomously land on the lunar environment. Moreover, we perform transfer learning on Mars and Titan environments. Our results show that DDPG and SAC can learn good landing policies, that can be transferred to other environments. Good policies can be learned by the SAC algorithm also in the case of atmospheric disturbances - e.g. gusts.