Self-Driving AI - Introduction

As a side project, I've been exploring the use of reinforcement learning in order to train a car to drive itself. This AI is specifically built for driving control, and assumes that the physical world has already been converted into state data by a vision neural network.

Through reinforcement learning, AI has been able to achieve superhuman levels of performance in many media. AlphaZero is now the greatest chess player in the world, and OpenAI Five has successfully defeated one of the best Dota 2 teams in the world in a Best of 3 match-up.

The challenge of applying reinforcement learning to the real world is that of simulation. While environments such as chess and video games can easily be duplicated thousands of times in order to vastly increase the AI's training speed, certain aspects of the real world are too varied to be able to completely reproduce them.

However, Tesla has been working on a solution, an extremely detailed, extremely flexible simulation of the real-world, which they use to train their self-driving AI on both normal examples, and edge cases which one cannot readily produce in the real world, such as a car falling out of the sky. This turns the 'weakness' of simulations into a strength, allowing near-infinite training data to be produced for the AI to learn from.

Taking this approach, I have opted to build increasingly complex simulations of the real world, and train the self-driving AI purely based on these simulations. This project intends to be a proof of concept that, with a simulation engine which accurately reflects the physics of the real world, one can produce a self-driving AI which could then simply be uploaded to a physical car with a compatible vision NN, and be used to control the car's driving, without needing any real-world data at all.

The benefits of this are vast:

• No massively expensive labelling costs

• No lengthy turnaround times for labelling

• With the right procedural generation of edge cases and new environments, truly infinite training data

Most of all, if one is to apply the same conclusion which has been readily observed in Chess, and complex RTS games like Dota 2: a car trained via reinforcement learning will be the greatest driver in the world.