Visual global localization based on deep neural networks for self-driving cars

Abstract

In this work, we present a visual global localization system based on Deep Neural Networks (DNNs) for self-driving cars, called DeepVGL (Deep Visual Global Localization), which receives real-time images from a forward-facing camera installed on car’s roof and infers their corresponding position in global coordinates. To this end, DeepVGL is trained with pairs of coordinates and associated images belonging to datasets of autonomous vehicles built with sensor data aligned in time and space through a process of Simultaneous Localization And Mapping (SLAM). To assess the performance of DeepVGL, we carried out experiments using datasets composed of camera images collected by different self-driving cars on trips made over long time spans (over 4 years), thus including significant changes in the environment, traffic volume and weather conditions, as well as different times of the day and seasons of the year. We also compared DeepVGL with a state-of-the-art global localization system based on Weightless Neural Networks (WNN). Finally, we executed experiments using datasets composed of LIDAR range images obtained by a self-driving truck on trips made over reasonable time spans (over 3 months). The experimental results show that DeepVGL can correctly estimate the global localization of the self-driving car up to 75% of the time for an accuracy of 0.2 m and up to 96% of the time for an accuracy of 5 m. The results also show that DeepVGL outperforms WNN, which can correctly locate the self-driving car up to 76% of the time for 0.2 m accuracy, but only up to 89% of the time for 5 m accuracy. Finally, the results show that DeepVGL works better with LIDAR range images than camera images, locating the autonomous truck up to 95% of the time for 0.2 m accuracy and 98% of the time for 5 m accuracy.