Tesla Solves for General Obstacle Detection by Use of Occupancy Networks

Photo: Ashok Elluswamy/Twitter

Tesla found a solution to general obstacle detection and used it to enable sophisticated collision avoidance using Occupancy Networks. The Autopilot team shared their progress at the CVPR Conference.

The Tesla Autopilot team attended the IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR) in June and now shared what they talked about with the general public, in a YouTube video. One of the most interesting topics was “Occupancy Networks,” which is a solution to the problems of general obstacle detection and its use to provide sophisticated collision avoidance.

Presented some of the recent work from the Tesla Autopilot team at CVPR this year, especially about "Occupancy Networks" - our approach to solve general obstacle detection and using it to enable sophisticated collision avoidance. Full talk here: https://t.co/wpGXlNHaWl (1/12)

— Ashok Elluswamy (@aelluswamy) August 21, 2022

Over the past decade, there has been a revolution in computer vision. Many vision tasks such as object detection, semantic segmentation, optical flow estimation, and more can now be solved with unprecedented accuracy using deep neural networks. Many of these issues are represented in the realm of 2D imaging, however the physical world we live in is not 2D but 3D, so thinking in 3D is critical to making intelligent systems interact with their 3D environment.

The 3D world consists of length, width, and height, but Tesla takes into account one more dimension—time. “You're thinking about the world in three dimensions and the fourth dimension being time,” said CEO Elon Musk during the Q2 2020 Earnings Call. The company worked in this direction, developing equipment and methods for machine learning. Now, during the CVPR Conference, the Tesla Autopilot team has spoken more about it.

Director of Autopilot Software at Tesla, Ashok Elluswamy, in a Twitter thread, shared a short summary of Tesla's presentation. He said that typical approaches such as image-space segmentation of free space or pixel-wise depth have many issues. To address these issues, the company is working on Occupancy Networks, which “predict volumetric occupancy of all the things around the car. i.e. every voxel or continuous point in 3D space has a probability of being occupied and also its future motion,” explained Elluswamy. He said that occupancy networks have several good features:

• Predict geometric occupancy
• Use multi-camera & video context
• Predict dynamic occupancy flow
• Persistent through occlusions
• Resolution where it matters
• Efficient memory and compute
• Runs in ~10 ms

All of this allows the car to avoid any unidentified objects in its surroundings. It also allows the car to “peep out” around corners. By creating a picture of the surrounding world and predicting the movement of certain objects, the car is able to make unprotected turns, such as a human pilot can do.

Elluswamy said that the occupancy representation of these networks allows for differentiable rendering of images, based on the Neural Radiance Fields (NeRF) work. However, unlike typical NeRFs, which are generated per scene, these occupancy networks generalize across scenes. These predictions are already being used to avoid a lot of collisions. Elluswamy wrote that Autopilot prevents about 40 accidents a day when drivers mistakenly press the accelerator pedal instead of the brake pedal.

Elluswamy said that many of these improvements will be available to testers in the FSD Beta 10.69 release, which began rolling out to a limited number of drivers on Saturday. In the coming weeks, these improvements will become available to all Tesla owners in the US and Canada who are approved for FSD Beta.

Although Tesla cannot yet avoid 100% of collisions, as there are still a number of difficult tasks to be solved, this technological progress is very interesting for the company's Autopilot team. Ultimately, they strive to build a car that literally never crashes.

© 2022, Eva Fox | Tesmanian. All rights reserved.

_____________________________

We appreciate your readership! Please share your thoughts in the comment section below. 

Article edited by @SmokeyShorts; follow him on Twitter

Follow @EvaFoxU