Tesla’s Full Self-Driving visualization improvements were put to the test by an enthusiast in the safety of his home. Visualization improvements are among the main components that indicate whether Tesla has refined its Neural Net enough to release FSD.
Tesla Driver found a creative way to conduct a fun Neural Net test while in quarantine. The success of FSD and Tesla’s Robotaxi fleet mostly depends on the accuracy of the Neural Net. So the Tesla owner conducted a little experiment to see if his Model 3's neural network could accurately identify objects it would eventually encounter on the road.
In the first test, Tesla Driver wanted to see if his car could correctly identify five traffic cones he had placed in front of it. There were two small bright orange cones, one medium-sized cone, one large foldable cone, and finally a bright red traffic pole.
Credit: Tesla Driver
The Tesla was able to identify all the orange traffic cones, but not the red traffic pole. On the car’s center display, the traffic pole was also visualized as an orange traffic cone. Sometimes it was depicted correctly as a red traffic pole, but the image was not consistent. The Tesla would switch from a traffic pole and traffic cone on its display.
Tesla Driver’s test also revealed that the cameras had some challenges gauging the distance of the cones in relation to others. The same challenges were present when a human was present. Sometimes the car did not accurately visualize how the object or person were placed in relation to each other.
Based on the results, it appears that Tesla's Neural Net still has some difficulty with depth perception, which may also be a reason why Elon Musk says curves are so tricky. During an interview with Third Row Podcast, a former Tesla Autopilot Engineer, Eshak Mir, talked about the Neural Net’s difficulty with curves.
Mir explained that before Tesla’s Autopilot rewrite, the Neural Net only utilized the car’s rear right side repeaters and backup cameras to navigate a curb. As a result, Teslas had some difficulty judging the angle of approach needed to navigate a curb properly.
“You’re 80% seeing the actual curve, the rest you’re guessing. Because the distance [of the curb], how high the curb is and all that come into play,” said Mir.
Based on Tesla Driver’s test, the way the Neural Net visualizes its environment still needs some work, especially when it comes to seeing the world in 3D. However, there was one significant improvement proven by the enthusiast’s test.
Just a couple of months ago, in December 2019, Tesla’s Neural Net mistakenly labeled a boy in a bright orange shirt as a traffic cone. It was quite an entertaining mistake. Tesla Driver upped the ante when he asked his fiancée, Holly, to wear a bright orange traffic cone costume and stand in front of the car.
Former Tesla Autopilot Engineer Hints At How Much FSD Will Improve On Third Row Podcast https://t.co/2J6F1Zudlh pic.twitter.com/HYNsh5lAGh
— Tesmanian.com (@Tesmanian_com) April 8, 2020
The Tesla vehicle didn’t seem confused by the woman wearing a traffic cone costume. It identified her as a human with a traffic cone behind her. The visualization revealed that Neural Net knew it was “seeing” both a woman and some traffic cone in front of it.
However, it had difficulty understanding the finer details of who it was seeing. The Neural Net couldn’t separate the person from the costume. Unfortunately, the Neural Net was unable to correctly visualize Holly as a person when she placed her arms inside the costume and crouched closer to the ground like a traffic cone.
In general, Tesla Driver’s entertaining tests revealed that the company's Neural Net has indeed improved. Tesla has refined its Neural Net enough to probably release a feature-complete FSD by the end of the year. However, there’s a long way to go before Tesla can unleash its Robotaxi fleet, or pursue full autonomy for that matter.
Featured Image Credit: Tesla Driver/YouTube
