Tesla has acquired the machine learning startup DeepScale, CNBC, Techcrunch, and other news outlets have reported. The company’s CEO, Forrest Iandola, announced Monday that he had joined Tesla’s Autopilot team.
Iandola explained his company’s mission to Ars during a phone call, just after the company raised $15 million from venture capitalists in April 2018.
DeepScale was building image recognition software based on convolutional neural networks.
A key step for any self-driving software system is perception: identifying cars, pedestrians, bicycles, and other objects around the car. Identifying objects accurately is crucial because it allows software to make informed predictions about where they might move in the future. Most companies working on the problem use a technique called “convolutional neural networks” (CNNs) to tackle this problem. You can check out our deep dive on CNNs for full details on how they work.
DeepScale focuses on improving the speed and efficiency of convolutional neural networks, drawing on Iandola’s past work as a computer science graduate student. The company’s techniques will be particularly helpful to Tesla. Tesla is relying heavily on machine learning techniques to achieve full self-driving capabilities without the lidar sensors or high-definition maps being used by most of Tesla’s competitors.
Making neural networks a lot smaller
A famous 2012 paper known as AlexNet, after lead author Alex Krizhevsky, first showed the power of neural networks for image recognition. The AlexNet authors figured out how to harness the parallel computing power of GPU cards to train much larger convolutional neural networks than had been used in the past. That allowed them to deliver far better performance on a standard image recognition task than any previous algorithm.
One notable downside to the AlexNet algorithm, however, was that it was huge, with 60 million trainable parameters. Before he founded DeepScale, Iandola was a PhD candidate at the University of California, Berkeley, where he developed techniques for reducing the size of neural networks like AlexNet.
Using a variety of optimizations, Iandola and his co-authors demonstrated that they could achieve AlexNet-like performance while reducing the number of parameters by a factor of 50. That reduced the physical size of a trained AlexNet network from 240MB to less than 5MB. Using additional compression techniques developed by other researchers, including switching from 32-bit to 8-bit parameters, they were able to reduce the size of their model by another factor of 10—producing convolutional neural networks with AlexNet-like performance that were less than half a megabyte.
In his 2018 interview with Ars, Iandola argued that this kind of optimization is significant for companies trying to bring image recognition technology to market. Companies like Tesla regularly push out new versions of their neural networks to customer vehicles, which often have limited bandwidth. It’s much easier to push out half a megabyte of data than 240 megabytes.
Smaller models will become particularly significant as companies start to build custom silicon for machine learning applications. Iandola pointed out this advantage in a 2016 paper: “When deploying CNNs on Application-Specific Integrate Circuits (ASICs), a sufficiently small model can be stored directly on-chip, and smaller models may enable ASICs to fit on a smaller die.” This has obvious cost advantages, and it can also improve performance, since the chip doesn’t need to bring model parameters in from external memory.
DeepScale sought to commercialize Iandola’s research
As Iandola was wrapping up his research at Berkeley around 2015, he was looking for a way to commercialize the technology. He quickly realized that the self-driving car boom was an opportunity to apply his research to a practical problem.
“The research was particularly focused on making some of the most efficient neural networks—energy efficiency, running really fast,” Iandola told Ars in 2018. “The autonomous driving market was just taking off, and we found a nice opportunity there.”
“What our solutions do is to identify things on the road,” Iandola said. “We can tell you what kind of objects we’re seeing and how far away they are. In object recognition, we’ve had an order of magnitude improvement in error rate.”
Iandola noted that market leader Waymo had impressive technology, but “there’s a lot of custom hardware in there that is expensive.” DeepScale’s role, he said, “is less about having the first capability than getting things down to a cost point and reliability point where it could be mass-produced.”
“We don’t build any hardware,” he added. Instead, the company uses “commodity processors and sensors.” He added: “Our superpower is cutting the compute by a factor of 100.”
DeepScale seems like a good fit for Tesla
Three years ago, Elon Musk promised that customers would be able to achieve full self-driving with the hardware the company was shipping at the time. Earlier this year, Tesla tacitly admitted that wasn’t true with the rollout of a new custom chip for machine learning applications.
Still, Tesla is under a lot of pressure to achieve excellent machine learning performance with a limited computational budget. This is a particularly hard problem because Tesla is trying to do it without lidar sensors or high-definition maps—two resources most other self-driving car companies consider critical to getting the technology to work in a timely fashion.
On the flip-side, Tesla’s large fleet gives the company a vast amount of data to use for training neural networks. With hundreds of thousands of vehicles on the roads and the ability to query the fleet for “interesting” events, Tesla’s engineers can draw on billions of miles of real-world data to help train the neural networks that power Autopilot.
Tesla also has a constant need to replenish its Autopilot talent pool because the company has faced a steadyexodus of top talent over the last three years. For more than three years, Elon Musk has claimed that full autonomy is less than two years away. In 2015, he declared that full autonomy was “a much easier problem than people think it is.”
That attitude created friction with engineers on the Autopilot team who saw Musk’s aggressive timetables as unrealistic. Tesla Autopilot chief Sterling Anderson quit in late 2016, shortly after Musk promised that the new hardware would be capable of full autonomy. Two more Autopilot bosses have departed the company since then, along with many engineers.