After being hyped for over two years, Tesla finally unveiled its supercomputer called “Dojo” during the company’s AI day on August 19, which showcased many of the company’s new technologies like the humanoid project.
CEO Elon Musk had a lot to live up to given his claim that Dojo would have the capacity of over an exaFLOP, or one quintillion floating-point operations per second.
Tesla already possesses a massive amount of video data from its more than 1 million vehicles, and this data is used to train its neural networks.
When the company set out to train its computer vision nets with current hardware options, it was met with disappointment and the belief that the company would be better off doing it internally.
The Dojo D1 chip uses 7-nanometer technology while delivering breakthrough bandwidth and compute performance.
The D1 chip is the second chip designed by Tesla itself, and it followed the FSD chip that is present in FSD computer hardware 3 in Tesla vehicles.
D1 Chip Specifications
The D1 chip also contains 362 TeraFLOPs of processing power, and the company claims that it has a GPU level compute and CPU level flexibility, with networking switch IO.
According to Ganesh Venkataramanan, Tesla senior director of Autopilot hardware and leader of the Dojo project, “This was entirely designed by the Tesla team internally. All the way from the architecture to the package. This chip is like a GPU-level computer with a CPU level flexibility and twice the network chip level IO bandwidth.”
The chips are able to connect to each other without any glue, which led to Tesla connecting 500,000 nodes. The interface, power, and thermal management are added, which results in a nine pFLOPs training tile with 36TB per second of bandwidth. All of this is in a less than one cubic food format.
Instead of cutting the wafer into pieces, Tesla decided to leave 25 SoCs on the wafer and use high-quality silicon. This enables the chips to communicate with each other without losing any speed, as well as maintain the motherboard’s quality.
Dojo only required 120 fully functional wafers, which is impressive given Intel needed to make over 130,000 300mm wafers back in 2014. The cost of Dojo should also be substantially lower since it uses five-by-five sectioned small wafers.
Another impressive feature of Dojo is that it doesn’t have any Ram outside of the SoC. Instead, the company opted to use a cache, which is a faster tier of random access memory.
Building a True Dojo Supercomputer
For Tesla to build the first actual Dojo supercomputer, it must form a compute cluster with the training tiles. According to the company, it can combine 2 x 3 tiles in a tray and two trays in a computer cabinet, which would result in over 100 pFLOPs per cabinet.
Because of the massive bandwidth, Tesla says it can link all of these together to create the HexaPod, which will break the barrier of the ExaFlop of computers in a 10-cabinet system. This has been one of the main hurdles in the development of powerful supercomputers.
How Dojo Will Change Technology?
Tesla’s Dojo has the potential to really impact the future technology scenario, especially given the exponentially increasing demand for speed and capacity for neural-network training. If all goes as planned, Dojo would achieve the best AI training performance and enable extremely complex neural net models while being power and cost-efficient.
With the introduction of the Dojo M1 chip, the tech world is likely headed toward an extremely competitive supercomputer market. Supercomputers hold the potential to solve many of society’s greatest problems, and they can turn us into superhumans. These types of computing chips can solve global privacy and identity issues, lead to incredible advancements in healthcare and medical research, create sustainable supply-chain logistics, and much more.
Supercomputers like Dojo can also have enormous economic consequences, with the first group or nation to achieve efficient supercomputing infrastructure taking the advantage. They will lead to new and accurate ways of predicting the future and analyzing current conditions, and computing power will start to affect every industry.
While the system is still being put together, Musk claims that it will be operational next year.
If this is the case, Dojo would become the fastest AI training computer ever developed while maintaining power efficiency and a small format compared to most other supercomputers.
It will likely be used by Tesla to train its neural networks for self-driving technology, but the company also plans to make it available for other developers.
Dojo is a major step toward achieving this world of supercomputing, and it is just one of many like it still yet to come.
While the technology will likely be used in Tesla’s own self-driving vehicles at first, nobody should be surprised if it eventually makes its way into various different sectors, both public and private.
As we continue to progress further into a data-driven world, the real-world use of such supercomputers is approaching quickly.