Tesla’s head of AI, Andrej Karpathy unveiled Tesla’s AI Computer, Dojo during a conference on computer vision and pattern recognition. As per the specifications provided by Tesla, Dojo Computer will be the world’s 5th most powerful computer.
Elon Musk tweeted recently, about holding a Tesla AI day. In a month or so, the event will be held, where the recruiting process is going to happen. As the current AI team has been progressing on AI Dojo, more are to join the team. Once the Dojo is ready, the supercomputer will possibly be designed to be made available to other companies (not confirmed, if it will be available only for Tesla vehicles)
Looking at holding Tesla AI Day in about a month or so. Will go over progress with Tesla AI software & hardware, both training & inference. Purpose is recruiting.
— Elon Musk (@elonmusk) June 21, 2021
Back in 2020, Elon Musk hinted on Twitter that the Dojo Computer would be a “beast”. It was expected that it will be in a race with Intel and AMD, which came out to be true. As mentioned by Karpathy in the conference, the computer has 720 nodes of 8x A100 80GB.
Our main engineering locations are Palo Alto, Austin & Seattle, but working remotely is acceptable for exceptional candidates
— Elon Musk (@elonmusk) August 15, 2020
Know more about Dojo computer
“Dojo” means training or school for Japanese martial arts. The computer was named so because it is basically a training school for computers. The details revealed by Karpathy can be summed up as,
- 1.8 EFLOPS (720 nodes * 312 TFLOPS-FP16-A100 * 8 gpu/nodes)
- 720 nodes of 8x A100 80GB. (5760 GPUs total)
- 640 Tbps of total switching capacity
- 10 PB of “hot tier” NVME storage @ 1.6 TBps
Furthermore, the computer is to train its upcoming self-driving AI and neural nets which power Tesla’s autopilot. The recent Tesla autopilot FSD Version 9 update includes pure vision. Which uses regular cameras to detect objects surrounding the vehicle. Meant to reduce costs, such change would need much advanced AI technology.
Additionally, the supercomputer can have one quintillion (1018) floating-point operations per second. Tesla has been working on these complex neural networks for years aside from vehicle developments. And they have hinted that the computer would be ready by year-end.
Further details on the Tesla Dojo computer were not revealed. As commented by Karpathy,
“We have a neural net architecture network and we have a data set, a 1.5 petabytes data set that requires a huge amount of computing. So I wanted to give a plug to this insane supercomputer that we are building and using now. For us, computer vision is the bread and butter of what we do and what enables Autopilot. And for that to work really well, we need to master the data from the fleet, and train massive neural nets and experiment a lot. So we invested a lot into the compute. In this case, we have a cluster that we built with 720 nodes of 8x A100 of the 80GB version. So this is a massive supercomputer. I actually think that in terms of flops, it’s roughly the number 5 supercomputer in the world.”
Once development is finished, it is known that the Tesla Dojo computer will be made available to other companies too.
Neural networks
In simple words, programmers tried to imitate what goes on inside the human brain. The brain has cells connected by neurons, where signals are sent in electrical or chemical ways. However, the brain tissues are known as biological neural nets.
The developers of artificial networks started “artificial neural networks” and continued to work on them despite multiple failures. The neural networks are not like regular programming with an “IF-THEN-ELSE” structure. Artificial neural networks are trained with experience, not programmed. Choosing right or wrong would be analyzed from the previous thousands of data available.
Tesla’s AI computer, Dojo will be used in the FSD (Tesla’s self-driving technology). With the supercomputer, the driver’s bad behavior can be corrected. Like if there is a yellow light and the driver hasn’t slowed down. Then the system immediately alerts the driver to slow down.
