Who could not have noticed the DeepSeek AI model bursting onto the scene recently? So significant were the purported cost savings of this model, (10x) compared to ChatGPT, the emergence of this competitor model wiped half a trillion dollars (yes trillion with a “t”) off the value of nVidia in a day. nVidia produces approximately 90% of all the GPUs (Graphics Processing Unit), the fundamental compute unit of AI. DeepSeek appears to require significantly less of these chips or at least much less technically advanced chips.
To train/make an AI model like DeepSeek or ChatGPT requires supercomputer (aka High Performance Computing (“HPC”)) levels of performance. This is where the claimed savings have been made. The DeepSeek model only required 10% of the high performance computing power that ChatGPT requires.
So, what was different about the creation of the DeepSeek model? As it currently appears, nothing startling and given the code has been released and is open source everyone now knows and has access to it.
The “secret sauce” appears to be:
- Optimising: the communication between the GPUs, both within and between servers, has been written with very low level code to maximise the utilisation of the graphics processing units.
- Routing: Information is routed selectively through the model with only the most relevant pieces of the model being used.
- Precision: Less bits (precision) are used to build the model.
This might all seem obvious when it’s laid out like this, but we need to remember how young this technology is. While AI has had many guises in the last seven decades, ChatGPT is barely two years old. ChatGPT has fundamentally shaped the way we perceive AI today. The emergence of Large Language Models (“LLM”) emerged from work done by seven people at Google in 2017, with a paper entitled "All you need is Attention". ChatGPT stood on those shoulders in the same way DeepSeek itself stands on ChatGPT's.
What does this mean for the demand for GPUs moving forward? The smart money says not much. Jevons Paradox effect will explain. Think about your favourite guilty pleasure: Tim Tams, Porsches - you choose. If they suddenly become ten times cheaper, do you continue to buy as many as you used to at a tenth the price? Of course you don't, in theory you buy more. And in the case of AI, you would do ten times as much as you used to. This leads us to Artificial General Intelligence (AGI), a goal everyone has been saying is twenty years away. Sam Altman, the founder of OpenAI, the organisation that give us ChatGPT, says AGI is low single digit years away and he is probably right.
This forecast demand has huge implications for the Data Centres that house this GPU compute. A single current generation GPU draws as much power as a bar radiator, of the order of 1kW. A current rack of GPU compute easily draws more than 100kW. But that is more than quadruple what we thought of as a ‘power dense’ rack only a couple of years ago. From an infrastructure perspective there is no AI without HPC. How much worse is this power dense rack problem going to get? For the sake of discussion let’s assume somewhere between two and four times in the next five years.
So, what does this mean for the AI Data Centre?
We used to think a Data Centre of 100MegaWatt was big. People now talk of GigaWatt Data Centres. Think about it as a power density of 200kW per rack, so five racks are 1MW. I was recently at the Pacific Telecommunications Council Meeting in Hawaii where I met a Data Centre operator who was talking of delivering 300kW per square meter.
They are going to require a significant amount of electricity, which is going to produce a lot of heat which will require specialised cooling, which will in turn will require even more electricity.
So how do we deal with all the excess heat generated? Forget air cooling, we have already moved on from the last generation of GPUs where that was possible. Going forward it will be one of two technologies; ‘Direct Liquid Cooling’, which is cold plates attached to the chips with flowing water or ‘Immersion Cooling’, where the entire server is bathed in something similar to baby oil. It's the only feasible way to evacuate the excess heat. The Supercomputer crowd having been doing this for decades.
Does everyone need to build their own Foundational AI Model? Up until now that didn't seem particularly feasible, not when it requires tens of thousands of GPUs. This is in the realm of the hyperscalers, or the ‘Magnificent Seven’ as the stock market likes to refer to them.
But what if training a Foundational Model only required a few thousand GPUs? This would be a game changer. You might contemplate that investment proposal if it gave you an ability to cure cancer or discover a life changing drug or be ten times more productive than your competitor. This is Jevons Paradox at work.
AI Data Centres will be a continually challenging paradigm given the ever-increasing demand for electrical power, figuring out how it is best distributed and solving and how to ‘plumb’ the chips and evacuate the excess heat.
Which leaves me thinking that nVidia will be just fine and that their competitors such as AMD will do well; a rising tide floats all boats. Perhaps the bigger question is where the cost-effective electrical power is coming from…!
About the Author:
With over 35 years of experience in the IT industry, Chris Clarkson is a technologist turned deal maker, passionate about delivering innovative solutions and creating value for customers and stakeholders. He has successfully managed hardware, software, and professional services business units, driving growth through strategic planning, sales and marketing execution, and high-level negotiations. His expertise lies in developing and implementing business plans that align technology with business objectives, ensuring sustainable success. With extensive knowledge of market dynamics and emerging trends, he focuses on building strong partnerships and leading teams to achieve transformative results.