NVIDIA’s next big move: Data centers

The data center is now the unit of computing.

The data center is now the unit of computing: Jensen Huang

In a recent conversation, NVIDIA CEO Jensen Huang discussed the revolutionary changes occurring in data centers due to the rapid advancement of AI technology. Huang provides insights into how AI is reshaping data center architecture, infrastructure, and operations, positioning NVIDIA at the forefront of this transformation.

"We're trying to build a brand new [data center] every year and every single year we deliver two or three times more performance."

The Data Center Revolution

Data centers are undergoing a massive transformation to support AI workloads. Huang identifies two major trends:

"We have a trillion dollars worth of data centers that we have to modernize and we have at least a trillion of new AI workloads coming on."

This dual approach involves upgrading existing infrastructure and building entirely new AI-centric data centers.

The scaling law used to be about pre-training. Now we've gone to multimodality, synthetic data generation, post-training has scaled up incredibly, and inference scaling has gone through the roof.

AI Factories: The New Data Center Model

Huang introduces the concept of "AI factories" - next-generation data centers designed specifically for AI workloads:

"There's going to be a new infrastructure, this new infrastructure are going to be AI factories that operate these digital humans."

These AI factories represent a fundamental shift in how we think about and design data centers.

Unprecedented Scale and Speed

The scale and speed of AI data center deployments are breaking records:

Huang cites X's (formerly Twitter) recent deployment as an example of the incredible pace of AI infrastructure buildout.

"100,000 GPUs, that's easily the fastest supercomputer on the planet as one cluster... We're talking about 19 days."

Energy Efficiency and Cooling Innovations

With the increased compute density of AI workloads, data centers are adopting advanced cooling technologies:

"Liquid cooled, energized, permitted, in the short time that was done... it's unbelievable."

These innovations are crucial for managing the power and cooling demands of AI-focused data centers.

Distributed Computing and Data Centers

As AI models and datasets grow, Huang foresees a shift towards more distributed data center architectures:

"Distributed training will have to work... and some form of federated learning and distributed asynchronous distributed computing is going to be discovered."

This suggests future AI data centers may be more interconnected and geographically dispersed.

Watch the full video