AMD Helios Packs 72 GPUs and 31TB HBM 4 Memory Into One AI Rack

AI infrastructure is moving from separate processors to full systems with GPUs, CPUs, memory, networking, and software. AMD's Helios platform is built on this idea. It has 72 GPUs in a single rack and focuses on memory, networking, software support, and accelerator connectivity. This shift shows how current AI workloads are spreading across more and more processors. 

AMD has revealed something different from a normal graphics card launch. It is called Helios, a complete AI rack containing 72 GPUs, AMD CPUs, large amounts of memory, high-speed networking, and software designed to make everything work like one giant computing system.

AMD Helios Brings 72 GPUs Into One Rack

Microsoft and Oracle are already betting on it, while OpenAI is buying AMD hardware and developing its own custom AI chip at the same time. Companies are building entire AI systems instead of simply making faster chips because one GPU is no longer enough. Modern AI models can contain hundreds of billions of parameters.

Training them and serving them to millions of users requires large amounts of memory and computing power. As a result, workloads get divided across dozens, hundreds, or even thousands of GPUs. Once workloads are spread across many GPUs, another problem emerges: those GPUs need to communicate constantly.

If moving data between GPUs is too slow, expensive processors spend time waiting instead of calculating. This is one reason Nvidia became dominant in AI infrastructure. Nvidia did not just build powerful GPUs. It built NVLink, a high-speed connection that allows GPUs to exchange data much faster than they could through traditional data center networking.

Combined with NVSwitch, Nvidia can connect 72 GPUs into one tightly integrated system. Nvidia then surrounds those GPUs with its networking, switches, and, most importantly, CUDA. Companies buying Nvidia are therefore not really buying a chip. They are buying an ecosystem. Helios is AMD’s attempt to challenge that ecosystem.

Helios Uses 72 Instinct MI455X GPUs

A Helios rack contains 72 Instinct MI455X GPUs. Each uses HBM4 high-bandwidth memory, giving the complete rack roughly 31TB of memory. That matters because keeping more of an AI model inside one rack means less information has to travel between different racks. In AI infrastructure, moving data can matter as much as processing it.

AMD is also taking a different approach from Nvidia. Instead of controlling every layer with proprietary technology, Instinct is built heavily around open standards. Inside the rack, AMD uses UALink to connect accelerators. Between racks, AMD uses its Pen Standard networking technology alongside Ethernet and Ultra Ethernet.

AMD is essentially betting that companies will want AI infrastructure without being locked into one company’s entire technology stack. Hardware is only half the battle. Nvidia has CUDA, and developers have spent years building AI software around it. AMD has ROCm, its open-source alternative, and this may be AMD’s biggest challenge.

Nvidia’s next major competitor might not be a single chip.

Even capable hardware becomes less attractive if companies need months of engineering work just to move their AI workloads onto it. Software compatibility and the ability to deploy existing workloads therefore remain important parts of the competition between the two platforms.

That is why Microsoft’s decision matters. According to the original report, Microsoft plans to use Helios for production-scale AI workloads on Azure, while Oracle has also committed to the platform. For AMD, that gives Helios something beyond a benchmark: real-world validation. 

OpenAI is buying accelerators from companies such as AMD and Nvidia while also developing custom silicon designed around its own AI workloads. Helios does not mean AMD has defeated Nvidia, and custom AI chips do not mean Nvidia is disappearing. What is changing is the number of serious alternatives.