Nvidia unveils NVHBM custom memory: controller moves into HBM stack, freeing 25% more compute

To understand NVHBM's value, you first need to see where today's AI accelerator bottleneck lies. Memory bandwidth is one of the most critical constraints on modern AI accelerators — which is why every new generation of AI chips ships with upgraded memory: from the H100's 80GB HBM3 and the H200's 141GB HBM3e, to the B200's 192GB HBM3e and the B300 (Blackwell Ultra)'s 288GB HBM3e, capacity and bandwidth keep climbing — but the memory controller's location never changed. In a standard HBM configuration (such as the current JEDEC HBM4e spec), the logic managing communication between the processor and its memory stacks — the memory controller — sits on the processor die (XPU). That controller consumes valuable silicon area that could otherwise hold more compute units, larger caches, or more power-efficient designs. In other words, at a time when compute density is approaching physical limits, every square millimeter of XPU area is precious; moving the memory controller out of the compute die is effectively returning scarce compute area to computation itself.

What exactly is the NVHBM approach? According to TechPowerUp and TweakTown, NVHBM integrates the memory controller directly into the base die of the 3D HBM stack, bringing the controller physically closer to the DRAM cells. The direct effects are threefold. First, space savings — since the controller no longer occupies XPU area, up to 25% more space is freed for additional compute units, meaning more matrix-multiply engines or vector processors can be packed into the same process node and package. Second, a smaller interface — the physical memory interface shrinks accordingly, with PHY and support area reduced by up to 67%, helping shorten signal paths, lower power consumption, and improve thermals. Third, higher bandwidth — compared to the current standard HBM4E, NVHBM delivers roughly 30% more bandwidth. For large-scale training and inference clusters, bandwidth is throughput, and a 30% bandwidth gain has a direct impact on trillion-parameter model training efficiency.

NVHBM's significance extends beyond Nvidia's own GPUs. According to multiple media reports, Nvidia is opening this memory architecture to third-party XPU customers through the NVLink Fusion ecosystem. NVLink Fusion is Nvidia's previously announced open interconnect program that lets third-party chipmakers design custom XPUs around Nvidia's NVLink technology and plug into Nvidia's data center ecosystem. Notably, Nvidia announced a $3.5 billion investment in MediaTek in late August, under which MediaTek will use Nvidia technology to design custom chips that plug into Nvidia data centers and gain access to the NVLink Fusion ecosystem — and NVHBM is one of the most attractive technology assets in that ecosystem. Of course, Nvidia still tightly controls core interconnect components like NVLink Switch chips, PHY layers, and communication controllers; third-party XPUs can be built around NVLink Fusion chiplets, but cannot operate independently of a fabric that includes Nvidia switches. This strategy of 'open compute, locked interconnect' echoes Nvidia's playbook with the CUDA ecosystem over the past several years.

For the AI infrastructure industry, NVHBM's emergence carries several deeper signals worth watching. First, memory architecture is becoming the new frontline of AI chip competition: as process scaling slows and compute density approaches limits, extracting performance from the memory side through architectural innovation is becoming a consensus path for leading players — and with it, the fight over who defines HBM standards is heating up across the supply chain, including Nvidia, SK Hynix, Samsung, and Micron. Second, the three-in-one ecosystem competition of compute, memory, and interconnect is now in full swing: Nvidia's NVLink Fusion program brings third-party XPUs into its ecosystem, expanding customer choice while further cementing its dominance over interconnect standards. Third, for cloud providers and AI companies, wider bandwidth and higher compute density mean unit compute costs may keep falling — a long-term tailwind for cost-sensitive large-model startups. What's worth watching next: when NVHBM enters mass production, which Nvidia GPU generation adopts it first, and how Samsung, SK Hynix, and Micron will respond to this wave of customized HBM.

Placed in a larger picture, NVHBM is part of Nvidia's combination punch of hardware architecture innovation plus ecosystem openness. Over the past few years, the AI chip race has centered on process and packaging: smaller nodes, larger HBM capacity, higher interconnect bandwidth. As these traditional levers approach physical limits, structural innovations like moving the memory controller from the compute die into the HBM stack become the new breakthrough — no added process cost, yet 25% more compute area and 30% more bandwidth. Meanwhile, by opening the technology to third-party chipmakers like MediaTek through NVLink Fusion, Nvidia expands its ecosystem influence while keeping the memory architecture standard firmly in its own hands. For cloud providers, this means more diverse choices for AI accelerators in future data centers; for developers, cheaper and faster inference compute opens up more room for application innovation. Whether NVHBM can actually reach mass production will be one of the most important hardware variables to track in the coming six months.