Nvidia Extends NVLink Fusion with Custom NVHBM Offering Up to 30% More Bandwidth and 15% Lower Power Consumption
Nvidia announced on August 26 that its NVLink Fusion program now includes NVHBM, a custom high‑bandwidth memory (HBM) architecture that moves the memory controller into the base die of the 3‑D stack. Nvidia claims the design can deliver up to 30 % more memory bandwidth per stack, 15 % lower HBM power consumption and up to 25 % more usable compute area compared with standard HBM4E samples, which are still in the sampling stage and slated for volume shipment around 2027.
Key Takeaways
- NVHBM relocates the memory controller from the GPU die to the HBM stack’s base die, shrinking the interface by up to 67 %.
- Nvidia measures NVHBM against early HBM4E samples, promising 30 % higher bandwidth, 15 % lower power and 25 % more compute area.
- Access to NVHBM is limited to partners in the NVLink Fusion program; Amazon’s Annapurna Labs is the first named participant.
- Similar concepts were disclosed by Marvell in December 2024, but Nvidia’s distribution model differs.
- NVHBM will not appear in products before HBM4E reaches mass production, which is expected in 2027.
| Metric | Value | Why it matters |
|---|---|---|
| Bandwidth gain | Up to 30 % more per stack | Higher data rates accelerate large AI model training. |
| Power reduction | 15 % lower HBM power | Improves efficiency in rack‑scale AI deployments. |
| Compute‑area increase | Up to 25 % more usable area | Leaves more silicon for cores and accelerators. |
| Interface shrinkage | Up to 67 % reduction vs. JEDEC HBM4E | Frees die real‑estate and shortens signal paths. |
What NVHBM Is and How It Works
NVHBM is not a new memory cell technology; instead, Nvidia re‑architects where the memory controller resides. In conventional HBM designs, the DRAM stack and its base die are supplied by a memory vendor, while the accelerator (GPU) die houses the controller and the physical interface. The JEDEC standard defines a wide parallel bus between the two, which adds latency and consumes die area.
Nvidia’s approach integrates the controller directly into the base die of the 3‑D stack, replacing the JEDEC bus with a narrower, serialized die‑to‑die link that Nvidia designs. This eliminates the “seam” between vendor and accelerator, allowing the company to claim up to a 67 % reduction in interface and support area while still improving bandwidth and power metrics.
Performance Claims Against HBM4E
The company benchmarked NVHBM against early HBM4E samples, which are still in the sampling phase with a projected volume launch sometime in 2027. According to Nvidia, the custom stack delivers:
- 30 % higher memory bandwidth per stack.
- 15 % lower power draw for the HBM subsystem.
- Up to 25 % more usable compute area on the accelerator die.
These figures are positioned as a response to the “growing performance limitations for frontier‑level AI models” that are increasingly constrained by memory bandwidth.
Partner Access and Program Gating
NVHBM is gated behind Nvidia’s NVLink Fusion program, which connects third‑party accelerators to Nvidia’s rack‑scale platform. As of the announcement, Amazon’s Annapurna Labs is the only publicly named partner. Nvidia’s blog notes that Annapurna will support NVLink Fusion with its Trainium 4 accelerator, but it does not explicitly confirm that Trainium 4 will use NVHBM.
Because NVHBM is tied to NVLink Fusion, any company wishing to adopt the memory architecture must join the program and meet Nvidia’s qualification criteria, effectively limiting early access to a small set of partners.
Industry Context and Similar Efforts
The concept of moving the controller into the stack’s base die is not exclusive to Nvidia. In December 2024, Marvell announced a comparable architecture in collaboration with Micron, Samsung and SK Hynix. Marvell’s figures—up to 25 % more compute area, 33 % greater memory capacity and a 70 % reduction in memory‑interface power—are close to Nvidia’s projections.
“The technology is not new. The distribution is.” – Neil Shah, analyst at Counterpoint Research
The distinction, according to analysts, lies in how the technology is packaged and which ecosystem it is tied to. Nvidia’s distribution model leverages NVLink Fusion, whereas Marvell’s approach is positioned as a more open solution for memory vendors and accelerator designers.
Why It Matters
If Nvidia’s bandwidth and power claims hold up in silicon, NVHBM could alleviate a key bottleneck in training ever‑larger AI models, where memory throughput often limits scaling. The reduced interface area also frees up silicon for additional compute units, potentially enabling denser GPU designs. However, the gating mechanism means that only a handful of partners will see the technology before HBM4E itself reaches mass production, likely delaying broader industry impact until after 2027.
Frequently asked questions
What is the fundamental difference between NVHBM and standard HBM4E?
NVHBM relocates the memory controller from the accelerator die into the base die of the HBM stack, replacing the wide JEDEC parallel bus with a narrower serialized link that Nvidia designs.
When is NVHBM expected to be available in products?
NVHBM is tied to the NVLink Fusion program and will not ship before HBM4E reaches volume production, which is anticipated sometime in 2027.
Which companies currently have access to NVHBM?
Amazon’s Annapurna Labs is the only publicly announced partner. Access is controlled through the NVLink Fusion partner list.
How does NVHBM compare to Marvell’s similar memory‑controller‑in‑stack approach?
Both aim to reduce interface area and power while increasing compute density. Marvell’s public numbers (up to 25 % more compute area, 33 % more capacity, 70 % power reduction) are comparable, but Nvidia’s solution is distributed exclusively via its NVLink Fusion ecosystem.
What impact could NVHBM have on AI training workloads?
Higher bandwidth and lower power consumption can speed up data movement for large models, potentially reducing training time and operational costs in rack‑scale AI deployments.
Bottom Line
NVHBM represents a strategic redesign of HBM architecture that promises measurable gains in bandwidth, power efficiency and compute area, but its availability is limited to a narrow set of NVLink Fusion partners and will likely lag behind the broader rollout of HBM4E. Companies that secure early access could gain a performance edge for frontier AI workloads, while the rest of the market will have to wait until the technology matures post‑2027.
Reporting based on: original source.
Comments
Post a Comment