Gaudi 2
Data center GPU · Gaudi 2
Summary
Despite outperforming the NVIDIA A100, the Intel Gaudi 2 was quickly superseded by the NVIDIA H100, and is not widely available across cloud providers.
Launched
Q2 2022
VRAM
96 GB
Mem. bandwidth
2,450 GB/s
On-demand from
Tech specs
VRAM 96 GB HBM2e
Memory bandwidth 2,450 GB/s
Interface OAM
TDP 600 W
Supported data types
FP32TF32FP16BF16FP8
Cloud rental prices
2 providers · lowest price per GPU, per hour
On-demand
$1.25 /GPU/h
Reserved
$0.68 – $1.04 /GPU/h
Models that fit in VRAM
| Total VRAM | From | 16-bit inference | 8-bit inference | 4-bit inference | |
|---|---|---|---|---|---|
| 8× Gaudi 2 | 768 GB | $5.46/h | | | |
Open-weights models that fit in VRAM. Estimated using 🤗 accelerate, plus approximation for up to 8K context.
Media
Resources
Detailed documentation from the manufacturer.
Similar GPUs
Other accelerators you might compare.