NVIDIA H100
Data center GPU · Hopper · SXM, PCIe, NVL variants
NVIDIA's flagship Hopper-generation data center GPU. The H100's three variants share the same core Hopper architecture but differ in performance, form factor, interconnect, memory, and power.
The H100 SXM is the highest-performance variant, and the most widely available. It supports fast NVLink connectivity between GPUs, making it the default for multi-GPU training. The H100 PCIe is a lower-power card in a PCIe form factor. It is well suited to inference and single- or few-GPU training jobs. The H100 NVL has more memory (94 GB), alongside higher memory bandwidth, and is targeted at LLM inference.
Tech specs
| Spec | H100 SXM | H100 PCIe | H100 NVL |
|---|---|---|---|
| VRAM | 80 GB HBM3 | 80 GB HBM2e | 94 GB HBM3 |
| Memory bandwidth | 3,350 GB/s | 2,000 GB/s | 3,900 GB/s |
| Interface | SXM | PCIe | PCIe |
| CUDA cores | 16,896 | 14,592 | 14,592 |
| Tensor cores | 528 (Gen 4) | 456 (Gen 4) | 456 (Gen 4) |
| TDP | 700 W | 350 W | 400 W |
Cloud rental prices
28 providers · available in 31 countries · lowest price per GPU, per hour
| Provider | On-demand | Reserved | Spot |
|---|---|---|---|
 AceCloud | — | $3.26 | — |
 AWS | $6.88 | $2.97 | — |
| | $11.06 | $4.86 | $2.04 |
 CoreWeave | $6.16 | — | $2.46 |
 Crusoe | $3.90 | — | — |
 Denvr | $2.30 | — | — |
| | $11.06 | $4.86 | $1.13 |
 Hyperstack | $3.20 | $2.72 | — |
| | $2.69 | $2.32 | $1.19 |
 Lambda | $3.99 | — | — |
 Leafcloud | $4.71 | — | — |
 Massed Compute | $3.14 | — | — |
 Nebius | $3.85 | — | $2.15 |
 Oracle | $10.00 | — | — |
 Runpod | $2.99 | — | — |
| | $3.62 | — | — |
 Seeweb | $2.16 | $1.84 | — |
| | $2.75 | — | — |
 Together AI | $3.99 | — | — |
 Vast.ai | $2.20 | — | $1.60 |
 Verda | $3.25 | $2.99 | $1.14 |
 Voltage Park | $1.99 | — | — |
 Vultr | $2.30 | — | — |
| Provider | On-demand | Reserved | Spot |
|---|---|---|---|
 DigitalOcean | $2.99 | — | — |
| Hyperstack | $2.50 | $1.75 | — |
| Lambda | $3.29 | — | — |
 Latitude.sh | $3.37 | $1.18 | — |
 LeaderGPU | — | $2.21 | — |
| Massed Compute | $2.73 | — | — |
 OVHcloud | $3.20 | — | — |
| Runpod | $2.89 | — | — |
| | $3.28 | — | — |
| Provider | On-demand | Reserved | Spot |
|---|---|---|---|
 Atlantic.Net | $3.94 | $3.58 | — |
| | $6.98 | $3.84 | $1.40 |
| Massed Compute | $2.92 | — | — |
| Runpod | $3.19 | — | — |
| Vast.ai | — | — | $0.81 |
Models that fit in VRAM
Open-weights models that fit in VRAM. Estimated using 🤗 accelerate, plus approximation for up to 8K context.
Media
Peak theoretical performance
| Spec | H100 SXM | H100 PCIe | H100 NVL |
|---|---|---|---|
| FP8 Tensor Core | 1,979 TFLOPS | 1,513 TFLOPS | 1,670 TFLOPS |
| INT8 Tensor Core | 1,979 TOPS | 1,513 TOPS | 1,670 TOPS |
| BF16 Tensor Core | 990 TFLOPS | 756 TFLOPS | 835 TFLOPS |
| FP16 Tensor Core | 990 TFLOPS | 756 TFLOPS | 835 TFLOPS |
| TF32 Tensor Core | 494 TFLOPS | 378 TFLOPS | 417 TFLOPS |
| FP32 | 67 TFLOPS | 51 TFLOPS | 60 TFLOPS |
| FP64 | 34 TFLOPS | 26 TFLOPS | 30 TFLOPS |
| FP64 Tensor Core | 67 TFLOPS | 51 TFLOPS | 60 TFLOPS |
Performance figures assume no sparsity; in cases where only sparse performance figures are published by the manufacturer, these are halved to give approximate dense performance.
Resources
Detailed documentation from the manufacturer.
Similar GPUs
Other accelerators you might compare.