NVIDIA A100
Data center GPU · Ampere · SXM/PCIe, 80GB/40GB variants
NVIDIA's flagship Ampere-generation data center GPU. The A100's four variants share the same core Ampere architecture but differ in memory and interface, as well as memory bandwidth and power.
The A100 SXM 80GB is the highest-performance variant, and the most widely available. Together with the lower-memory A100 SXM 40GB, the A100's two SXM variants are the ones better suited to multi-GPU training, with fast interconnects between each of the GPUs.
The A100 PCIe 80GB and A100 PCIe 40GB can be a good fit for workloads requiring just one or two GPUs.
Tech specs
| Spec | A100 80GB SXM | A100 40GB SXM | A100 80GB PCIe | A100 40GB PCIe |
|---|---|---|---|---|
| VRAM | 80 GB HBM2e | 40 GB HBM2 | 80 GB HBM2e | 40 GB HBM2 |
| Memory bandwidth | 2,039 GB/s | 1,555 GB/s | 1,935 GB/s | 1,555 GB/s |
| Interface | SXM | SXM | PCIe | PCIe |
| CUDA cores | 6,912 | 6,912 | 6,912 | 6,912 |
| Tensor cores | 432 (Gen 3) | 432 (Gen 3) | 432 (Gen 3) | 432 (Gen 3) |
| TDP | 400 W | 400 W | 300 W | 250 W |
Cloud rental prices
21 providers · available in 30 countries · lowest price per GPU, per hour
| Provider | On-demand | Reserved | Spot |
|---|---|---|---|
 AceCloud | — | $1.63 | — |
 AWS | $3.43 | $1.46 | — |
| | $3.40 | $1.36 | $0.75 |
 CoreWeave | $2.70 | — | $1.21 |
 Crusoe | $2.30 | — | — |
 Denvr | $0.58 | — | — |
| | $5.07 | — | $0.65 |
 Hyperstack | $1.60 | $1.36 | — |
 Lambda | $2.79 | — | — |
 Massed Compute | $1.38 | — | — |
 Oracle | $4.00 | — | — |
 OVHcloud | $3.13 | — | — |
 Runpod | $1.49 | — | — |
| | $1.52 | — | — |
 Verda | $1.79 | $1.65 | $0.63 |
 Vultr | $2.80 | — | — |
| Provider | On-demand | Reserved | Spot |
|---|---|---|---|
| AWS | $2.74 | $1.17 | — |
| Denvr | $1.15 | — | — |
| | $3.48 | $1.22 | $0.32 |
| Lambda | $1.99 | — | — |
| Oracle | $3.05 | — | — |
| | $1.54 | — | — |
| Verda | $1.29 | $1.19 | $0.45 |
| Provider | On-demand | Reserved | Spot |
|---|---|---|---|
| | $3.67 | $1.36 | $0.68 |
 Cirrascale | — | $2.60 | — |
| Crusoe | $2.00 | — | — |
| Hyperstack | $1.35 | $0.95 | $1.08 |
| | $1.49 | $1.27 | $0.89 |
 LeaderGPU | $1.03 | $0.57 | — |
 Leafcloud | $1.82 | — | — |
| Massed Compute | $1.35 | — | — |
| Runpod | $1.39 | — | — |
 Seeweb | $1.13 | $0.96 | — |
| Vultr | $2.40 | — | — |
| Provider | On-demand | Reserved | Spot |
|---|---|---|---|
| Cirrascale | — | $2.30 | — |
| Denvr | $1.15 | — | — |
| | $0.89 | $0.76 | $0.79 |
| Lambda | $1.99 | — | — |
Models that fit in VRAM
Open-weights models that fit in VRAM. Estimated using 🤗 accelerate, plus approximation for up to 8K context.
Media
Peak theoretical performance
| Spec | A100 80GB SXM | A100 40GB SXM | A100 80GB PCIe | A100 40GB PCIe |
|---|---|---|---|---|
| INT8 Tensor Core | 624 TOPS | 624 TOPS | 624 TOPS | 624 TOPS |
| BF16 Tensor Core | 312 TFLOPS | 312 TFLOPS | 312 TFLOPS | 312 TFLOPS |
| FP16 Tensor Core | 312 TFLOPS | 312 TFLOPS | 312 TFLOPS | 312 TFLOPS |
| TF32 Tensor Core | 156 TFLOPS | 156 TFLOPS | 156 TFLOPS | 156 TFLOPS |
| FP32 | 19.5 TFLOPS | 19.5 TFLOPS | 19.5 TFLOPS | 19.5 TFLOPS |
| FP64 | 9.7 TFLOPS | 9.7 TFLOPS | 9.7 TFLOPS | 9.7 TFLOPS |
| FP64 Tensor Core | 19.5 TFLOPS | 19.5 TFLOPS | 19.5 TFLOPS | 19.5 TFLOPS |
Performance figures assume no sparsity; in cases where only sparse performance figures are published by the manufacturer, these are halved to give approximate dense performance.
Resources
Detailed documentation from the manufacturer.
Similar GPUs
Other accelerators you might compare.