NVIDIA A100 40GB PCIe
Data center GPU · Ampere
Summary
One of four A100 configurations.
For a version of this GPU with more VRAM, see the A100 PCIe 80GB.
For multi-GPU training where interconnects are important, consider the A100 SXM 80GB or A100 SXM 40GB.
Launched
Q3 2020
VRAM
40 GB
Mem. bandwidth
1,555 GB/s
On-demand from
Tech specs
VRAM 40 GB HBM2
Memory bandwidth 1,555 GB/s
Interface PCIe
CUDA cores 6,912
Tensor cores 432 (Gen 3)
TDP 250 W
Supported data types
FP64FP32FP16BF16INT8INT4
Cloud rental prices
4 providers · lowest price per GPU, per hour
On-demand
$0.89 – $1.99 /GPU/h
Reserved
$0.76 – $2.87 /GPU/h
Spot
$0.79 /GPU/h
| Provider | On-demand | Reserved | Spot |
|---|---|---|---|
 Cirrascale | — | $2.30 | — |
 Denvr | $1.15 | — | — |
| | $0.89 | $0.76 | $0.79 |
 Lambda | $1.99 | — | — |
Models that fit in VRAM
Open-weights models that fit in VRAM. Estimated using 🤗 accelerate, plus approximation for up to 8K context.
Peak theoretical performance
INT8 Tensor Core 624 TOPS
BF16 Tensor Core 312 TFLOPS
FP16 Tensor Core 312 TFLOPS
TF32 Tensor Core 156 TFLOPS
FP32 19.5 TFLOPS
FP64 9.7 TFLOPS
FP64 Tensor Core 19.5 TFLOPS
Performance figures assume no sparsity; in cases where only sparse performance figures are published by the manufacturer, these are halved to give approximate dense performance.
Resources
Detailed documentation from the manufacturer.
Similar GPUs
Other accelerators you might compare.