NVIDIA L40S
Data center GPU · Ada Lovelace
Summary
NVIDIA's L40s FP8 performance makes it a great inference workhorse. For graphics/rendering workloads, consider the similar (but usually cheaper) NVIDIA L40. For multi-GPU training, look instead to the H100 or B200.
Launched
Q3 2023
VRAM
48 GB
Mem. bandwidth
864 GB/s
On-demand from
Tech specs
VRAM 48 GB GDDR6
Memory bandwidth 864 GB/s
Interface PCIe
CUDA cores 18,176
Tensor cores 568 (Gen 4)
TDP 350 W
Supported data types
FP64FP32TF32FP16BF16FP8INT8
Cloud rental prices
17 providers · available in 14 countries · lowest price per GPU, per hour
On-demand
$0.88 – $3.77 /GPU/h
Reserved
$0.78 – $2.37 /GPU/h
Spot
$0.48 – $0.99 /GPU/h
| Provider | On-demand | Reserved | Spot |
|---|---|---|---|
 AceCloud | — | $1.49 | — |
 Atlantic.Net | $1.67 | $1.58 | — |
 AWS | $1.86 | $0.80 | — |
 CoreWeave | $2.25 | — | $0.99 |
 Crusoe | $1.50 | — | — |
 DigitalOcean | $1.57 | — | — |
 LeaderGPU | $1.62 | $0.78 | — |
 Massed Compute | $0.88 | — | — |
 Nebius | $1.35 | — | $0.65 |
 Oracle | $3.50 | — | — |
 OVHcloud | $1.59 | — | — |
 Runpod | $0.99 | — | — |
| | $1.67 | — | — |
 Seeweb | $0.97 | $0.82 | — |
| | $0.97 | — | — |
 Verda | $1.37 | $1.26 | $0.48 |
 Vultr | $1.67 | — | — |
Models that fit in VRAM
Open-weights models that fit in VRAM. Estimated using 🤗 accelerate, plus approximation for up to 8K context.
Media
Peak theoretical performance
FP8 Tensor Core 733 TFLOPS
INT8 Tensor Core 733 TOPS
INT4 Tensor Core 733 TOPS
BF16 Tensor Core 362 TFLOPS
FP16 Tensor Core 362 TFLOPS
TF32 Tensor Core 183 TFLOPS
FP32 91.6 TFLOPS
Performance figures assume no sparsity; in cases where only sparse performance figures are published by the manufacturer, these are halved to give approximate dense performance.
Resources
Detailed documentation from the manufacturer.