AMD MI300X
Data center GPU · CDNA3
The most widely-available AMD GPU, the MI300X has better specs than the NVIDIA H200, but is generally available at lower prices.
Its additional VRAM and higher memory bandwidth is especially useful for LLM inference applications. Software support for AMD GPUs has historically been less good than for equivalent NVIDIA GPUs, though this has been improving.
For a similar but slightly more powerful GPU, see the AMD MI325X.
Tech specs
Cloud rental prices
7 providers · available in 2 countries · lowest price per GPU, per hour
Additional providers who may offer AMD MI300X without public pricing: TensorWave, Runpod
Models that fit in VRAM
Open-weights models that fit in VRAM. Estimated using 🤗 accelerate, plus approximation for up to 8K context.
Media
Peak theoretical performance
Performance figures assume no sparsity; in cases where only sparse performance figures are published by the manufacturer, these are halved to give approximate dense performance.
Resources
Detailed documentation from the manufacturer.
Similar GPUs
Other accelerators you might compare.