What can an RTX 4090 run?

The RTX 4090 has 24GB of VRAM and 1008 GB/s of memory bandwidth. Run through the same engine as our calculator, it runs 33 of 51 current local LLMs fully on the GPU — the biggest is Flint.

VRAM

24 GB

Bandwidth

1008 GB/s

Models that fit

33/51

Llama 3.1 8B

~89 tok/s

See it on the cardPick a model — the fans do the math.

– of your card used

VRAM needed: – of 24 GB

Best quant: – · Est. speed: –

Choose a model to check the fit.

Change the context, quant or RAM and see live results for the RTX 4090.

Open the RTX 4090 in the calculator →

Every model, checked against the RTX 4090

Best quant means the highest-quality quantization that still fits fully in 24GB at 8K context. Speeds are llama.cpp estimates from memory bandwidth — the same math the calculator uses.

Model	Best quant	VRAM needed	Est. speed	Status
Llama 3.2 1B Llama	Q8_0	2.3 GB	~200 tok/s	Runs on GPU
Qwen3 0.6B Qwen	Q8_0	1.7 GB	~200 tok/s	Runs on GPU
Gemma 3 1B Gemma	Q8_0	2.0 GB	~200 tok/s	Runs on GPU
Llama 3.2 3B Llama	Q8_0	5.1 GB	~197 tok/s	Runs on GPU
Qwen3 1.7B Qwen	Q8_0	3.4 GB	~200 tok/s	Runs on GPU
Phi-4-mini Phi	Q8_0	5.9 GB	~168 tok/s	Runs on GPU
Qwen3 4B Qwen	Q8_0	5.8 GB	~171 tok/s	Runs on GPU
Gemma 3 4B Gemma	Q8_0	6.8 GB	~143 tok/s	Runs on GPU
Llama 3.1 8B Llama	Q8_0	10.6 GB	~89 tok/s	Runs on GPU
Qwen3 8B Qwen	Q8_0	11.0 GB	~86 tok/s	Runs on GPU
Mistral 7B v0.3 Mistral	Q8_0	9.8 GB	~98 tok/s	Runs on GPU
Ministral 8B Mistral	Q8_0	10.7 GB	~88 tok/s	Runs on GPU
gpt-oss 20B OpenAI	Q6_K	19.0 GB	~200 tok/s	Runs on GPU
Qwen3 14B Qwen	Q8_0	18.5 GB	~50 tok/s	Runs on GPU
Phi-4 14B Phi	Q8_0	18.7 GB	~50 tok/s	Runs on GPU
Gemma 3 12B Gemma	Q8_0	17.2 GB	~54 tok/s	Runs on GPU
Mistral Small 3.1 24B Mistral	Q6_K	22.9 GB	~40 tok/s	Runs on GPU
Gemma 3 27B Gemma	Q4_K_M	23.4 GB	~39 tok/s	Runs on GPU
Gemma 4 31B Gemma	Q3_K_M	21.6 GB	~42 tok/s	Runs on GPU
Gemma 4 26B-A4B Gemma	Q6_K	23.7 GB	~184 tok/s	Runs on GPU
Qwen3 32B Qwen	Q4_K_M	22.7 GB	~41 tok/s	Runs on GPU
Qwen3 30B-A3B Qwen	Q5_K_M	23.7 GB	~200 tok/s	Runs on GPU
Qwen3-Coder 30B-A3B Qwen	Q5_K_M	23.7 GB	~200 tok/s	Runs on GPU
DeepSeek-R1 Distill 32B DeepSeek	Q4_K_M	23.5 GB	~39 tok/s	Runs on GPU
Llama 3.3 70B Llama	—	48.0 GB	—	Won't fit
Llama 4 Scout Llama	—	71.3 GB	—	Won't fit
gpt-oss 120B OpenAI	—	75.2 GB	—	Won't fit
Qwen3 235B-A22B Qwen	—	150.4 GB	—	Won't fit
Llama 4 Maverick Llama	—	255.8 GB	—	Won't fit
DeepSeek-V3 DeepSeek	—	427.8 GB	—	Won't fit
DeepSeek-R1 DeepSeek	—	427.8 GB	—	Won't fit
Qwen3.5 9B Qwen	Q8_0	10.9 GB	~87 tok/s	Runs on GPU
Qwen3.5 35B-A3B Qwen	Q4_K_M	22.9 GB	~200 tok/s	Runs on GPU
Qwen3.6 35B-A3B Qwen	Q4_K_M	22.9 GB	~200 tok/s	Runs on GPU
Qwen3.6 27B Qwen	Q5_K_M	21.1 GB	~44 tok/s	Runs on GPU
Qwen3.8 27B Qwen	Q5_K_M	21.7 GB	~43 tok/s	Runs on GPU
Gemma 4 E4B Gemma	Q8_0	10.2 GB	~156 tok/s	Runs on GPU
DeepSeek V4-Flash DeepSeek	—	181.4 GB	—	Won't fit
GLM-5.3-Flash GLM	—	203.6 GB	—	Won't fit
Kimi K2.6 Kimi	—	635.6 GB	—	Won't fit
MiniMax M3 MiniMax	—	272.9 GB	—	Won't fit
Mistral Small 4 Mistral	—	76.3 GB	—	Won't fit
Kimi K3 Kimi	—	1763.3 GB	—	Won't fit
Muse Spark Open Muse	—	520.0 GB	—	Won't fit
Flint Recursive	Q3_K_M	21.3 GB	~200 tok/s	Runs on GPU
Beam 1.0 Reflection	—	318.7 GB	—	Won't fit
MiniMax M3.1 Flash MiniMax	—	272.9 GB	—	Won't fit
Mistral Large 4 Mistral	—	667.3 GB	—	Won't fit
Ling-3.1-flash-1019 Ling	—	355.8 GB	—	Won't fit
LFM3-1.7B Liquid	Q8_0	2.6 GB	~200 tok/s	Runs on GPU
LFM3-VL-2B Liquid	Q8_0	2.8 GB	~200 tok/s	Runs on GPU

Shopping for this card? See RTX 4090 prices on Amazon → (affiliate link — it never changes the recommendations, which are physics-based)

Want to run 70B models?

A 70B model at Q4 needs about 48 GB of VRAM — more than the RTX 4090's 24GB. The cheapest single card that runs Llama 3.3 70B fully on-GPU is the Apple M5 Pro (64GB) (54GB). Check Apple M5 Pro (64GB) prices on Amazon →

Or skip the hardware: rent a Apple M5 Pro (64GB)-class GPU by the hour and run 70B today. RunPod · Vast.ai

RTX 4090 FAQ

What's the biggest LLM an RTX 4090 can run?

Fully on-GPU at 8K context, the biggest model that fits the RTX 4090 (24GB) is Flint at Q3_K_M (~200 tok/s). Larger models need a card with more VRAM or system-RAM offload, which is much slower.

How fast is an RTX 4090 for local AI?

On a mid-size model like Llama 3.1 8B (Q8_0), expect roughly 89 tokens/sec. Speed scales with memory bandwidth (1008 GB/s) more than anything else.

Can an RTX 4090 run a 70B model?

Not fully on-GPU: a 70B at Q4 needs about 48 GB and the RTX 4090 has 24GB. It can crawl with system-RAM offload, but a bigger card or a rented cloud GPU is the practical route.

Is the RTX 4090 good for local AI in 2026?

It runs 33 of the 51 models we track — a strong local-AI card. The full breakdown is in the table above.

Keep exploring

More GPU guides: RTX 5090 · RTX 5080 · RTX 5070 · RTX 5060 Ti 16GB · All GPU guides