What can an RTX 4060 run?

The RTX 4060 has 8GB of VRAM and 272 GB/s of memory bandwidth. Run through the same engine as our calculator, it runs 16 of 51 current local LLMs fully on the GPU — the biggest is Qwen3.5 9B.

VRAM

8 GB

Bandwidth

272 GB/s

Models that fit

16/51

Llama 3.1 8B

~34 tok/s

See it on the cardPick a model — the fans do the math.

– of your card used

VRAM needed: – of 8 GB

Best quant: – · Est. speed: –

Choose a model to check the fit.

Change the context, quant or RAM and see live results for the RTX 4060.

Open the RTX 4060 in the calculator →

Every model, checked against the RTX 4060

Best quant means the highest-quality quantization that still fits fully in 8GB at 8K context. Speeds are llama.cpp estimates from memory bandwidth — the same math the calculator uses.

Model	Best quant	VRAM needed	Est. speed	Status
Llama 3.2 1B Llama	Q8_0	2.3 GB	~146 tok/s	Runs on GPU
Qwen3 0.6B Qwen	Q8_0	1.7 GB	~200 tok/s	Runs on GPU
Gemma 3 1B Gemma	Q8_0	2.0 GB	~177 tok/s	Runs on GPU
Llama 3.2 3B Llama	Q8_0	5.1 GB	~53 tok/s	Runs on GPU
Qwen3 1.7B Qwen	Q8_0	3.4 GB	~84 tok/s	Runs on GPU
Phi-4-mini Phi	Q8_0	5.9 GB	~45 tok/s	Runs on GPU
Qwen3 4B Qwen	Q8_0	5.8 GB	~46 tok/s	Runs on GPU
Gemma 3 4B Gemma	Q8_0	6.8 GB	~39 tok/s	Runs on GPU
Llama 3.1 8B Llama	Q5_K_M	7.6 GB	~34 tok/s	Runs on GPU
Qwen3 8B Qwen	Q5_K_M	7.9 GB	~33 tok/s	Runs on GPU
Mistral 7B v0.3 Mistral	Q6_K	7.9 GB	~33 tok/s	Runs on GPU
Ministral 8B Mistral	Q5_K_M	7.8 GB	~34 tok/s	Runs on GPU
gpt-oss 20B OpenAI	—	14.2 GB	—	Won't fit
Qwen3 14B Qwen	—	11.3 GB	—	Won't fit
Phi-4 14B Phi	—	11.6 GB	—	Won't fit
Gemma 3 12B Gemma	—	11.4 GB	—	Won't fit
Mistral Small 3.1 24B Mistral	—	17.5 GB	—	Won't fit
Gemma 3 27B Gemma	—	23.4 GB	—	Won't fit
Gemma 4 31B Gemma	—	25.3 GB	—	Won't fit
Gemma 4 26B-A4B Gemma	—	18.0 GB	—	Won't fit
Qwen3 32B Qwen	—	22.7 GB	—	Won't fit
Qwen3 30B-A3B Qwen	—	20.3 GB	—	Won't fit
Qwen3-Coder 30B-A3B Qwen	—	20.3 GB	—	Won't fit
DeepSeek-R1 Distill 32B DeepSeek	—	23.5 GB	—	Won't fit
Llama 3.3 70B Llama	—	48.0 GB	—	Won't fit
Llama 4 Scout Llama	—	71.3 GB	—	Won't fit
gpt-oss 120B OpenAI	—	75.2 GB	—	Won't fit
Qwen3 235B-A22B Qwen	—	150.4 GB	—	Won't fit
Llama 4 Maverick Llama	—	255.8 GB	—	Won't fit
DeepSeek-V3 DeepSeek	—	427.8 GB	—	Won't fit
DeepSeek-R1 DeepSeek	—	427.8 GB	—	Won't fit
Qwen3.5 9B Qwen	Q5_K_M	7.6 GB	~35 tok/s	Runs on GPU
Qwen3.5 35B-A3B Qwen	—	22.9 GB	—	Won't fit
Qwen3.6 35B-A3B Qwen	—	22.9 GB	—	Won't fit
Qwen3.6 27B Qwen	—	18.2 GB	—	Won't fit
Qwen3.8 27B Qwen	—	18.7 GB	—	Won't fit
Gemma 4 E4B Gemma	Q5_K_M	7.3 GB	~59 tok/s	Runs on GPU
DeepSeek V4-Flash DeepSeek	—	181.4 GB	—	Won't fit
GLM-5.3-Flash GLM	—	203.6 GB	—	Won't fit
Kimi K2.6 Kimi	—	635.6 GB	—	Won't fit
MiniMax M3 MiniMax	—	272.9 GB	—	Won't fit
Mistral Small 4 Mistral	—	76.3 GB	—	Won't fit
Kimi K3 Kimi	—	1763.3 GB	—	Won't fit
Muse Spark Open Muse	—	520.0 GB	—	Won't fit
Flint Recursive	—	26.1 GB	—	Won't fit
Beam 1.0 Reflection	—	318.7 GB	—	Won't fit
MiniMax M3.1 Flash MiniMax	—	272.9 GB	—	Won't fit
Mistral Large 4 Mistral	—	667.3 GB	—	Won't fit
Ling-3.1-flash-1019 Ling	—	355.8 GB	—	Won't fit
LFM3-1.7B Liquid	Q8_0	2.6 GB	~119 tok/s	Runs on GPU
LFM3-VL-2B Liquid	Q8_0	2.8 GB	~108 tok/s	Runs on GPU

Shopping for this card? See RTX 4060 prices on Amazon → (affiliate link — it never changes the recommendations, which are physics-based)

Want to run 70B models?

A 70B model at Q4 needs about 48 GB of VRAM — more than the RTX 4060's 8GB. The cheapest single card that runs Llama 3.3 70B fully on-GPU is the Apple M5 Pro (64GB) (54GB). Check Apple M5 Pro (64GB) prices on Amazon →

Or skip the hardware: rent a Apple M5 Pro (64GB)-class GPU by the hour and run 70B today. RunPod · Vast.ai

RTX 4060 FAQ

What's the biggest LLM an RTX 4060 can run?

Fully on-GPU at 8K context, the biggest model that fits the RTX 4060 (8GB) is Qwen3.5 9B at Q5_K_M (~35 tok/s). Larger models need a card with more VRAM or system-RAM offload, which is much slower.

How fast is an RTX 4060 for local AI?

On a mid-size model like Llama 3.1 8B (Q5_K_M), expect roughly 34 tokens/sec. Speed scales with memory bandwidth (272 GB/s) more than anything else.

Can an RTX 4060 run a 70B model?

Not fully on-GPU: a 70B at Q4 needs about 48 GB and the RTX 4060 has 8GB. It can crawl with system-RAM offload, but a bigger card or a rented cloud GPU is the practical route.

Is the RTX 4060 good for local AI in 2026?

It runs 16 of the 51 models we track — a workable entry point for small and mid-size models. The full breakdown is in the table above.

Keep exploring

More GPU guides: RTX 5090 · RTX 5080 · RTX 5070 · RTX 5060 Ti 16GB · All GPU guides