How to Run an LLM Locally: A Beginner's Guide
You can run your own private AI — a ChatGPT-style assistant — right on your computer. It's free, works offline, and nothing you type leaves your machine. Here's the whole thing in 4 steps.
Check what your GPU can run
The #1 beginner mistake is downloading a model too big for your graphics card — it crashes or crawls. Go to willitrunit.com, pick your GPU, and it shows every model that fits and the right quant to use. 30 seconds, saves hours.
Install an app
Easiest: LM Studio (friendly app, search + click to download). Prefer the terminal? Ollama — then ollama run llama3. See our Ollama vs LM Studio vs Jan guide to choose.
Download a model that fits
Pick a model at the size/quant the calculator recommended for your VRAM (e.g. a 7B or 13B at Q4). Smaller = faster; bigger = smarter but heavier. Start small, then size up.
Start chatting
Open the chat and go. That's a private AI running entirely on your hardware — no subscription, no rate limits, no data leaving your PC.
What you need
A PC with a dedicated GPU (8GB+ VRAM is a comfy start) or a Mac with Apple Silicon. Not sure yours is enough? Read Can Your Gaming PC Run Local AI? If you're shopping, see the best budget GPUs.