CloudYeti / Local AI primer

The local AI primer: run AI on your own computer in about an hour

You can run a real AI model on your own Mac or PC. There is no API bill, and your data never leaves your machine. This page covers the four decisions you have to make: your hardware, your runner, your model, and your first real task.

I think this is the best place to start with local AI, because it is short on purpose. You learn what matters, get a model running, and get back to your own work.

1. Why run AI locally

It costs almost nothing per question. You already paid for the computer. Ask a local model a thousand questions and there is no API bill. Electricity is the one cost you still pay.

It is private by default. Contracts, medical notes and private journals stay on your machine. Nothing is sent to an AI company.

It always works. On a plane, while camping, or when your AI provider's API is down, a local model still runs.

You can choose the model. There are thousands of open models, including fine-tuned versions for specific jobs. Some of them answer questions that ChatGPT and Claude refuse.

Local models are good today at chat, writing, summarizing documents, code help, working with private data, and small always-on agents. Local AI today is roughly where paid cloud models were a year ago.

2. What your machine can run

A model is a large set of numbers stored in files. To run at a usable speed, most or all of those numbers need to fit in fast memory. If the model does not fit, it gets much slower or does not run at all.

Here is the rough map of what runs comfortably. The "B" means billions of parameters.

Memory (VRAM or unified)Model size that runs comfortablyWhat it feels like
8 GB3B to 4BQuick chat and simple tasks
16 GB12B to 14BSolid everyday helper
32 GBAbout 30BSeriously capable, including coding
64 GB and up70B classGetting close to cloud quality

3. Pick a runner

A runner is the app that loads the model and lets you talk to it. There are dozens. You only need to know three:

Most runners are wrappers around the same llama.cpp engine. On Apple Silicon there is also Apple's MLX engine, and Ollama and LM Studio both support it now. Pick one and move on.

4. Pick a model and read its name

A model name works like a license plate. Take qwen3-coder:14b-q4:

Quantization shrinks a model so it fits in less memory, with a small loss in quality. For example, a model that is 28 GB at full size is about 12 GB at Q5 and about 9 GB at Q4. Many setups use Q4 because it is fast and the quality loss is usually small.

Dense vs. mixture of experts (MoE). A dense 30B model uses all 30 billion parameters for every word, so it is heavier and slower. An MoE model has the same total size but only part of it is active at a time. A name like 35b-a3b means 35 billion total, 3 billion active. An MoE model still needs the memory for its full size, but the active size tells you how fast it will feel.

Specific model picks change every month, so check what is new before you download. The course keeps a current shortlist for each memory size.

5. Run your first model

Install Ollama from ollama.com, then run a model from your terminal:

ollama run qwen3:8b
>>> How far is the moon from Earth?

The desktop download also gives you a chat window that looks and works like ChatGPT, with all your downloaded models in a menu.

One lesson I learned on camera: close the apps you are not using. A local model shares your computer's memory with everything else. While recording the course, a reply got much faster once I closed about 20 open apps.

6. Connect a local model to real work

Chatting is the start. The useful part is plugging a local model into the tools you already use. Ollama can launch coding agents like Claude Code and OpenAI's Codex on top of a local model:

ollama launch codex

In the course I asked Codex, running a local 35B mixture of experts model with 3 billion active parameters, to create a folder and build a simple website for a made-up coffee shop. It made the folder, wrote the HTML file, and had a working page in 1 minute 20 seconds. I did not copy or paste anything. That was on my 128 GB Apple Silicon M3 Mac.

You can also call a local model from your own scripts and small apps through its local API, the same way you would call a cloud API.

7. When local is the wrong choice

Use a cloud model when you need:

The grown-up setup is hybrid. Easy tasks stay local. Hard tasks go to the cloud. One pattern that works right now: a big cloud model like Claude Opus writes the plan, a local model does the coding, and the big model checks the result. You pay for the reasoning and get the routine work free.

8. The full course, still short

Why I made it a short primer

Most people do not need a 15-hour course to start with local AI. They need to know what their machine can run, which app to install, which model to download, and one real job to give it. That fits in about an hour.

Once a model is running on your own computer, you learn the rest by using it. The course grows from there: when students ask for a topic, I add a lesson.

This page is the short version. The course walks through every step on screen, with the commands in each lecture's resources. It also covers local vision, speech to text, text to speech and image generation.

Local AI Crash Course on Udemy

Udemy course by Saurav Sharma · about an hour

Local AI Crash Course: Run Models Locally and Do Real Work

★★★★★ 4.7 (20 ratings) · 147 students
19 lectures · 1h 6m · updated November 2026

Get the course on Udemy → Discount code LOCAL_AI_YT applied in the link

Want the free version first? The 26-minute YouTube primer covers the same four decisions.

Stuck on a setup, or deciding whether your team should run models locally? Book a 55-minute call.

More on local AI from CloudYeti