Local AI

Qwen 3.8 Flash Next on a 128GB Mac: local test

I ran the 94 GB Qwen 3.8 Flash Next locally with llama.cpp on a 128GB MacBook Pro. It built a working speedometer app from one prompt and fixed its own bug, but took about 13 minutes.

17 min · 2026-11-10 · Watch on YouTube

The main idea

Some people call Qwen 3.8 Flash Next the best model for a 128GB Mac. One viewer with a 128GB M5 Max told me the best model is still the Qwen 3.6 35B model. So I tested the new one on my 128GB MacBook Pro. The model is about 94 GB. It has 125 billion parameters, but only six billion are active for each token it writes. That is why a model this big can still run at a usable speed.

I set it up with llama.cpp and chatted with it first. A short story ran at about 30 tokens per second. A curl measurement gave 39.6 tokens per second for the prompt and 11.3 tokens per second for the output. Then I used the Pi coding agent and asked for a single-file speedometer app that measures the model's own speed. Pi first gave a context error, so I restarted the server with a 65,536 token context window.

The model wrote the app and hit a macOS bug. It debugged the bug and fixed it. The final app is a chat UI with a live speedometer, a changeable endpoint, and max tokens. The whole run took about 13 minutes and my laptop fans were loud. The biggest speed up came from the setup. With my skills turned on, the agent sent about 20,000 tokens before the model wrote a word. With lean flags, it starts in seconds.

Key points

Chapters

  1. 1:05llama.cpp setup step by step
  2. 3:15Chat test at about 30 tok/s
  3. 5:07Speed test with curl
  4. 6:37Pi coding agent and the context error
  5. 8:43Building the speedometer app
  6. 15:55Verdict
Free · by emailGet the next AI tip by emailOne short tip like this one when a new one is ready. Free. Unsubscribe any time.

More CloudYeti videos