Qwen 3.8 Flash Next on a 128GB Mac: local test
I ran the 94 GB Qwen 3.8 Flash Next locally with llama.cpp on a 128GB MacBook Pro. It built a working speedometer app from one prompt and fixed its own bug, but took about 13 minutes.
The main idea
Some people call Qwen 3.8 Flash Next the best model for a 128GB Mac. One viewer with a 128GB M5 Max told me the best model is still the Qwen 3.6 35B model. So I tested the new one on my 128GB MacBook Pro. The model is about 94 GB. It has 125 billion parameters, but only six billion are active for each token it writes. That is why a model this big can still run at a usable speed.
I set it up with llama.cpp and chatted with it first. A short story ran at about 30 tokens per second. A curl measurement gave 39.6 tokens per second for the prompt and 11.3 tokens per second for the output. Then I used the Pi coding agent and asked for a single-file speedometer app that measures the model's own speed. Pi first gave a context error, so I restarted the server with a 65,536 token context window.
The model wrote the app and hit a macOS bug. It debugged the bug and fixed it. The final app is a chat UI with a live speedometer, a changeable endpoint, and max tokens. The whole run took about 13 minutes and my laptop fans were loud. The biggest speed up came from the setup. With my skills turned on, the agent sent about 20,000 tokens before the model wrote a word. With lean flags, it starts in seconds.
Key points
- The model is about 94 GB with 125 billion parameters, and six billion are active per token.
- On macOS, raise the GPU memory limit to your RAM minus 10 GB.
- A chat test ran at about 30 tokens per second.
- The curl test gave 39.6 tokens per second for the prompt and 11.3 tokens per second for the output.
- Pi errored at the default context, so I restarted the server with a 65,536 token context window.
- Starting Pi with the no-skills flag avoids about 20,000 tokens sent before the first word.