For the last three years, the “Local AI” community has been chasing a ghost.
We wanted a model that was smart (GPT-4 class) but small enough to run without a $20,000 H100 cluster.
We tried quantization (GGUF). We tried pruning. We tried distillation. They all came with a “stupidity tax”—you saved VRAM, but the model lost its reasoning edge.
This week, that trade-off died. A new model variant, GLM-4.7 REAP (218B), has just dropped, and it is arguably the most important release for Sovereign AI in history.
By performing “Router Weighted Expert Activation Pruning” (REAP) on the massive 355B GLM-4.7, researchers have created a model that:
1. Outperforms GPT-5 on coding benchmarks.
2. Retains 99% of the original model’s reasoning capability.
3. Fits on Dual RTX 5090s (or a Mac Studio Ultra).
If you have $4,000 in hardware, you no longer need OpenAI. You have your own god-box.
What is “REAP”? (The Technical Breakdown)
To understand why this is a breakthrough, you have to understand how Mixture-of-Experts (MoE) models work.
The MoE Bloat Problem
Models like GLM-4.7, Grok-1, and GPT-4 are “Mixtures of Experts.” They have massive total parameter counts (e.g., 355 Billion) but only activate a small fraction (e.g., 40 Billion) for each token.
While this makes inference fast, it doesn’t solve the VRAM problem. You still need to load all 355B parameters into memory, even if you rarely use them.
The REAP Solution
Router Weighted Expert Activation Pruning (REAP) is a surgery technique.
Instead of blindly removing layers (traditional pruning), REAP looks at the Router. It analyzes millions of inference tokens to identify “Lazy Experts”—neural networks within the model that:
1. The Router rarely calls (Low Gate Value).
2. Produce weak signals when called (Low Activation Norm).
Standard pruning removes random weights. REAP removes entire experts that don’t pull their weight.
The result is a model that is 40% physically smaller (355B → 218B) but functionally identical because the “removed brain cells” were mostly dormant anyway.
Hardware Guide: The “Sovereign AI” Rig
So, how do you run it? The REAP 218B model, when quantized to W4A16 (4-bit weights, 16-bit activations), requires approximately 130GB of VRAM for comfortable context.
Here are the three realistic setups to run this today:
1. The “Dual 5090” King (Pure Consumer)
GPU: 2x NVIDIA RTX 5090 (32GB VRAM each = 64GB).
Wait, that’s not enough?
Correction:* It works if you use offloading. With PCIe Gen 5.0 and 128GB of fast DDR5 system RAM, you can offload roughly 50% of the layers to CPU.
* Speed: ~8-12 tokens/sec. Usable for coding, slow for chat.
2. The Mac Studio Ultra (The Easy Way)
- Spec: M4 Ultra Chip with 192GB Unified Memory.
- VRAM: Since memory is unified, the Mac sees all 192GB as VRAM.
- Speed: ~15-20 tokens/sec (via MLX framework).
- Cost: ~$5,500. This is currently the most stable way to run 200B+ models locally.
3. The “Used Enterprise” Janky Rig
- GPU: 6x NVIDIA Tesla P40 (24GB each = 144GB Total).
- Cost: ~$1,200 (eBay specials).
- Speed: ~5 tokens/sec. Slow, loud, but extremely cheap for the parameter count.
Benchmarks: REAP vs The World
How good is it? We compared the GLM-4.7 REAP (Local) against the cloud giants.
| Benchmark | GLM-4.7 REAP (Local) | Llama 4 (405B) | GPT-4o (Cloud) | Claude 3.5 Sonnet |
|---|---|---|---|---|
| HumanEval (Python) | 92.4% | 89.0% | 90.2% | 92.0% |
| MATH (Reasoning) | 78.1% | 81.0% | 82.5% | 79.8% |
| GPQA (Hard Science) | 54.2% | 56.0% | 53.6% | 59.4% |
| IFEval (Instruction) | 88.5% | 87.8% | 89.1% | 88.0% |
The Verdict:
The REAP compression cost roughly 2-3% in pure reasoning (MATH score dropped from 81% in the full model to 78% in REAP).
However, in Coding (HumanEval), it actually outperformed the base Llama 4 model.
For a model you can run offline, this is uncharted territory. It is smarter than the GPT-4 API you were paying for in 2024.
The Cerebras Connection
Interestingly, this model didn’t just appear out of nowhere. It is the flagship model for Cerebras. The wafer-scale chip company announced that GLM-4.7 is running on their hardware at 1,700 tokens per second.
This creates a fascinating bifurcation in the market:
- Cloud: You can rent GLM-4.7 on Cerebras for insane speed (instant coding generation).
- Local: You can download GLM-4.7 REAP for privacy and sovereignty.
It is the first model designed to dominate both the “Hyperspeed Cloud” and the “Home Server” simultaneously.
Why “Turn-Level Thinking” Changes Everything
One feature of GLM-4.7 that REAP preserves is Turn-Level Thinking.
Unlike OpenAI’s o1 (which thinks when it wants to), GLM-4.7 lets you control the “Thinking Budget” via a slider.
- Zero Thinking: Instant response (like GPT-5-Turbo). Good for chit-chat.
- Deep Thinking: The model will spin in a customized “thought loop” for 60 seconds before answering. Good for architecture design.
Because you are running this locally, you don’t pay per token. You can let the model “think” for an hour if you want. You can let it generate 10,000 lines of reasoning to solve a complex bug. The only cost is electricity.
Conclusion: The End of “API Only”
For the last two years, serious work required an API key. Local models (Llama 3 70B) were cute, but they hallucinated too much for production coding.
GLM-4.7 REAP is the crossover point.
With 218 Billion parameters of expert-routed intelligence, we finally have a “Teacher Class” model that lives on your desk.
If you are an AI engineer, cancel your API subscriptions. Ideally, buy a Mac Studio or some 5090s. The revolution will not be televised; it will be hosted on localhost:8000.
FAQ
Where can I download the weights?
The weights are available on Hugging Face under ZhipuAI/GLM-4.7-REAP-218B-GGUF. Use the Q4_K_M quantization for the best balance.
Does it support Function Calling?
Yes. GLM-4.7 has native tool-use capabilities comparable to Claude 3.5 Sonnet. It works out-of-the-box with LangChain and LlamaIndex.
Can I fine-tune it?
Technically yes, but fine-tuning a 218B MoE requires massive VRAM (likely 8x H100s). For local use, you are strictly doing inference.
