Open vs closed
What a task costs
Each mark is one model on ARC-AGI-2: higher scores are better, marks further left cost less per task. Blue squares are open-weight models you can download and run; orange rings are closed models you can only rent.
| Model | Vendor | Weights | ARC-AGI-2 score | Cost per task |
|---|---|---|---|---|
| GPT-6 Astra | OpenAI | Closed | 95% | $1.12 |
| GPT-6.1 Sol | OpenAI | Closed | 94.2% | $0.254 |
| Claude Opus 5.5 | Anthropic | Closed | 93.3% | $0.408 |
| Gemini 4 Argon | Closed | 91.2% | $0.325 | |
| Gemini 3.8 Flash | Closed | 89.2% | $0.4 | |
| GPT-5.5 | OpenAI | Closed | 85% | $1.87 |
| Claude Sonnet 5.5 | Anthropic | Closed | 83.9% | $0.546 |
| DeepSeek V4.1 Flash 1017 | DeepSeek | Open | 78.1% | $0.163 |
| Grok 4.6 | xAI | Closed | 67.1% | $0.757 |
| GLM-5.3-Flash | Z.ai (Zhipu) | Open | 65.8% | $0.093 |
| Grok 4.7 | xAI | Closed | 61.4% | $2.01 |
| DeepSeek V4 Flash 0731 | DeepSeek | Open | 61.4% | $0.042 |
| Claude Haiku 5.5 | Anthropic | Closed | 61.2% | $0.085 |
| DeepSeek V4 Pro 0813 | DeepSeek | Open | 61.2% | $0.598 |
| Kimi K3 | Moonshot AI | Open | 60.4% | $1.59 |
| GPT-6 Luna | OpenAI | Closed | 59.3% | $0.062 |
| Muse Spark Open | Meta | Open | 57.5% | $0.728 |
| Qwen3.8-27B | Alibaba (Qwen) | Open | 42.4% | $0.447 |
| Inkling Small | Thinking Machines Lab | Open | 40.1% | $0.233 |
Source: ARC Prize verified leaderboard, results as of 28 Oct 2026. Cost per task is what the benchmark run paid at API list prices; running an open model on your own hardware costs electricity and the machine instead.
Local performance
Rigs that run it
Measured results from people running open models on their own hardware. Decode is how fast the answer streams; prefill is how fast the model reads your prompt. The top three in each tier lead; scroll a row to see up to ten.
Local mini
One consumer device: a single GPU with up to 32 GB, an Apple Silicon Mac with up to 64 GB, or a small PC. Models up to about 35B parameters.
| # | Model | Hardware | Engine | Decode tok/s | Prefill tok/s |
|---|---|---|---|---|---|
| 1 | Qwen3.6-35B-A3B UD-Q4_K_M (20.61 GiB) | 1x NVIDIA RTX 5090 32 GB | llama.cpp | 271 | 9,240 |
| 2 | gpt-oss-20b MXFP4 (11.27 GiB) | 1x NVIDIA RTX 4090 24 GB | llama.cpp | 221.9 | 8,022 |
| 3 | Qwen3-30B-A3B Q4_K_M | 1x AMD Radeon AI PRO R9700 32 GB (RDNA4) | llama.cpp | 183.5 | 3,033 |
| 4 | Gemma 4 12B Q6_K (9.1 GiB) | 1x NVIDIA RTX 5090 32 GB | llama.cpp | 122.3 | 7,160 |
| 5 | Qwen3-30B-A3B-Instruct-2507 4-bit MLX | Apple M4 Max, 64 GB unified memory | MLX | 113.3 | 1,754 |
| 6 | Qwen3.8-27B NVFP4 (Digisensus build) | 1x NVIDIA RTX 5090 32 GB (power limit 480 W) | vLLM (patched) | 113 | 4,475 |
| 7 | Qwen3.5-35B-A3B Q4_0 (ubergarm, 19.78 GiB) | 1x NVIDIA RTX 3090 Ti FE 24 GB (450 W, tuned with LACT) | ik_llama.cpp | 107.3 | 2,936 |
| 8 | Qwen3.5-35B-A3B Q4_0 (ubergarm, 19.78 GiB) | 1x AMD Radeon RX 7900 XTX 24 GB | llama.cpp | 104.8 | 2,105 |
| 9 | Qwen3.6-27B Q4_K_M (15.66 GiB) | 1x NVIDIA RTX 5090 32 GB | llama.cpp | 77.1 | 3,826 |
| 10 | Qwen3-Coder-30B-A3B-Instruct 4-bit MLX | Apple M4 Pro (20-core GPU), 48 GB unified memory | oMLX | 72.6 | 750 |
Local medium
A workstation: two to four GPUs, one 48–96 GB card, a DGX Spark, a 128 GB Strix Halo box or a 96–256 GB Mac. Models of about 70B to 150B parameters.
| # | Model | Hardware | Engine | Decode tok/s | Prefill tok/s |
|---|---|---|---|---|---|
| 1 | gpt-oss-120b MXFP4 (GGUF, 59.02 GiB) | 1x NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | llama.cpp (CUDA) | 196.3 | 5,518 |
| 2 | Qwen3.8-Flash-Next NVFP4 (Local Inference Lab build) | 1× NVIDIA RTX PRO 6000 Blackwell Max-Q (325 W limit) | vLLM (Local Inference Lab image) | 173.2 | 12,073 |
| 3 | gpt-oss-120b MXFP4 (GGUF, 59.02 GiB) | Apple M2 Ultra, 192GB, 76 GPU cores (Mac Studio) | llama.cpp (Metal) | 79.7 | 1,245 |
| 4 | gpt-oss-120b MXFP4 (GGUF, 59.02 GiB) | NVIDIA DGX Spark (GB10 Grace Blackwell, 128GB unified LPDDR5X) | llama.cpp (CUDA) | 58.7 | 2,444 |
| 5 | gpt-oss-120b MXFP4 (GGUF, 59.02 GiB) | AMD Ryzen AI Max+ 395 'Strix Halo', 128GB unified RAM (Framework Desktop), Fedora 43 | llama.cpp (Vulkan) | 56.6 | 720 |
| 6 | Qwen3.5-122B-A10B 4-bit (MLX) | Apple M3 Ultra, 60-core GPU, 96GB unified memory (Mac Studio) | oMLX | 55.6 | 651 |
| 7 | Qwen3.5-122B-A10B INT4 AutoRound + FP8 shared experts | NVIDIA DGX Spark (GB10, SM121, 128GB LPDDR5X unified, ~273 GB/s) | vLLM (sm121 build) | 36 | — |
| 8 | MiniMax-M2.7 UD-Q3_K_S (GGUF, 87.2 GiB) | AMD Ryzen AI Max+ 395 'Strix Halo', 128GB unified RAM (Framework Desktop), Fedora 43 | llama.cpp (Vulkan) | 31.1 | 242 |
| 9 | Qwen3.5-122B-A10B UD-Q5_K_XL (GGUF, 85.6 GiB) | AMD Ryzen AI Max+ 395 'Strix Halo', 128GB unified RAM (Framework Desktop), Fedora 43 | llama.cpp (Vulkan) | 22.3 | 309 |
| 10 | Qwen3.5-122B-A10B NVFP4 (W4A4) | NVIDIA DGX Spark (GB10, SM121), single node | vLLM nightly | 15.1 | 1,602 |
Local extreme
Frontier-size open models on hardware you own: 512 GB Macs and Mac clusters, 4–8 GPU servers, CPU and GPU hybrids. Models of 200B parameters and up.
| # | Model | Hardware | Engine | Decode tok/s | Prefill tok/s |
|---|---|---|---|---|---|
| 1 | DeepSeek-V4-Flash-0731 FP8 + MXFP4 experts | 8× RTX PRO 6000 Blackwell Server Edition (768 GB GDDR7) | vLLM + DSpark speculative decoding | 204 | — |
| 2 | MiniMax M2.7 UD-Q5_K_XL (157.8 GiB) | 8× RTX 3090 (24 GB each) | ik_llama.cpp | 61.1 | 788 |
| 3 | Qwen3.5-397B-A17B 4-bit MLX | 1× Mac Studio M3 Ultra (80-core GPU) 512 GB | oMLX | 35.3 | 481 |
| 4 | GLM-5.1 NVFP4 (lukealonso build, 434 GB) | 6× RTX PRO 6000 Blackwell Workstation Edition (96 GB each) | vLLM (TP 2 × PP 3) | 33.3 | — |
| 5 | DeepSeek-V4-Flash-0731 MXFP4 routed experts (CPU+GPU split) | 8× RTX 5090 + x86 CPU host (CPU model not stated) | KTransformers + SGLang | 32.7 | — |
| 6 | Kimi K2 Thinking INT4 (native weights) | 4× Mac Studio M3 Ultra (2× 512 GB + 2× 256 GB, 1.5 TB total) over TB5 RDMA | exo (MLX, RDMA over Thunderbolt 5) | 25 | — |
| 7 | Kimi K2.6 Q4_X (GGUF, ubergarm) | 1× RTX PRO 6000 (96 GB) + AMD EPYC 9355, DDR5-6400 (experts in system RAM) | ik_llama.cpp | 20.2 | 729 |
| 8 | GLM-5.2 UD-Q4_K_XL (unsloth GGUF) | 2× RTX PRO 6000 + AMD EPYC 9B45, 12-channel DDR5-6400 | llama.cpp (fork) | 20 | — |
| 9 | DeepSeek-V3.2 4-bit MLX | 1× Mac Studio M3 Ultra (80-core GPU) 512 GB | oMLX | 19.3 | 210 |
| 10 | GLM-5.2 4-bit MLX (oQ4e), MTP on | 1× Mac Studio M3 Ultra 512 GB | oMLX | 19 | 194 |
Single-user numbers unless a tile says otherwise. Figures come from the linked source and use its prompt length, quantization and engine version, so compare tiles as reports rather than a controlled benchmark. We link to Hugging Face where a result is published there, then GitHub, then the original post.
Ranked daily
News
Releases, engines, hardware and benchmarks from the people and projects building local inference. Ranked by impact and freshness, not by volume.
Model
Moonshot releases Kimi K3.1 Pro weights: a 2.8T-parameter MoE with 104B active huggingface.co
Open-weight and natively multimodal, with a 1M-token context, trained quantization-aware in MXFP4 weights and MXFP8 activations; post-trained from Kimi K3 and released under the Kimi K3.1 Pro License.
Engine
SGLang 0.5.23 adds Paged Experts to run MoE models larger than GPU memory on one consumer GPU github.com
The opt-in Paged Experts mode is up to 30× faster than CPU offload; the release also adds Muse Spark Open, MiniMax-M3.1-Flash and faster first tokens for MoE models.
Engine
oMLX 0.7.4 lets a second Mac speed up prefill for big models up to 2.9× github.com
With an M5 Max 128GB as prefill accelerator, an M3 Ultra 512GB prefills a 64K prompt of DeepSeek V4.1 Flash at 3,123 tok/s instead of 1,060; decode speed is unchanged and Thunderbolt or 5GbE is enough.
Model
Tencent announces Hy4 with vision and computer use; open weights to follow hy.tencent.ai
Tencent says Hy4 places its Hy models among the strongest open models for the first time; the weights are not out yet, and Tencent says it is working with vLLM on support for the release.
Research
Anthropic releases FlashProve, open-source proofs that a reply came from a committed model github.com
A zkSNARK prover that proves Llama 4 Maverick (400B MoE, 4-bit) inference at about 110 tokens/s on one H100, roughly 70–95× vLLM's GPU time, with ~19 MB proofs per 1,024 tokens; prover, C++ and Lean verifiers are Apache 2.0.
Model
Mistral Large 4 weights released: 1T-parameter MoE with GGUF and NVFP4 builds huggingface.co
Mistral opened weights for Large 4 (about 1T total, 49-52B active, 1M context, modified-MIT license with a model-as-a-service carve-out); the Q2_K GGUF runs on a 512 GB M3 Ultra at about 15 tok/s.
Engine
vLLM 0.33.0 adds Recursive Flint, Muse Spark Open and Beam-1.0, and drops CUDA 12.9 github.com
927 commits from 389 contributors; release wheels and images are now CUDA 13.0 only and built on PyTorch 2.14.
Model
MiniMax publishes M3.1 Flash weights: 428B MoE with 23B active huggingface.co
MiniMax released quantization-aware-trained weights for its multimodal M3.1 Flash four days after the hosted launch, under a community license with branding and commercial-authorization conditions.
Research
Zro open-sources Sable, bit-exact proofs that a committed model produced a reply github.com
An integer version of the model plus a sumcheck/LogUp-GKR proof system; on one RTX PRO 6000 it proves a 4,096-token Qwen2.5-32B request in 92.5 s with a 135 MiB proof. Code and paper are MIT-licensed.
Research
Qwen 3.8 27B on a laptop can also interpret model weights simonwillison.net
Simon Willison reproduced one of the weight-space programming examples locally: Qwen 3.8 27B on his laptop read a small model's raw weights and described what it was trained to do.
Industry
Revolut moves about 2,000 engineers to self-hosted GLM for coding youtube.com
In a talk at AI Engineer New York, Revolut's head of AI operations David Abitbol said 56 NVIDIA B200s serve coding agents at a blended $0.30 per million tokens, about a tenth of its previous API bill, with engineering lead time down 70%.
Model
Recursive releases Flint, a 40B-A2.5B coding model under Apache 2.0 recursive.com
A small-active-parameter MoE coding model that the vendor says runs at over 125 tok/s on an M4 Pro MacBook Pro.
Engine
Ollama 0.40.14 respects models' own sampling defaults on its OpenAI-compatible API github.com
Clients that send no temperature or top_p now get the model's packaged values instead of 1.0; also faster top-k sampling on MLX.
Engine
Unsloth adds faster generation, multi-GPU training options and CPU training github.com
The release also adds desktop themes and publishes DeepSeek-V4.1 512K-context benchmarks of its GGUF builds.
Model
DeepSeek ships V4.1-Flash-1017 update under MIT license huggingface.co
A minor update to the 763B-parameter V4.1-Flash that DeepSeek says improves reasoning, coding and agentic scores (HLE no-tools 36.8 to 42.6) while producing shorter outputs.
Research
FireFlash: an open RL recipe lifts GLM-5.3-Flash to 46.2% on Terminal-Bench 4.0 agenteconomics.org
Five days of full-parameter RL on 64 B300 GPUs; the authors report Fable 5-level Terminal-Bench scores at over 30× lower cost per task than Fable 5 at maximum effort, serving the model themselves at GLM-5.3-Flash prices.
Research
GPT-6 Astra edits Qwen3.5-0.8B's weights directly, lifting MATH-500 from 7.9% to 48.1% rikiyat.github.io
MIT CSAIL researchers report that editing 0.016% of the small open model's weights, without gradients or training, fixed its reasoning loops and tripled its IFEval score; they call it weight-space programming.
Research
Zyphra Hierarchical Attention: a tree-structured index makes attention subquadratic arxiv.org
Keys are summarised into a tree that queries search top-down; Zyphra projects 35x over dense attention at 10M tokens and says existing models can be adapted with about a billion tokens of distillation.
Model
Meta releases Muse Spark Open, an 816B-parameter open-weight MoE model huggingface.co
Meta published weights for an 816B-total, 42B-active MoE model with 1M context under its Muse Model License, with a stated target of two M5 Ultra Mac Studios.
Industry
GLM-5.3 goes into production on T-Systems' sovereign AI cloud in Germany docs.llmhub.t-systems.net
T-Systems' AI Foundation Services now serves Z.ai's GLM-5.3 with a 1M context window, hosted and processed in Germany, on all paid plans.
Hardware
RTX Spark laptops with up to 128GB unified memory go on sale unite.ai
NVIDIA's Grace CPU plus Blackwell RTX laptop chip shipped in machines from ASUS, Dell, HP, Lenovo, Microsoft and MSI; NVIDIA claims up to 1 petaflop of FP4.
Engine
NVIDIA: three ways to speed up MoE decoding on DGX Spark developer.nvidia.com
Using Ling-3.0-flash (124B, 5.1B active) at batch 1: Humming MXFP4 MoE kernels, online FP8 for the output layer, and MTP speculative decoding, including a combination that ends up slower than the baseline.
Research
Weight-space programming result reproduced independently, with a size caveat x.com
Alex Krentsel reports reproducing the MIT results and notes the edited models are tiny (about 20,000 parameters in his test), far from frontier scale.
Benchmark
NVIDIA runs full GLM 5.3 on an 8-node DGX Spark cluster at 31 tok/s developer.nvidia.com
With 1 TB of pooled memory and an NVFP4 checkpoint, eight Sparks decode GLM 5.3 at 31 tok/s single-stream with MTP speculative decoding, using a new RoCE all-to-all backend merged into FlashInfer.
Engine
vLLM 0.32.0 adds NVFP4 KV cache on SM8x and SM12x GPUs github.com
The release brings 4-bit KV cache to Ampere and consumer Blackwell (and to ROCm), adds kernel-download and preload commands for faster startup, and supports EmbeddingGemma 2 and MiniCPM-V 4.7.
Run it yourself. Check it yourself.
Results and corrections
Send a resultBenchmarks to add, numbers to correct, sources we should follow.
Private inference research
Discuss researchThreat models, benchmarks and private-inference prototypes.