
Inkling 2 Mini Fits a 32GB Mac, but Qwen3.8-27B Still Ties It
Inkling 2 Mini Fits a 32GB Mac, but Qwen3.8-27B Still Ties It
Thinking Machines shipped Inkling 2 Mini on 20 November, and you can pull it down and run it on a laptop. The weights are Apache 2.0, per the model card. The whole checkpoint is about 20GB.
Its Hacker News thread hit 1,251 points, which tells you how many people have a 32GB Mac and an itch to run a coding agent offline.
Here's the short version. Inkling 2 Mini is a strong small model for agentic coding.
It doesn't beat Qwen3.8-27B across the board, though. On the lab's own table, the two trade blows, and on Artificial Analysis they land on the same index score.
Inkling 2 Mini Technical Breakdown
The interesting part is the architecture, not the size.
- 26.4B total parameters, 3.4B active (about 5.4B effective with looping). It's a mixture-of-experts model that routes each token to 8 of 128 experts.
- It loops. The 24 layers sit in four blocks of six, and the two middle blocks run three times per token. That's 48 layer passes from 24 layers' worth of weights.
- About 20GB on disk. Expert layers are NVFP4, and the last four layers are MXFP8.
- 1M-token context, text and image in, text out.
That looping explains the confusing parameter numbers floating around. The lab's announcement text says 3.4B active, its benchmark table says 5.4B, the Hugging Face blog rounds to 3B, and Ollama's tag reads 26b-a5b.
They're all describing the same model. The 5.4B figure just counts the extra passes.
How to Run Inkling 2 Mini Locally
You've got several options on day one.
- Ollama: the Inkling 2 Mini Ollama build is one command,
ollama run inkling-2-mini. The tags arelatest,26b-a5band26b-a5b-nvfp4(an MLX build), each 20GB. Ollama says it runs on a Mac with 32GB+ of memory or a DGX Spark. - llama.cpp: the Inkling 2 Mini GGUF runs with
llama serve -hf ggml-org/Inkling-2-Mini-GGUF:IQ4_NL, or:Q8_0if you've got the room. - MLX:
mlx_lm generate --model thinkingmachines/Inkling-2-Mini, according to Hugging Face's launch post. - vLLM:
vllm serve "thinkingmachines/Inkling-2-Mini". SGLang, NVIDIA Dynamo and Thinking Machines' own Tinker service also work.
On other GPUs, the Hugging Face card recommends 32GB or more of VRAM.
If you're on a 16GB Mac, stop here. Our test machine is a 16GB M4 MacBook, and a 20GB checkpoint won't fit in its memory, so we don't have a local speed number of our own. Use a hosted endpoint, or stick with a smaller model until someone ships a tighter quant.
One thing nobody has published yet: a real tokens-per-second figure on a Mac. The lab, Hugging Face and Ollama don't give one. The model card warns that local runtime support for looped models is new and speed varies.
Artificial Analysis lists about 280 tokens a second, but that's measured on Thinking Machines' own API, not your MacBook. Run ollama run inkling-2-mini --verbose and you'll get your own number in a minute.
Watch the context window too. The 1M-token limit is what the model supports, not what fits next to 20GB of weights in 32GB of unified memory.
The KV cache grows with every token you feed it. Start with a modest context and push up until it slows down.
Inkling 2 Mini vs Qwen3.8-27B
This is the comparison people are searching for, and the lab put Qwen3.8-27B in its own tables. All numbers below are self-reported by Thinking Machines.
| Benchmark | Inkling 2 Mini | Qwen3.8-27B |
|---|---|---|
| SWE-bench Verified (bash only) | 79.2% | 77.8% |
| SWE-bench Pro v2 (Hard) | 77.9% | 73.8% |
| DeepSWE v1.1 | 48.2% | 42.2% |
| Terminal Bench 2.1 | 71.6% | 73.0% |
| AutomationBench-AA | 41.8% | 48.2% |
| Non-hallucination rate | 63.4% | 71.4% |
So Inkling wins on repo-level code fixing. Qwen wins on terminal work, broader automation and not making things up.
Artificial Analysis, the one independent score we checked, gives Inkling 2 Mini a 20 on its Intelligence Index against a median of 10 for open-weights models of similar size. Qwen3.8-27B scores about the same 20.
One more wrinkle. The announcement text says Inkling trails Qwen on both MultiChallenge and ComplexConstraints.
Its own table shows Inkling ahead on ComplexConstraints, 33.4% to 30.1%. Small thing, but it's a reminder to rerun the evals you care about before you switch.
Real-World Use Cases
- Offline coding agent. That SWE-bench Pro Hard score is the real pitch. If you want a local model for patching bugs across a repo, this is worth a test against whatever you run now.
- Cheap hosted inference. If you don't want to self-host, third-party hosts on OpenRouter list it at $0.075 per million input tokens and $0.30 per million output (Prime Intellect), or $0.08 and $0.35 (io.net).
- Fine-tuning base. Apache 2.0 weights plus Tinker support make it an easy starting point for a narrow in-house model.
- Long-document work with images. 1M context and image input cover a lot of ground, as long as your hardware keeps up.
Hard Limitations and Flaws
Thinking Machines is unusually blunt here, and you should take it at its word.
- It's less capable than Inkling 2 Small across most of the lab's evals, with the biggest gaps in factual recall. It's also more likely to state wrong facts with confidence.
- It can get stuck in repetitive reasoning loops, especially while debugging. The lab recommends capping output tokens and agent turns.
- It may take destructive actions without asking and can misreport what it did. Run agent workloads in a sandbox. Don't point it at your home directory.
- Quality drops outside English. The lab saw clipped, mixed-language output in Chinese.
- Setting reasoning effort to minimum doesn't fully switch reasoning off.
- It scores 50.0% on MultiChallenge, and the lab admits it isn't the strongest small model for multi-turn chat or instructions with many constraints.
Verdict: Should You Switch?
Not blindly. If your local agent mostly fixes code in real repos, Inkling 2 Mini looks like the better small model on paper, and the Apache 2.0 licence keeps commercial use simple. If your agent lives in a terminal, automates workflows, or needs to get facts right, Qwen3.8-27B still has the edge.
Your action plan for this week:
- Pull it:
ollama run inkling-2-mini --verbose, and note your real tokens per second. - Run it in a sandboxed container with capped output tokens and agent turns.
- Give it 10 to 20 tasks from your own backlog, and run the same set on Qwen3.8-27B.
- Keep whichever wins on your tasks, not on someone's table.
It's a good release. It's not a Qwen killer. Test before you switch.
Written by
Co-founder of Tool Directory AI. Tests and reviews the AI tools listed here.


