The archive · AI & Models · Technical decision · 2026
Colibrì streams a 744B GLM-5.2 onto a 25GB-RAM laptop: 937 HN points
A solo dev runs a 744B-parameter GLM-5.2 on 25GB of RAM by streaming MoE experts from an SSD — glacial but real, 937 HN points
Colibrì (JustVugg/colibri)
What the business is
An open-source C inference engine (a single ~1,300-line file) that runs huge MoE LLMs on consumer PCs by streaming experts from disk.
How it started
In early July 2026, Italian developer Vincenzo tried GLM-5.2, Zhipu AI's open 744B Mixture-of-Experts model, and found the output on par with Claude or GPT. His 12-core laptop had 32GB of RAM (25GB usable); running the model conventionally meant an out-of-memory crash. He converted it to int4, studied its multi-token-prediction head, then wrote a C engine, ~1,300 lines, that keeps only the dense layers (attention, shared experts, embeddings, ~17B params, ~9.9GB at int4) resident and pages the 21,504 routed experts (~370GB, ~19MB each) off a virtual NVMe drive on demand.
What happened
The Show HN post on 2026-07-09 drew 937 points in days; commenters debated SSD wear, compared the design to antirez's streaming work, and called it 'the hacker spirit.' Tom's Hardware covered it on July 11: the quantized 1.5TB model ran at roughly 0.05-0.1 tokens per second on the author's CPU — 'unusable for practical conversation' but the first demonstration that frontier-class open models need not be cloud-only. The README later added a mirror feature: a second SSD holding a subset of shards, byte-validated at startup, to spread disk contention.
How it ended up
Still a solo, unfunded proof of concept as of 2026-09-03. No company, no funding round — runnable software whose artifact is the argument that frontier AI can work on hardware you already own.
Background
In July 2026, Vincenzo — an Italian developer posting as JustVugg — was impressed by GLM-5.2, Zhipu's open 744B-parameter MoE model, finding its answers 'similar to models like Claude or GPT.' Then he asked the question nobody expected to answer yes: could it run on his own machine, a 12-core laptop with 25GB usable RAM? Standard inference would OOM instantly, so he stopped trying to shrink the problem into memory and instead changed what had to be in memory at all.
His engine, Colibrì, exploits MoE structure: each token activates only ~40B of the 744B parameters, and of those only the routed experts (~11GB of weights) change token to token. So the dense part stays resident (~9.9GB at int4) while the 21,504 experts — ~370GB on disk — are streamed in on demand with a per-layer LRU cache, a pinned hot-store, and the OS page cache as free L2. The whole engine is one ~1,300-line C file: no BLAS, no Python at runtime, no GPU.
The Show HN post took 937 points and 240 comments; Tom's Hardware followed two days later, framing it as 'frontier-level AI on 25GB of RAM,' at 0.05-0.1 tokens per second — impractical for conversation, decisive for the argument that frontier open models are not hardware-exclusive. The author keeps iterating on SSD streaming performance, adding mirrors and cache tuning, with no company and no raise behind it.
What has to be true
- He inverted the constraint: rather than shrinking the model to fit RAM, he moved the rarely-used experts to disk — only ~11GB of weights change per token, the rest stays reusable in memory.
- Radical transparency helped: a single C file with no dependencies, no GPU, no runtime — anyone with a compiler can inspect, run, and contest the claim.
- Timing reset a ceiling: when 'local model' meant 8B-70B, a proof that 744B could run at all moved the local-AI argument to a new plane, which is why a hardware outlet covered a hobby project.
- Honesty about limits — cold starts, 0.1 tok/s, SSD-wear questions — built credibility instead of hype, and the community responded to the person, not a press kit.
What can be applied
A slow-but-real proof can out-earn a fast one: showing a 744B model runs on 25GB of RAM at 0.1 tok/s changed what people believed possible, drawing more attention than a polished cloud demo would.
Aftermath
As of 2026-09-03 the project remains a solo, unfunded proof of concept. The author is iterating on disk-streaming performance — mirrors that spread reads across SSDs, per-layer LRU caches, an optional pinned hot-store — and GLM-5.2 stays the flagship load. Tom's Hardware's closing question still frames the bet: at some point it may be feasible to run a really clever model on high-end consumer hardware at a decent clip. No company, no funding round; the artifact is the argument.
Sources
- Show HN: Getting GLM 5.2 running on my slow computer
- Colibrì proof-of-concept gains frontier-level 1.5-TB AI model — novel approach runs on only 25GB of RAM
spotted an error? The archive wants to know.
Your turn
You just read one. Describe what you are building, and see who is betting on the same thing.
Free account · 3 free questions · no card