The archive · AI & Models · Product decision · 2025–2026
Cactus (YC S25) bets phones run open models locally with cloud handoff
YC S25 open-source engine runs Gemma-class text, speech and vision models on phones — Launch HN 2025-09-18, 123 points; repo 6k stars by 2026.
Cactus Compute
What the business is
Cactus Compute builds Cactus, an open-source hybrid edge-cloud inference engine that runs text, speech and vision models on smartphones, wearables and robots, with its own rotation-based quantization (CQ), per-device CPU/GPU kernels and confidence-based cloud fallback.
How it started
Cactus Compute, a YC Summer 2025 startup whose repo credits Henry Ndubuaku and whose Hacker News launch was introduced by co-founders Henry and Roman, launched its engine on 2025-09-18. The pitch was direct: AI inference belongs on phones, wearables and robots, so the team built a hybrid engine that runs open models on-device and hands hard queries to the cloud.
What happened
The technical bet has three layers: Cactus Quants (rotation-and-codebook quantization from 4-bit to 1-bit), Cactus Kernels (CPU/GPU kernels for Apple, Samsung, Pixel and other silicon) and Cactus Graph (a zero-copy computation graph). The repo published benchmark tables that made the launch concrete: Gemma-4-E2B at CQ4 runs 729 tokens/s prefill and 37 tokens/s decode on an iPhone 17 Pro within roughly 644MB peak RAM, and CQ4 keeps near-F16 accuracy on MMLU (59.45 vs 62.33). A confidence-based handoff routes queries to the cloud only when the local model is unsure. By the September 2026 fetch the repo had grown to 962 commits, 6.0k stars and 501 forks, and the company had added Needle, a 26M-parameter on-device tool-calling model.
How it ended up
Still live and open as of 2026-09-05: the repo is actively developed with iOS/Android/Python builds and models pre-uploaded to its Hugging Face org; funding and team size are not disclosed in the material.
Background
Cactus Compute, a YC Summer 2025 startup, built Cactus as an open-source hybrid edge-cloud inference engine for phones, wearables and robots. Its bet: aggressive quantization and per-chip kernels can run open models on hardware people already own, making local inference the default and the cloud a fallback for queries the on-device model is not confident about.
The company launched on Hacker News on 2025-09-18 and the post drew 123 points and 63 comments. What carried it was concrete evidence in the repo: rotation-based CQ4 quantization holds near-F16 accuracy (MMLU 59.45 vs 62.33), and Gemma-4-E2B at CQ4 runs 729 tokens/s prefill on an iPhone 17 Pro inside roughly 644MB of RAM.
By the 2026-09-04 fetch the repository showed 962 commits, 6.0k stars and 501 forks, with iOS, Android and Python artifacts, models pre-uploaded to the Cactus-Compute Hugging Face org, and a second product line, Needle, a 26M-parameter on-device tool-calling model. The company is still live and the project still open as of 2026-09-05.
What has to be true
- The engine is framed as hybrid edge-cloud with handoff on confidence — the whole design assumes most useful inference should happen on the device, not in a data center.
- CQ4 answers the standard objection to quantization with tables: near-F16 accuracy on MMLU (59.45 vs 62.33) makes the quality loss small enough to bet the product on.
- Real-device benchmarks (729 tokens/s prefill on an iPhone 17 Pro at ~644MB peak RAM) gave the HN launch a falsifiable claim instead of a marketing page.
- Open sourcing the runtime created distribution: a developer can brew install Cactus and run a model in minutes, which is how a deep-tech repo reached 6.0k stars and seeded later bets like Needle.
What can be applied
Open-source the runtime and publish per-device numbers: quantization plus kernels made the demo credible, and the repo became distribution for later bets like on-device tool calling.
Aftermath
As of 2026-09-05 the project is live and growing: the repo shows 6.0k stars, 501 forks and 962 commits, benchmarks now span 2025-era flagships (iPhone 17 Pro, Mac M5 Max) plus Android and wearables, and model support has expanded across Gemma, Liquid, Qwen, whisper and parakeet families, with pre-built iOS, Android and Python artifacts and models pre-uploaded to the Cactus-Compute Hugging Face org. The team has added Needle, a 26M-parameter on-device tool-calling model, as a second open line. No funding figures or team size are disclosed in the material.
Sources
- Launch HN: Cactus (YC S25) – AI inference on smartphones
- cactus-compute/cactus — Cactus: hybrid edge-cloud AI engine
spotted an error? The archive wants to know.
Your turn
You just read one. Describe what you are building, and see who is betting on the same thing.
Free account · 3 free questions · no card