Kog bets GPU limits are software, not silicon: 30x faster LLM inference on GPUs you own

Kog is a bet against the direction of the entire AI inference industry. While markets celebrated Cerebras' IPO for purpose-built inference chips, French startup Kog claimed the real headroom sits inside the GPUs enterprises already own. Its May 2026 Hacker News tech preview demonstrated 3,000 tokens per second per request on the AMD MI300X and Nvidia H200 — standard datacenter hardware — using a purpose-built 2-billion-parameter model it then open-sourced as Laneformer 2B.

The method comes from the founder's unusual double training. Gaël Delalleau studied solid-state physics at École Polytechnique and competed four times in DEFCON's Capture The Flag tournament. He describes Kog's approach as understanding 'the laws of physics, and the laws of the GPU' — reverse-engineering hardware down to assembly and binary code and using it for a purpose it wasn't designed for. That means weeks or months of manual research per chip, a constraint that shapes the whole company: with 11 people, Kog can only support a few architectures at a time.

The positioning matters in a crowded field. ZML, also French, bypasses CUDA with a hardware-agnostic software layer; Kog explicitly distinguishes itself, comparing itself to Stanford's Hazy Research with an even deeper focus on GPU acceleration. Its wedge is that decode speed — the sequential part of generation — is bottlenecked by software, not silicon, and that newer GPUs carry memory bandwidth that the vendor stack never unlocks. The demo validated that enough to generate more than 200 tangible business leads.

The market told Kog to change targets. Prospects were unwilling to fine-tune small specialized models, so the startup pivoted its engineering focus from its 2B demo model to accelerating the large models businesses already run. That is a far harder technical target, and Delalleau concedes the company must prove the approach transfers to LLMs before it can raise a Series A. European sovereignty tailwinds — Bpifrance, French Tech 2030, Scaleway — keep the company funded while it tries.