OneTriangle bets KV-cache transfer can slash long-context inference costs

An AI inference startup serving open-weight models (DeepSeek V4 Flash, Qwen3.6 27B) from managed GPU stacks, while developing cross-model KV-cache transfer so a small model prefills prompts and a larger model decodes from the transferred cache.

That a small model can prefill and hand its KV cache to a large one — large-model quality at small-model cost, and the transfer layer owns inference economics.

$125,000 micro-seed from Y Combinator (June 2026).

Hannah Chung (CEO) and Medha Venkatapathy (CTO), MIT CSAIL graduates, co-founded OneTriangle (initially TrustAI) in 2026 with three other MIT engineers whose backgrounds span Google DeepMind, Jane Street and MIT Lincoln Laboratory. Their premise: inference has surpassed training as AI's largest compute cost, and for long-context and agent workloads prefill dominates both cost and latency — a hidden tax every serving team pays.

Accepted into Y Combinator's Summer 2026 batch, OneTriangle raised a $125,000 micro-seed in June 2026. An August 18 research note reported a learned Minitron 4B-to-Llama 3.1 8B handoff: an 8,192-token prompt reached the larger model's first token in 38.25ms with the small model's cache resident versus 302.69ms native prefill (7.9x), at 82.52% top-token agreement over 32 steps. The company claims 20% lower prefill costs and 40% faster time-to-first-token versus present standards and says it is upstreaming the technique into vLLM. On August 24 it launched hosted DeepSeek V4 Flash at $0.15 per million input and $0.35 per million output tokens on an eight-H100 stack, with a delayed tier 40% lower.

Live but unproven: the hosted model service is publicly priced and running, while the cache-transfer technique remains research-stage; OneTriangle's own benchmarks show the headline 7.9x first-token speedup falls to 1.25x when the small model's read is included, and no independent verification or named customer has been published.

Bet on a structural cost shift: inference has overtaken training as AI's biggest compute cost, and prefill dominates long-context and agent workloads.

Technical wedge: cross-model KV cache transfer has never shipped in production, so a working transfer layer could own inference economics.

Speed-to-market: hosting DeepSeek V4 Flash gives customers a concrete product while cache-transfer research matures.

Founder fit: MIT CSAIL founders with DeepMind and Jane Street experience attacking a serving-infrastructure problem.

Ecosystem play: upstreaming into vLLM makes the technique reach every vLLM deployment instead of only OneTriangle's own stack.

When everyone resells the same open weights, serving performance is the only wedge — but a lab speedup that shrinks end-to-end and is only self-benchmarked is not yet a moat.

As of August 24, 2026 OneTriangle runs a paid, publicly priced DeepSeek V4 Flash service on eight H100s and lists Qwen3.6 27B in its catalogue, with usage-based pricing and no subscription. The KV-cache-transfer research remains pre-production: its strongest published result is the Minitron 4B-to-Llama handoff, which the company's own note reports collapses from 7.91x to 1.25x when the small model's read is included. No customer names, revenue, or independent benchmarks had been published, and the 'cheapest, fastest' claim had not been independently verified.