The archive · AI & Models · Technical decision · 2026
OneTriangle bets KV-cache transfer can slash long-context inference costs
YC S26's OneTriangle prefills on small models, hands the KV cache to big models to decode, claiming 20% lower prefill costs and 40% faster first token.
OneTriangle
What the business is
An AI inference startup serving open-weight models (DeepSeek V4 Flash, Qwen3.6 27B) from managed GPU stacks, while developing cross-model KV-cache transfer so a small model prefills prompts and a larger model decodes from the transferred cache.
Starting capital：$125,000 micro-seed from Y Combinator (June 2026).
How it started
Hannah Chung (CEO) and Medha Venkatapathy (CTO), MIT CSAIL graduates, co-founded OneTriangle (initially TrustAI) in 2026 with three other MIT engineers whose backgrounds span Google DeepMind, Jane Street and MIT Lincoln Laboratory. Their premise: inference has surpassed training as AI's largest compute cost, and for long-context and agent workloads prefill dominates both cost and latency — a hidden tax every serving team pays.
What happened
Accepted into Y Combinator's Summer 2026 batch, OneTriangle raised a $125,000 micro-seed in June 2026. An August 18 research note reported a learned Minitron 4B-to-Llama 3.1 8B handoff: an 8,192-token prompt reached the larger model's first token in 38.25ms with the small model's cache resident versus 302.69ms native prefill (7.9x), at 82.52% top-token agreement over 32 steps. The company claims 20% lower prefill costs and 40% faster time-to-first-token versus present standards and says it is upstreaming the technique into vLLM. On August 24 it launched hosted DeepSeek V4 Flash at $0.15 per million input and $0.35 per million output tokens on an eight-H100 stack, with a delayed tier 40% lower.
How it ended up
Live but unproven: the hosted model service is publicly priced and running, while the cache-transfer technique remains research-stage; OneTriangle's own benchmarks show the headline 7.9x first-token speedup falls to 1.25x when the small model's read is included, and no independent verification or named customer has been published.
Background
OneTriangle (formerly TrustAI) is a San Francisco AI inference startup from Y Combinator's Summer 2026 batch, founded by MIT CSAIL graduates Hannah Chung and Medha Venkatapathy with three fellow MIT engineers. Its bet is that inference — not training — has become AI's dominant compute cost, and that the way to cut it is a technique nobody has shipped in production: transferring the key-value (KV) cache between models.
The idea: a small model reads the long prompt (the expensive prefill step), its KV cache is stripped of positional encoding and mapped into a large model's space, and the large model only decodes the answer. OneTriangle claims 20% lower prefill costs and 40% faster time-to-first-token versus present standards, and says it is upstreaming the technique into vLLM. An August 2026 research note reported a Minitron 4B-to-Llama 3.1 8B handoff reaching first token 7.9x faster than native prefill on an 8K-token prompt, at 82.5% top-token agreement over 32 generated steps.
To sell before the research matures, OneTriangle launched managed hosting for open-weight models — starting with DeepSeek V4 Flash in August 2026 at $0.15 per million input tokens and $0.35 per million output (a delayed tier 40% lower), prices slightly above DeepSeek's own API. It claims roughly 10,000 output tokens per second across 128 concurrent requests on an eight-H100 configuration, positioning the product on serving speed and long-context economics rather than sticker price.
The company's own numbers temper the pitch: including the small model's initial read cuts the speedup from 7.9x to 1.25x, and the agreement test is a bounded continuation rather than proof of equal quality. OneTriangle is live and publicly priced, but the transfer-layer bet is still unproven as of late August 2026.
What has to be true
- Bet on a structural cost shift: inference has overtaken training as AI's biggest compute cost, and prefill dominates long-context and agent workloads.
- Technical wedge: cross-model KV cache transfer has never shipped in production, so a working transfer layer could own inference economics.
- Speed-to-market: hosting DeepSeek V4 Flash gives customers a concrete product while cache-transfer research matures.
- Founder fit: MIT CSAIL founders with DeepMind and Jane Street experience attacking a serving-infrastructure problem.
- Ecosystem play: upstreaming into vLLM makes the technique reach every vLLM deployment instead of only OneTriangle's own stack.
What can be applied
When everyone resells the same open weights, serving performance is the only wedge — but a lab speedup that shrinks end-to-end and is only self-benchmarked is not yet a moat.
Aftermath
As of August 24, 2026 OneTriangle runs a paid, publicly priced DeepSeek V4 Flash service on eight H100s and lists Qwen3.6 27B in its catalogue, with usage-based pricing and no subscription. The KV-cache-transfer research remains pre-production: its strongest published result is the Minitron 4B-to-Llama handoff, which the company's own note reports collapses from 7.91x to 1.25x when the small model's read is included. No customer names, revenue, or independent benchmarks had been published, and the 'cheapest, fastest' claim had not been independently verified.
Sources
- OneTriangle launches with YC S26
- OneTriangle launches DeepSeek V4 Flash hosting at $0.15 per million input tokens
- OneTriangle: The cheapest, fastest lightweight inference
spotted an error? The archive wants to know.
Your turn
You just read one. Describe what you are building, and see who is betting on the same thing.
Free account · 3 free questions · no card