Outlines (.txt): masking invalid tokens at sampling made LLM output structured

Outlines is an open-source Python library that guarantees the structure of LLM output during generation: instead of asking a model for JSON and repairing or retrying what comes back, it constrains sampling itself so every token emitted keeps the output valid against a regex, JSON Schema, Pydantic model or context-free grammar. The project was started in mid-2023 by Rémi Louf and Brandon T. Willard, whose backgrounds were probabilistic, relational and symbolic programming rather than LLM research, and who said they began it simply to understand how text generation works.

The founding bet was that reliability is won at sampling time, not after it: convert the target regex or schema into a finite-state machine, re-derive that machine over the language model's token vocabulary in one pass, then mask out invalid tokens at each step with a dictionary lookup. That made guided generation fast enough for production — near the speed of unconstrained sampling — and guaranteed parseable output for any model, including small ones. The approach was written up in 'Efficient Guided Generation for Large Language Models' (arXiv 2307.09702, first version July 2023).

Louf announced the library on Hacker News on 2023-08-14 with the title 'Show HN: LLMs can generate valid JSON 100% of the time'. The post drew 854 points and 303 comments, and the discussion pitted the token-masking approach against Microsoft TypeChat, guidance, LMQL and llama.cpp's grammar sampling. The authors argued their one-pass index was far cheaper than full-vocabulary masking at every step. The project later moved from the normal-computing GitHub org to dottxt-ai/outlines as the maintainers formed the company .txt (dottxt).

As of 2026 the library is still actively maintained — the 2026-09-04 snapshot shows 15.7k stars, 868 forks and 1,325 commits — and the README advertises model integrations spanning vLLM and Ollama servers plus OpenAI, Gemini and Dottxt APIs, while naming NVIDIA, Cohere and HuggingFace among trusted users. .txt sells the commercial layer: an early-access structured-generation API, schema audits that measure compliance rates, and enterprise-grade libraries. No funding or revenue figures appear in the material.