The archive · AI & Models · Product decision · 2025–2026
Forge: guardrails take an 8B local model from 53% to 99% on agentic tasks (687 HN pts)
Open-source guardrails layer that validates, rescues and retries LLM tool calls, lifting an 8B self-hosted model toward frontier-agent accuracy; Show HN 687 pts
Forge (antoinezambelli/forge)
What the business is
An MIT-licensed open-source Python reliability layer that sits between a client and a self-hosted LLM backend, validating, rescuing and retrying malformed tool calls so local agents run dependable agentic loops.
How it started
Antoine Zambelli created Forge (MIT, copyright line 2025–2026) as a reliability layer for self-hosted LLM tool-calling, working with Ollama, llama.cpp, llamafile, vLLM and Anthropic backends. On 2026-05-19 he showed it on Hacker News under the headline that guardrails take an 8B model from 53% to 99% on agentic tasks; the README reported the same 8B model going from single digits to 84% across a 26-scenario eval suite.
What happened
The project grew three integration paths: a drop-in proxy server speaking OpenAI chat-completions and Anthropic Messages so existing harnesses (opencode, aider, Cline, Claude Code) gain guardrails without rewrites; a WorkflowRunner for structured agent loops with context compaction and priority-queued shared inference slots; and composable middleware for foreign orchestration loops. A v0.9.0 release made the proxy the flagship entry point, standalone installers bundle a private Python runtime, and the guardrail framework plus ablation study were published as Zambelli, Forge: Closing the Agentic Reliability Gap Between Self-Hosted and Frontier Language Models (DOI 10.1145/3786335.3813193).
How it ended up
Still live as an open-source project as of 2026-09-04: 87 commits, 2.2k stars and 173 forks, with no company, funding round or commercial product disclosed in the material.
Background
Forge is an MIT-licensed open-source reliability layer for self-hosted LLM tool-calling: give it a set of tools and a local model, and it validates every tool call, rescues malformed output and retries with corrective feedback, with optional workflow structure (required steps, prerequisites, terminal tools) layered on top. Its bet is that the loop around the model — not a bigger model — is what makes agents reliable, so an 8B local model can stand in for frontier APIs.
The mechanism is a stack applied in order: response validation checks each call against the declared tools; rescue parsing recovers tool calls emitted in vendor formats like code-fenced JSON, Mistral-style [TOOL_CALLS] blocks or Qwen XML; and a retry loop feeds validation errors back to the model instead of returning a malformed response. Forge's 26-scenario eval suite reported an 8B local model rising from single digits to 84%, while the same guardrails lifted Sonnet 4.6 from 85% to 98% (v0.6.0 measurements).
Three integration paths grew out of it — a drop-in proxy server speaking OpenAI chat-completions and Anthropic Messages, a WorkflowRunner for structured loops, and composable middleware for foreign loops. The Show HN on 2026-05-19 drew 687 points and 252 comments, and the repo carried 2.2k stars and 173 forks by the 2026-09-04 fetch; the framework and an ablation study were published under a DOI (10.1145/3786335.3813193).
What has to be true
- It attacked the actual failure mode of local agents — malformed and invalid tool-call output — with engineering instead of waiting for model progress, making the reliability problem tractable today.
- The guardrail stack is inspectable and falsifiable: response validation, rescue parsing and retry nudges are concrete mechanisms backed by a 26-scenario eval suite and an ablation study with a DOI.
- The drop-in proxy design meant existing harnesses like Claude Code, opencode or aider gained the guardrails with zero rewrites, distributing through tools developers already run.
- The headline number was deliberately provocative — an 8B model from 53% to 99% — and it reframed what local models could do, which is what earned a 687-point, 252-comment HN launch.
What can be applied
Fix the loop, not the model: transparent validation, rescue parsing and corrective retries turned an 8B model's flaky tool calls into a solvable problem, and lifted a frontier model from 85% to 98%.
Aftermath
As of the 2026-09-04 fetch, Forge remains an active MIT open-source project: 87 commits, 2.2k stars and 173 forks on GitHub, with the standalone proxy described as its most popular entry point and v0.9.0 positioned as an intentional breaking move toward proxy-first usage. The 26-scenario eval harness, model and backend guides, and the published paper remain the project's evidence base. No company, funding round or commercial product is disclosed in the material.
Sources
spotted an error? The archive wants to know.
Your turn
You just read one. Describe what you are building, and see who is betting on the same thing.
Free account · 3 free questions · no card