The archive · Developer & Business Tools · Strategic decision · 2022–2025
Arroyo (YC W23): Cloudflare acquires its SQL stream processor for Pipelines
Flink-class stream processing made developer-friendly: Arroyo wrote a Rust SQL engine, open-sourced it, and sold to Cloudflare in April 2025.
Arroyo
What the business is
Arroyo is an open-source (Apache-2.0) stream-processing engine: developers write SQL to run stateful aggregations, joins and transformations on real-time data, deployed self-hosted or as a managed service.
How it started
Micah Wylde and co-founder Jackson started Arroyo in 2022 after years building stream processing at Lyft and Splunk. The dominant engine, Apache Flink, was powerful but, in their view, too low-level for non-expert users and operationally heavy — which is why most teams still ran batch pipelines — so they set out to build a streaming engine product engineers and data scientists could actually use, with SQL as the declarative API and Rust for operational simplicity.
What happened
The engine was open-sourced in summer 2023 under an Apache license; by the acquisition the company said dozens of businesses were running Arroyo pipelines for data ingestion, anti-fraud, IoT observability and financial trading. The team concluded the engine alone was only part of the product — democratizing streaming meant a full platform with backfill, query testing and serverless deployment — and, after about a year of conversations with Cloudflare, concluded the developer platform already owned the missing pieces: R2 for object storage, Queues for data in transit and Workers for running user code.
How it ended up
On 2025-04-10 Cloudflare announced it had acquired Arroyo, bringing serverless SQL stream processing to its Developer Platform: the engine became the basis for streaming SQL inside Cloudflare Pipelines, which launched in open beta the same day, and Arroyo remained open-source and self-hostable with the team joining Cloudflare.
Background
Arroyo is an open-source, cloud-native stream-processing engine founded in 2022 by stream-processing veterans from Lyft and Splunk. Developers write SQL to run stateful aggregations, joins and transformations over real-time data — the kind of pipelines behind recommendations, anti-fraud and observability — without hiring a team of streaming specialists.
The founding bet was that Flink had proven the power of fault-tolerant, stateful stream processing but had failed at usability: its API was too low-level and its stateful services demanded endless operations, leaving most companies on hourly or daily batch. Arroyo rebuilt the concept in Rust with SQL as the interface and an object-storage-native state backend, then open-sourced it in summer 2023 under an Apache license.
By 2025 the company reported dozens of production users, and on 2025-04-10 Cloudflare announced the acquisition: Arroyo's engine became the streaming SQL layer of Cloudflare Pipelines (launched in open beta the same day), integrated with Queues, R2 and Workers, while the engine stayed open-source and self-hostable. The acquisition drew 113 points and 38 comments on Hacker News.
What has to be true
- Flink was technically right but adopted only where specialists existed; SQL and Rust attacked the actual bottleneck — accessibility and operations — rather than raw throughput.
- An object-storage-native state backend removed the burden of running stateful pipelines, which behave like specialized databases.
- Open-sourcing in summer 2023 created the credibility and community that let dozens of companies run the engine before any platform deal existed.
- Cloudflare already owned storage (R2), transport (Queues) and compute (Workers), so joining it turned a standalone engine into a serverless data platform without building cloud infrastructure.
- Keeping the engine Apache-licensed after the acquisition protected the open-source trust the project had built.
What can be applied
Accessibility was the wedge: Flink-grade streaming with a SQL API and no JVM ops. The open-source engine built the credibility that ended with Cloudflare as both the platform and the acquirer.
Aftermath
As of 2026-09-05 Arroyo operates inside Cloudflare: the engine remains open-source and self-hostable on VMs, Kubernetes and serverless container platforms, and the team's stated work is integrating SQL stream processing with Cloudflare Pipelines, Queues, R2 and Workers, with UDFs run on Workers on the roadmap.
Sources
- Just landed: streaming ingestion on Cloudflare with Arroyo and Pipelines
- Arroyo (YC W23) has been acquired by Cloudflare (Hacker News discussion)
spotted an error? The archive wants to know.
Your turn
You just read one. Describe what you are building, and see who is betting on the same thing.
Free account · 3 free questions · no card