Super Enrich: From Open-Source Scraping Demo to Multi-Provider Enrichment Tool
Super Enrich started as Fire Enrich, an open-source demo Firecrawl built to show off their scraping API. I forked it and spent the following weeks turning a single-user, single-provider demo into something that could run for real people: pluggable scraping and LLM providers, full authentication, and the security hardening a public tool needs that a demo never did.
Key Insight
The gap wasn't AI quality. Fire Enrich's phased pipeline, each stage building on what the last one verified instead of asking one model call to guess everything at once, already produced solid extractions. The actual blocker was distribution: one provider pair, no concept of a user, no hardening for public traffic. That's a much smaller problem than "build a better enrichment engine," and it's the one I actually had to solve.
Context
Data enrichment tools tend to fall into two camps. Fully managed platforms, Firecrawl's own hosted product among them, do good work but charge per lookup in a way that adds up fast once you're enriching daily instead of occasionally. Raw APIs and open-source scrapers hand you the building blocks and leave the assembly, headless browsers, rate limiting, auth, all of it, to you. I wanted something that sat between those two, and I wanted to build it rather than keep looking for it.
Problem
The specific gap: no option existed that was both cheap enough for routine use and didn't require standing up scraping infrastructure yourself. Fire Enrich, Firecrawl's own open-source demo of their API, was close. It had a working phased extraction pipeline. What it didn't have was a way to run it for more than one person, a way to choose a cheaper provider when Firecrawl's own pricing didn't make sense for a given run, or any of the security hardening a public-facing tool needs that a demo repo never bothered with.
Users
Sales teams doing account research before outreach, recruiters screening companies attached to candidate emails, analysts building a market map from a lead list. Anyone with a spreadsheet of emails and no interest in opening ten tabs per company to find out who's behind them.
Research & Input
I looked at three paths before deciding. Building the scraping and extraction layer from scratch, which meant owning headless browser infrastructure, proxy rotation, and rate limiting before writing a single line of product logic. Paying for a fully managed platform like Firecrawl directly, which handles all of that but bills per lookup in a way that only makes sense occasionally, not as a routine tool. And extending something that already had the hard part solved. Fire Enrich was that third path. It's Firecrawl's own open-source demo, MIT licensed, built to show off their scraping API, and it already had a working phased extraction pipeline. Starting there meant I wasn't rebuilding scraping and extraction from zero, I was fixing the parts that made it a single-user demo instead of a real tool: no auth, one locked provider pair, no security pass.
Solution
Kept the original MIT license and Mendable AI's copyright exactly as they were, then rebuilt around three additions. A pluggable provider registry on both sides: Firecrawl, Tavily, or Serper for scraping, OpenAI, Gemini, or OpenRouter for extraction, chosen per run instead of hardcoded. A full multi-user auth layer with Better Auth on Postgres (Neon, through Drizzle), with a session check gating every page and API route that used to have none. And a security pass, including fixing an SSRF hole where a redirect could route a scraper request to a private or loopback address, done specifically because a demo doesn't have to survive public traffic and a real tool does.
Workflow
Upload a CSV, the app works out which column holds the email. Pick fields from a preset or describe them in a sentence and let an LLM turn that into a structured field list. From there the pipeline runs per row across six phases, Discovery, Profile, Metrics, Funding, Tech Stack, and General, each one handed what the previous phase already verified. Every phase searches the web in parallel, then an LLM has to fill a strict Zod schema or return nothing. Results stream to the browser row by row as they finish.
Product Decisions
Pluggable providers instead of matching Fire Enrich's original locked pair. Locking someone into one scraper and one model locks them into one price point, and the whole point of this project was giving people a cheaper path when they want one.
Phased extraction stayed as designed rather than getting collapsed into a single call for speed. Splitting industry, headcount, funding, and tech stack into separate phases that each build on verified prior output avoids the confident-sounding nonsense a single big extraction call tends to produce.
Kept the license and attribution untouched. This is someone else's open-source work at its core, and the fork should say so plainly rather than quietly.
Technical Implementation
Next.js 15 (App Router) and React 19, TypeScript throughout, Tailwind with Radix and shadcn for UI. The pipeline itself is a custom multi-agent setup, each "agent" a focused prompt paired with a Zod schema, run through the Vercel AI SDK against whichever provider is selected. Scraping goes through Firecrawl, Tavily, or Serper (a fourth adapter, TinyFish, partway built). Auth is Better Auth, email and password plus optional Google OAuth, on Postgres through Drizzle ORM. Zod validates every extraction before the app trusts it.
Tradeoffs
A few deliberate tradeoffs, not oversights. BYOK provider keys sit in plaintext in localStorage, tracked as follow-up hardening rather than treated as finished. Open signup currently rides the operator's own environment keys with no allowlist on which provider a user can pick and no server-side row cap, workable for a small user base, not for scale. Rate limiting only fully covers the scrape endpoint so far; a tiered system with auth backoff is built but not wired into every route yet. Error handling inside the pipeline is still all-or-nothing per row: one phase throwing wipes every phase's results for that row, including the ones that already succeeded.
Cost & Complexity
Solo build, no team. [fill in: actual timeframe and hosting cost, docs show first fork commit 2026-06-29 through active work as of 2026-07-10, so roughly two weeks of sessions so far]. Infra cost is whatever your Postgres and hosting setup actually runs, worth confirming before this goes live rather than me guessing. Engineering cost went mostly into debugging, not building: chasing a search-query grouping bug across six pipeline phases, a Serper integration silently never fetching page content, and an AI SDK version mismatch that broke two providers at once.
Result & Learning
The enrichment engine, provider switching, and the auth and database foundation all work end to end today. Three separate bugs that were silently producing zero usable results (query grouping, the missing Serper content fetch, an AI SDK version mismatch) are fixed, which mattered more than any single feature, since a tool that returns nothing is worse than one that's just slow.
What stuck with me: a query string that reads as grouped to a human doesn't necessarily parse that way to an API, site:X OR "name" kw1 kw2 looks scoped and isn't without parentheses. And "same major version" across two packages doesn't guarantee they speak the same internal spec, that's what broke Google and OpenRouter extraction until I traced it. Doing the SSRF pass before opening this up publicly also reframed what "done" means: a demo has to work when used as designed, a public tool has to survive being used the way it wasn't.
Next Improvements
Per-phase error isolation in the orchestrator, so one failing phase doesn't wipe results the other phases already got.
An empty-content guard around extraction calls, so a phase with no scraped content doesn't burn an LLM call on nothing.
An allowlist on which provider a signed-up user can pick, plus a server-side cap on rows per run, since open signup currently rides the operator's own keys with no ceiling.
Finish wiring the tiered rate limiter into every route, not just scrape.
Ownership checks on job cancellation, and Zod validation on the enrich and generate-fields request bodies.
Consolidate the two live cn() helpers and drop one of the two committed lockfiles before this goes fully public.