Research Engineer in Toronto

Every week, someone asks which data provider is actually best: for company records, for finding the right person, for fresh job listings. Right now the answers come from the vendors themselves. You’ll build the independent version. You’ll run rigorous, automated, public benchmarks of the providers in Alexandria and beyond, and publish them weekly. The same results will feed straight back into Alexandria so it learns which provider to call for which job. This isn’t our internal evals role. You’re measuring the market, in public, where every number will be challenged by the vendors it ranks. You’ll own it end to end: the datasets, the ground truth, the scoring, the harness, the weekly release, and the loop into Alexandria’s provider selection. No one hands you a methodology. You write it, defend it, and ship it every week.

What You’ll Do

• Design and run head-to-head benchmarks of data providers against verified ground truth.
• Build and maintain the test datasets and ground truth, and keep them from going stale or leaking.
• Own the automated harness and the weekly benchmark release. Every run has to be reproducible, versioned, and defensible.
• Measure what buyers actually care about: accuracy, coverage, freshness, speed, and cost.
• Work with marketing to ship public leaderboard pages that are useful and hold up under scrutiny.
• Close the loop into Alexandria so benchmark results change which provider gets called for what.
• Keep expanding the categories we benchmark, inside Alexandria and beyond it.

What We’re Looking For

• You’ve shipped evals or benchmarks, and you can explain exactly why your results were trustworthy.
• You’ve done this somewhere that matters: a frontier lab, a data company like Scale, Surge, micro1 or Mercor, a third-party benchmark org, or a public benchmark project. OSS contributors very welcome.
• You’re strong in Python and API integrations, and comfortable with messy vendor APIs, rate limits, and inconsistent schemas.
• You know how to build test sets and scoring methods: sampling, labeling, inter-rater agreement, and when to trust an LLM judge and when not to.
• You use AI heavily and keep upgrading your own workflow. You ship without waiting for instructions.
• You write findings clearly for engineers, marketers, and the vendors on the other side of the leaderboard.

What We’re Not Looking For

• Someone who wants to run benchmarks someone else designed.
• Someone who picks the metric that makes the story look good. Our numbers have to survive the vendors who lose.
• A pure researcher who won’t build the harness, or a pure engineer who won’t think hard about methodology.
• Someone who needs a fully specced ticket, or a quarter, to ship the first result.

A Note On Pace

We operate at an absurd level of urgency because the window for what we’re building won’t stay open forever. If that excites you, keep reading. If it doesn’t, no hard feelings, but this role probably isn’t for you.

Benefits & Perks

• Salary that makes sense: $250,000–$290,000 USD/year, / $210,000–$224,000 CAD/year (Toronto), based on impact, not tenure.
• Generous PTO: 15 days mandatory, anything after 24 days, just ask (holidays excluded). Take the time you need to recharge.
• Parental leave: 12 weeks fully paid, for all parents.
• Wellness stipend: $100 USD/month for the gym, therapy, massages, or whatever keeps you human.
• Learning & Development: Expense up to $1,000 USD/year toward anything that helps you grow professionally.
• Sabbatical: 3 paid months off after 4 years, do something fun and new.

Interview Process

• Application Review: We want the benchmark, leaderboard, eval, or dataset you built, and how you knew it was right. We care about what you’ve shipped, not where you went to school.
• Intro Chat (~25 min): A quick conversation to get to know each other. We’ll cover what you’ve been working on, what drew you to Firecrawl, and what you want next. Time for your questions too.
• Technical Chat (~45 min): A real problem from our world: design a benchmark that decides whether FullEnrich or DataLegion finds the right person’s current title.
• Workflow Chat (~30 min): Show us how you actually work: your AI tools, your setup, and a recent thing you shipped faster than you would have a year ago.
• Founder Chat (~25 min): Culture, pace, ownership, and how you like to work. Time for your questions too.
• Paid Work Trial (~40 hours): Ship a small benchmark end to end on a real provider category, paid at a contractor rate.

If you want to be the person the whole market checks before picking a data provider, you should join us.