M

how to build your own 1 million lead database

6 min readView source ↗

Cover image

the businesses you want to reach are already listed somewhere public. a million of them across enough sources isn't a stretch, it's just a matter of pulling that data into a format you can send to, and owning the database yourself.

here's the full setup to do that, from the tools to the scrapers.

Want 500 free leads? maps.hiivearts.com

The stack

three pieces run everything.

claude code. anthropic's coding agent that runs in your terminal. you describe what you want built in plain english and it writes the code, runs it, and fixes its own errors. you don't need to know how to write a scraper. you need to know how to describe one.

a vps. a virtual private server is a cheap remote machine that stays on. your scrapers run there around the clock instead of your laptop, so a job pulling tens of thousands of records runs overnight while you're doing something else. a basic one runs a few dollars a month (enough for most scraping loads).

headless browsers. a lot of directories load their data with javascript, so a plain request pulls back an empty page. a headless browser opens the site the way a real one does, waits for the content to render, and reads it. claude code sets this up with playwright or puppeteer when a site needs it.

that's the base. claude code to build, the vps to run it, headless browsers for the sites that need a real render.

pulling bulk contact data

before you build anything custom, a lot of contact data is available through apis you can pull from directly.

rapidapi is a marketplace of those apis. business listings, contact enrichment, email lookups, company data, all sitting behind endpoints you can call. you grab an api key for a provider, and claude code writes a short script that hits the endpoint and dumps the results to a file.

if you're pulling at volume, spread the load across a few different providers and keys rather than one. each api has its own coverage and its own rate limits, so rotating between them gets you more records and keeps any single source from throttling you mid-pull. one provider might be strong on a vertical another is thin on, and the overlap fills the gaps.

this gets you a base of structured data fast, without writing a scraper for every source.

building custom scrapers

the apis cover the common sources. the leads nobody else is emailing come from the sources that don't have a clean api, and that's where custom scrapers come in.

point claude code at a site. a licensing lookup, an association member list, a niche directory, a nonprofit registry. tell it what you want off each listing: name, business, email, phone, website, location. it writes the scraper, handles the pagination, and runs it on the vps.

the parts that matter once you're pulling from many sources:

dedup. the same business shows up across five directories with five spellings. match on name, location, and email domain and collapse the duplicates, or you pay to email the same person five times.

verification. run the emails through a verifier before you send, so bounces don't wreck your domain reputation.

output. write to a csv or a database you can query, so you can pull a specific slice later instead of re-scraping the source.

you describe each of these to claude code once and it builds them in. the result is a pipeline you own that keeps pulling fresh records for as long as you run it.

google maps

one source is worth calling out on its own.

google maps is one of the largest and freshest business databases available. every small business that wants to be found puts itself there, with a category, a location, a phone, often a website, and a public review count you can read as a rough size signal. it covers a huge range of local smbs, and on the niche ones fewer people are sending, so reply rates on those tend to run higher (varies by vertical and how you approach them).

it also stays current on its own. listings change as businesses open, close, move, and collect reviews, so the data doesn't go stale the way a one-time list does.

scraping maps well at scale is its own build. we already did it.

the steps

pick a source. a directory, registry, association list, or licensing lookup with the businesses you want.

open the site and look at how it's laid out. is it paginated, is there a search page, does the data load on click.

buy a cheap hetzner vps and get claude code running on it.

tell claude code the url and the exact fields you want off each listing.

let it write the scraper and run it on a small sample first.

check the sample. right fields, clean values, no missing rows.

tell it to add headless browser handling if the page came back empty.

run the full pull once the sample looks right.

dedup the output against your other sources.

verify the emails before anything gets sent.

write the clean records to your csv or database.

schedule it to re-run so the source stays fresh.

tip: ask claude code about details for any of these steps & give it this article as an outline for the project to give it more context.

the shortcut

I have already built this internally for our team, and we just launched a beta 3 weeks ago.

a google maps database directly. you pick the niche, the city, the review threshold, whether they have a website, and pull exactly the slice you want. it's self-serve, so you run your own query and get your own list.

you can try it free and get 500 leads -> maps.hiivearts.com

If you want a the full system, infrastructure, dashboards, knowledge base setup, and sub-sequence automations, built out for you, let's talk.

cal.com/leviwelch/30min

Related articles