Building a self-hosted SEO monitoring stack on a homelab

Why I built it instead of buying

Every SEO tool I looked at wanted a monthly subscription, per site. And I don’t have one site — I have a portfolio, a handful of freelance template builds, and more on the way. Paying a per-seat fee for rank tracking I’d only run occasionally felt backwards.

So I did the build-vs-buy walkthrough properly. Rank tracking and technical crawling are commodity problems — there are mature open-source engines for both, and rebuilding them would be a waste of time. The one piece actually worth building is the thin layer on top: the thing that turns raw crawl and rank data into a short, prioritized “fix this first” list. That’s the product. Everything underneath it is just plumbing I can assemble.

The stack

Here’s what I landed on, all self-hosted on an old Dell Latitude E6330 that’s now my homelab:

SerpBear for rank tracking — Node, Docker-friendly, with native Google Search Console integration. One gotcha the docs bury: it doesn’t scrape Google itself. You have to bring your own scraper backend. I’m running Serper.dev.
advertools for crawling. I nearly reached for Greenflare, but it’s a desktop GUI app with no command-line interface — awkward to drive on a headless server. advertools is built for exactly this: one call runs a full crawl and hands back a structured table of status codes, titles, meta descriptions, canonicals, and h1s that I can feed straight into the pipeline.
DeepSeek as the glue brain. A weekly digest is low-volume, so I don’t need anything pricey — DeepSeek turns the raw crawl and rank data into the prioritized action list.
Postgres for state. This is the part that makes it more than a toy: every crawl, finding, and recommendation is stored, so the digest dedupes week over week instead of nagging me about the same meta tag forever.
Telegram for delivery.

How it actually runs

There’s no fancy orchestrator. A plain host cron job fires every Monday morning: crawl each site, dedup the findings against Postgres, let DeepSeek write the digest, and drop it into a Telegram channel. Deterministic, boring, reliable — which for a data pipeline is exactly what I want. I deliberately kept the agent framework out of this half; cron owns the data flow.

The first real thing it caught

The first genuine finding was on my own portfolio, and it stung a little. Four of five public routes were canonicalizing to a single page, with identical titles and meta descriptions and no <h1> at all — because the site was a single-page app that only rendered its content after JavaScript ran. To a crawler, every page looked the same. That was almost certainly why Search Console was showing me zero impressions.

The digest ranked it as fix number one. I fixed it by prerendering each route to real static HTML at build time. The next crawl flipped all eleven findings from “open” to “resolved.” That was the first proof the whole loop works: find, fix, verify.

Proving it on more than one site

A monitoring tool that only works on a single site isn’t a tool. So I added more tenants and made sure every row of data is keyed by site. The first new one immediately broke a hidden single-tenant assumption — the crawler had been requiring a parseable sitemap, which the new site didn’t have — so I fixed that too. Isolation holds now: findings, crawls, and history stay cleanly separated per site.

The gaps I didn’t see coming

The pipeline was good at finding issues on demand. What I underestimated was everything around that:

It couldn’t tell when it had failed. If a crawl died, or the flaky old laptop was asleep at run time, I’d just get silence — indistinguishable from “all clear.” I added a dead-man’s switch, so a missing weekly heartbeat is itself an alert.
It couldn’t see a deindexed site. A stray noindex tag or a Disallow: / is about the highest-severity SEO failure there is, and the crawl was blind to it. Now it’s a first-class check.
It could run out of credits silently. When the scraper’s free credits hit zero, rank tracking just stops. A cheap “credits low” check closes that hole.

The honest part

Here’s what I have to be straight about: every site I’m tracking has essentially zero search traffic. The pipeline has been validated entirely on self-inflicted technical bugs — never yet on real search demand. The genuinely smart analytics I wired in — classifying pages as gaining versus dying, flagging titles that under-perform their ranking — are built but dormant, because they need real impressions to chew on.

So that’s where this honestly sits: a solid, self-hosted monitoring stack that works, waiting patiently for something worth measuring. The next phase is to give it that — honest content people might actually search for (this post included), and faster indexing so the data finally starts to flow.