What changed in rank data collection
Something changed in how search engines serve SERP data. It started with Google making it harder to retrieve full search results in bulk. Data providers found workarounds for a while, but those workarounds stopped working. Today, retrieving the top 100 results for a query costs ten times what it did a year ago. The cost of collecting rank data at competitive depth increased tenfold, industry-wide.
Every rank tracking tool faces the same math. The difference is in how each one responded.
Most providers chose the quiet route: reduce how deep they track, move from daily to weekly, or add pricing tiers for what used to be standard. Some removed the second device from the default plan entirely. The industry standard became “less data, same price” or “same data, higher price.”
We chose a different path.
Why depth and frequency matter more than most people realize
Before explaining what we built, it’s worth understanding why cutting depth or frequency isn’t just “less data.” It fundamentally changes what you can and can’t do with a rank tracker.
Daily frequency + full depth lets you:
- Correlate your initiatives with their actual impact. You shipped a content update on Tuesday. Did rankings move on Wednesday, or was it just Google volatility? You can’t answer that with weekly data. Daily tracking at depth lets you connect cause and effect reliably.
- See progress while not in the top 20. If your keyword moved from position 85 to 42, that’s meaningful progress. With top 20 tracking, both positions show as “20+.” You’re flying blind on the keywords where you’re making the most ground.
- Prioritize correctly. A high-volume keyword where you rank #21 is a completely different priority than one where you’re at 99+. Without depth, both look the same. You can’t allocate resources effectively if you can’t distinguish near-misses from long shots.
- Know whether a drop is minor or catastrophic. You dropped from position 8. Did you land at position 25, or did you fall off the first 10 pages entirely? The response to each is completely different. One might fix itself. The other needs immediate action. Without depth, you don’t know which one you’re looking at.
- Distinguish permanent drops from SERP volatility. Google’s search results fluctuate constantly. A keyword can drop 10 positions on Monday and recover by Wednesday. Daily tracking at depth shows you the pattern. Weekly tracking shows you a snapshot that might catch the dip, the recovery, or neither.
- Forecast more reliably. Forecasting SEO outcomes depends on knowing where keywords actually rank today, not just whether they’re “somewhere below position 20.” The more precise your starting positions, the more reliable your projections.
When a tool cuts to top 30 daily and full depth only twice a month, it breaks most of these use cases. When it moves to weekly, it breaks the rest.
If you run SEO for clients, the stakes are even higher. These aren’t abstract capabilities. They’re what drives your strategic decisions, your speed of reaction when something breaks, your ability to learn from experiments and course-correct, the credibility of the reports you share with clients, and ultimately your team’s profitability. Losing depth or frequency doesn’t just mean less data. It means slower decisions, weaker client conversations, and less confidence in every recommendation you make.
How the industry responded
We track this closely because our customers ask us how we compare. Here’s the pattern we see across the industry:
- Some moved rank tracking to weekly by default. Daily is available at additional cost.
- Some track only the top 30 daily, with full depth only twice per month.
- Some removed the second device from standard plans entirely.
- Some introduced premium tiers for features that used to be included.
The direction is clear: less data at the same price, or the same data at a higher price. The question we asked ourselves was whether there’s a way to give you more.
What we built instead: Dynamic Depth Crawling (DDC)
The insight behind Dynamic Depth Crawling is simple. You don’t need all 100 positions scraped for every keyword, every day. What you need is to know exactly where you rank, every day, wherever that position is.
DDC works differently from a fixed crawl. Instead of always collecting the same number of pages, it searches until it finds your website, then stops. For a keyword where you rank #3, it stops at page 1. For one where you rank #47, it goes to page 5. For one where you’re at #92, it goes to page 10.
The result: daily granularity at full competitive depth, at a cost structure that’s sustainable.
This means your primary device gets better data than before. Not “the same.” Better. You get your actual position, wherever it is, refreshed every day. We’re shipping Dynamic Depth Crawling by end of April, early May.
The tradeoffs we did make
We’d be dishonest if we said nothing changed. Here’s what we adjusted, and why:
Secondary device: daily to weekly. The core use case for tracking two devices is spotting divergence between mobile and desktop performance. These discrepancies don’t appear overnight, neither are they solved overnight. Weekly gives you the data to identify and act on device-specific trends. Daily frequency on a device most teams check less often was the clearest place to optimize.
Research, Vault, and draft campaigns: top 20. These features are unlimited and included in every SEOmonitor subscription at no additional cost, and we’re keeping them that way. Keyword Research and Keyword Vault were already tracked on a single device. For draft campaigns, we’ve reduced from two devices to one. Across all three, depth moves from top 100 to top 20. These features support discovery and planning, where directional data is sufficient. A keyword showing up at position 15 in your research tells you the same thing whether it came from 2 pages or 10.
What we kept intact: both devices tracked and included in every plan at no extra cost. Top 100 depth on both (weekly on secondary, dynamic daily on primary). Competitor data fully captured weekly across both devices. Every feature preserved. No use case broken.
What this means for your data going forward
| Area | What you get |
|---|---|
| Primary device | Dynamic depth daily. Your actual position, wherever you rank in the top 100, refreshed every day. |
| Secondary device | Top 100 weekly. Full competitive depth at a cadence that matches the use case. |
| Competitor data | Fully captured weekly on both devices, top 100. |
| Research, Vault, Drafts | Top 20 on one device. Directional data for discovery and planning. |
The principle
When costs change, you have two options: cut, or build something better.
Cutting is faster. It ships in a day. You reduce depth, remove a device, or move to weekly. Customers notice gradually, and by then it’s the new normal.
Building takes longer. It requires rethinking how the system works. But it’s the only option if you believe that daily, granular rank data is what makes a rank tracker worth using.
We chose to build. Dynamic Depth Crawling is our answer to the industry’s cost crisis. Not less data. A better system for collecting it.
Cosmin Negrescu is the CEO and co-founder of SEOmonitor.
Notes
1 Keyword Research: SEOmonitor’s research databases and Domain Explorer let you explore keyword opportunities by topic, including search volumes, difficulty scores, and current rankings. Also available through the API without restriction. Unlimited queries, included in every plan at no additional cost.
2 Keyword Vault: A saved collection of keywords you’re monitoring monthly but haven’t added to active tracking yet. Useful for building future campaigns, monitoring trends, or keeping a pipeline of opportunities. Unlimited, included in every plan.
3 Draft Campaigns: Before committing keywords to a paid tracking campaign, you can create a draft to preview rankings, estimate traffic potential, and build a forecast. Unlimited drafts, included in Pro and above plans.
* On terminology: When we say “top 20,” we mean the first 2 pages of Google search results. “Top 100” means the first 10 pages. Google doesn’t always show exactly 10 organic results per page, so these are approximations. The tracking depth is page-based, not position-based.