See what Googlebot actually does on your server
A reading shows what your site offers. Your server logs show what Googlebot actually took. Six screens, verified crawlers only, and not one visitor stored. What Log Intelligence shows you, feature by feature.
Coverage: which templates Google visits
Which templates Google visits and which it skips, week by week.
A reading can tell you a page is indexable, linked and in the sitemap. It cannot tell you whether Google ever came for it. Your server knows, because it writes every request into the access log. Upload those logs and the module reads them against your latest reading.
The Coverage screen shows, per URL template, how much of it verified Googlebot reached, with the change week over week and a trend. Templates come from the reading, not from the logs, so a template Googlebot ignores completely still shows up, at zero percent. That row is often the most useful one on the screen.
Uncrawled: the pages nobody came for
Pages Googlebot never came for. Not once.
Uncrawled lists the indexable URLs with no verified Googlebot hit in the period. You can filter by template and by click depth, so you see whether the gap sits deep in the site or right under the homepage.
The same question now reaches your weekly plan. With logs uploaded, each reading compares its indexable pages with a month of verified Googlebot traffic and raises one card for the pages it has not fetched. The card stays quiet with fewer than seven days of Googlebot traffic in that month, because an absence over three days is not an absence.
Orphans: addresses only the bots remember
URLs the bots know that your site has forgotten.
Orphans are URLs that bots fetched and your reading never found. Each row shows the top crawler that asked for it and when it was last seen.
If the list is full of your own pages with a trailing slash or an extra parameter, the cause is usually canonicalization, not your site. The diagnostics panel shows how many verified crawler hits match a page in the reading, and turns red below 95%, pointing you at the settings to fix.
Waste: where the crawl budget goes
Where crawl budget burns on errors and parameters.
Waste counts the bot hits that bought nothing: requests for non-indexable URLs, 4xx and 5xx responses. It breaks them down by query parameter, so one filter or tracking parameter eating thousands of fetches stands out on its own row.
Tracking parameters such as utm_*, gclid and fbclid are stripped before URLs are matched, and you can mark the parameters that really change the content, so they are kept.
Bot activity: the real Googlebot, and the rest
The real Googlebot on one line, impostors on another.
Anyone can write "Googlebot" in a request header. So every hit is checked against the IP ranges that Google, Bing, OpenAI and Perplexity publish. A request that claims a crawler from an address outside those ranges is counted as spoofed: stored, drawn separately, and never mixed into the numbers you decide on.
Some AI fetchers, ClaudeBot among them, publish no ranges. Their hits are still shown by name, and never marked spoofed, because there is nothing to check them against. If the published ranges are more than two weeks old, the import refuses to run at all. A report built on stale ranges would be wrong with full confidence.
Time to first crawl
How many days a new page waits before Google comes.
This screen counts the days from a URL first appearing in a reading to its first verified Googlebot hit. You get a median and a p90 per template, so you know whether new product pages wait two days or two weeks.
One honest caveat: on your first import every URL counts as first seen that day. The numbers become accurate from the second import onwards.
Privacy: your visitors are never stored
Log analysis that never stores a visitor. Nothing to anonymise.
Human traffic is discarded while the file is parsed. It is never written. What stays is verified crawler activity and, per day, one count of total, bot and human lines.
So there is no visitor IP address in our database to export, anonymise or delete. When an import finishes, you get an email with the counts, including how many human lines were thrown away.
Alerts and the weekly digest
An email when coverage drops or Googlebot hits errors.
Three alert rules, each with one threshold you set. Coverage of any template drops week over week by more than your number of points. Verified Googlebot sees server errors above your chosen share. Or verified Googlebot volume falls against its median of the last seven days.
Once a week a digest brings the coverage table with its changes, so you read it without opening the app.
What it deliberately does not do
Every open design decision in this module went to the simplest thing to build and run, even when that leaves a step to you. Better to know the edges before you start.
- No format auto-detection: you declare nginx, Apache, IIS or Cloudflare Logpush once, and check it against the first hundred lines of your own file.
- No log pull from S3, GCS or FTP, and no real-time ingest: you upload files.
- No Slack or webhook alerts, no rule builder, no custom dashboards.
- An import with more than 2% unparseable lines is rolled back. A half-imported day is worse than no day.