Googlebot

Google's main search crawler. Rules for Googlebot apply to Google Search, Discover and other Google Search features.

What happens if you block it

Your site drops out of Google Search, including AI Overviews. This is almost never what you want when opting out of AI.

Our take

Keep it allowed. To opt out of Gemini training, disallow Google-Extended instead.

robots.txt

To block it:

User-agent: Googlebot
Disallow: /

To allow it explicitly (useful when a wider rule blocks other bots):

User-agent: Googlebot
Allow: /

The operator's own words

Crawling preferences addressed to the Googlebot user agent affect Google Search (including Discover and all Google Search features), as well as other products such as Google Images, Video mode, Google News, and Discover.Google documentation, checked 2026-10-31

robots.txt allows it, but is it really getting through?

A firewall or CDN rule can reject Googlebot even when robots.txt welcomes it, and nothing in your CMS shows it. Our check sends a live request as Googlebot alongside a normal browser and a Googlebot control, so a bot-verification rule isn't mistaken for a block.

Other Google agents