Claude-SearchBot

Anthropic's search crawler. It indexes content to improve the quality of Claude's search results.

What happens if you block it

Claude won't index your content for search, which may reduce your visibility and accuracy in Claude's answers.

Our take

Allow it if you want to be found and cited in Claude search.

robots.txt

To block it:

User-agent: Claude-SearchBot
Disallow: /

To allow it explicitly (useful when a wider rule blocks other bots):

User-agent: Claude-SearchBot
Allow: /

The operator's own words

Disabling Claude-SearchBot on your site prevents our system from indexing your content for search optimization, which may reduce your site's visibility and accuracy in user search results.Anthropic documentation, checked 2026-10-31

robots.txt allows it, but is it really getting through?

A firewall or CDN rule can reject Claude-SearchBot even when robots.txt welcomes it, and nothing in your CMS shows it. Our check sends a live request as Claude-SearchBot alongside a normal browser and a Googlebot control, so a bot-verification rule isn't mistaken for a block.

Other Anthropic agents