# Claude-SearchBot

Operator: Anthropic
Purpose: AI search
Reviewed: 2026-09-14
HTTP identifier: Claude-SearchBot
robots.txt token: Claude-SearchBot

## What is Claude-SearchBot?

Indexes web content to improve Claude search responses.

## How to identify it

Look for the Claude-SearchBot identifier in a request’s User-Agent. This is a name match, not identity verification. Version strings may change.

## How to check identity

Compare the request IP with Anthropic’s current crawler list linked at https://claude.com/robots. The User-Agent alone does not authenticate the request.

## Access and robots.txt

Use the Claude-SearchBot robots.txt group to control this search-indexing path. Blocking it can limit discovery through Claude search; configure each host separately.

Optional full-site opt-out; merge with existing rules only if intended. Not a security control.

```text
User-agent: Claude-SearchBot
Disallow: /
```

## What it means in your analytics

Search crawl coverage is a discoverability signal. It cannot establish whether Claude recommended your product.

## Useful measures

Public-page coverage; Retrieval failures; Referrals separately.

## What to check next

Review failed retrievals on canonical public pages. Compare missing pages against navigation and sitemap coverage.

## Sources

- [support.claude.com](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler)
- [claude.com](https://claude.com/robots)

Operator facts are source-backed; interpretation and actions are Apostl guidance.

## Related profiles

- [ClaudeBot](https://apostl.dev/bots/claudebot)
- [Claude-User](https://apostl.dev/bots/claude-user)

[Start measuring agent traffic](https://apostl.dev/start/agent-analytics)