OpenAI
ChatGPT-User
Fetches pages for user actions in ChatGPT and custom GPTs.
User-requested fetchUser-requested fetch
Claude-User
Retrieves web content in response to Claude users.
Operated by Anthropic
See your agent traffic PurposeUser-requested fetch
HTTP identifierClaude-User
IdentityVerify beyond the name
THE SIGNAL
What this visit tells you
This client retrieves pages for Claude user requests. Verify the source before interpreting a visit as an assistant interaction.
Requested public pages
Public-page 403 and 404 responses
Subsequent page requests
Your next move
Separate public-page failures from security probes. Preserve the response, path and time before attributing a block to Claude.
The technical details
MarkdownHow to identify it
Look for the Claude-User identifier in a request’s User-Agent. This is a name match, not identity verification. Version strings may change.
How to check identity
Compare the request IP with Anthropic’s current crawler list linked at https://claude.com/robots. The User-Agent alone does not authenticate the request.
Access and robots.txt
Anthropic documents robots.txt controls for Claude-User. Apply rules on each host you manage; this setting is independent of its other crawler identities.
Optional full-site opt-out. Merge with existing rules only if intended. robots.txt does not secure private content.
User-agent: Claude-User
Disallow: /Seeing 403 or 404 responses?
403 means access was denied; 404 means the resource was not found. Compare the public path, request time, edge security event and origin response to find the cause.
Separate intended restrictions and secret-file probes from pages that should work. A User-Agent name alone does not justify allowing a request.
SourcesReviewed 2026-09-14
Identity and purpose are based on these sources. Analytics interpretation and suggested checks are Apostl guidance.
APOSTL PulseSee Claude-User in context
See their requests. Find the pages that matter.
See your agent trafficKEEP EXPLORING
Different bots, different intent
Perplexity
Perplexity-User
Fetches pages on behalf of Perplexity users.
User-requested fetch