Googlebot
Crawls pages for Google Search using desktop and smartphone variants.
Web searchUser-requested fetch
Google-Agent
Navigates websites for user-requested actions on Google infrastructure.
Operated by Google
See your agent traffic PurposeUser-requested fetch
HTTP identifierGoogle-Agent
IdentityVerify beyond the name
THE SIGNAL
What this visit tells you
A navigation sequence can include several pages within one delegated task.
Requested URLs
Response codes
Redirect chains
Your next move
Review the sequence and completed actions separately from raw request counts.
The technical details
MarkdownHow to identify it
Look for the Google-Agent identifier in a request’s User-Agent. This is a name match, not identity verification. Version strings may change.
How to check identity
Match the source IP with Google’s user-triggered-agents.json, rather than the common crawler list. Google also documents experimental Web Bot Auth under agent.bot.goog.
Access and robots.txt
This user-triggered fetcher may ignore robots.txt. Use server access controls for private content.
Seeing 403 or 404 responses?
403 means access was denied; 404 means the resource was not found. Compare the public path, request time, edge security event and origin response to find the cause.
Separate intended restrictions and secret-file probes from pages that should work. A User-Agent name alone does not justify allowing a request.
SourcesReviewed 2026-09-14
Identity and purpose are based on these sources. Analytics interpretation and suggested checks are Apostl guidance.
APOSTL PulseSee Google-Agent in context
See their requests. Find the pages that matter.
See your agent trafficKEEP EXPLORING
Different bots, different intent
Google-Extended
Controls content use for Gemini training and grounding; it does not control Google Search inclusion.
Content-use policyPolicy control · no separate crawlerGooglebot-Image
Fetches images for Google’s visual search features.
Web search