Googlebot
Crawls pages for Google Search using desktop and smartphone variants.
Web searchAI search
Google-CloudVertexBot
Crawls sites for owner-requested Vertex AI agents.
Operated by Google
See your agent traffic PurposeAI search
HTTP identifierGoogle-CloudVertexBot
IdentityVerify beyond the name
THE SIGNAL
What this visit tells you
These requests may belong to a configured knowledge source.
Fetched pages
Failed retrievals
Repeat fetches
Your next move
Check the Vertex AI data source and verify that its intended pages can be retrieved.
The technical details
MarkdownHow to identify it
Look for the Google-CloudVertexBot identifier in a request’s User-Agent. This is a name match, not identity verification. Version strings may change.
How to check identity
Use Google’s published IP ranges for this crawler class and its forward-confirmed reverse DNS procedure.
Access and robots.txt
The operator documents robots.txt rules for this identifier.
Optional full-site opt-out. Merge with existing rules only if intended. robots.txt does not secure private content.
User-agent: Google-CloudVertexBot
Disallow: /Seeing 403 or 404 responses?
403 means access was denied; 404 means the resource was not found. Compare the public path, request time, edge security event and origin response to find the cause.
Separate intended restrictions and secret-file probes from pages that should work. A User-Agent name alone does not justify allowing a request.
SourcesReviewed 2026-09-14
Identity and purpose are based on these sources. Analytics interpretation and suggested checks are Apostl guidance.
APOSTL PulseSee Google-CloudVertexBot in context
See their requests. Find the pages that matter.
See your agent trafficKEEP EXPLORING
Different bots, different intent
Google-Extended
Controls content use for Gemini training and grounding; it does not control Google Search inclusion.
Content-use policyPolicy control · no separate crawlerGooglebot-Image
Fetches images for Google’s visual search features.
Web search