Semrush
SemrushBot
Collects web and link data for Semrush analysis tools.
SEO analysisSEO analysis
SemrushBot-SI
Crawls for Semrush On Page SEO Checker.
Operated by Semrush
See your agent traffic PurposeSEO analysis
HTTP identifierSemrushBot-SI
IdentityVerify beyond the name
THE SIGNAL
What this visit tells you
Page checks may follow a configured optimization project.
Crawled URLs
Broken destinations
Crawl duration
Your next move
Compare the crawled URL with the page selected in the checker.
The technical details
MarkdownHow to identify it
Look for the SemrushBot-SI identifier in a request’s User-Agent. This is a name match, not identity verification. Version strings may change.
How to check identity
A User-Agent match identifies a claimed client. Check the source IP and any operator-published verification method before granting access.
Access and robots.txt
The operator documents robots.txt rules for this identifier.
Optional full-site opt-out. Merge with existing rules only if intended. robots.txt does not secure private content.
User-agent: SemrushBot-SI
Disallow: /Seeing 403 or 404 responses?
403 means access was denied; 404 means the resource was not found. Compare the public path, request time, edge security event and origin response to find the cause.
Separate intended restrictions and secret-file probes from pages that should work. A User-Agent name alone does not justify allowing a request.
SourcesReviewed 2026-09-14
Identity and purpose are based on these sources. Analytics interpretation and suggested checks are Apostl guidance.
APOSTL PulseSee SemrushBot-SI in context
See their requests. Find the pages that matter.
See your agent trafficKEEP EXPLORING
Different bots, different intent
Semrush
SiteAuditBot
Checks technical SEO in Semrush Site Audit.
SEO analysisSemrush
SemrushBot-BA
Crawls for Semrush Backlink Audit.
SEO analysis