Semrush
SemrushBot
Collects web and link data for Semrush analysis tools.
SEO analysisCollects external links for the SEOkicks backlink database.
See your agent trafficTHE SIGNAL
Requests may originate from links on other sites rather than your navigation.
Investigate referring links before adding redirects for obsolete URLs.
Look for the SEOkicks identifier in a request’s User-Agent. This is a name match, not identity verification. Version strings may change.
A User-Agent match identifies a claimed client. Check the source IP and any operator-published verification method before granting access.
The operator documents robots.txt rules for this identifier.
Optional full-site opt-out. Merge with existing rules only if intended. robots.txt does not secure private content.
User-agent: SEOkicks
Disallow: /403 means access was denied; 404 means the resource was not found. Compare the public path, request time, edge security event and origin response to find the cause.
Separate intended restrictions and secret-file probes from pages that should work. A User-Agent name alone does not justify allowing a request.
Identity and purpose are based on these sources. Analytics interpretation and suggested checks are Apostl guidance.
See their requests. Find the pages that matter.
See your agent traffic