Why Does AI Visit Our Website? Reading Seven Days of Bot Data at 247COMPASS

An analysis of 72,578 bot requests to 247COMPASS from November 4–10, 2026. Clarity categories and official documentation distinguish search, training-related collection, user-requested fetching, and advertising checks.

A Closer Look at AI Visibility Data: Why Does AI Visit Our Website? Reading 7 Days of Bot Data at 247COMPASS — GEO, AI Crawler, AI Assistant, AI Search, and Search Engine Crawler

Websites record requests from automated programs as well as visits from people. These programs include crawlers that build search indexes, bots that collect information for AI answers, and tools that act on a user’s request. Which bots access content, for what purpose, and what do their requests mean?

247COMPASS’s Cloudflare data, connected to Microsoft Clarity, recorded 72,578 bot requests from November 4–10, 2026. The observed activity included search crawling, content collection that may support model development, user-requested page fetching, and advertising-page checks.

This article retains Clarity’s classifications and compares the recorded counts with each bot’s officially documented role. It examines request distribution and differences in function, distinguishing observations supported by the logs from citations, visits, and conversions that require additional evidence.

Scope: The data covers November 4–10, 2026, and the official reference review date is November 11, 2026. The denominator for the percentages below is 72,578 bot requests during this period, rather than all site traffic including people.

Jump to the classification and role table for all 93 bots →

First, what do these numbers count?#

Clarity’s Bot Activity uses a different measurement unit from session recordings and heatmaps. It counts bot requests using logs supplied by a CDN or server . With the Cloudflare integration, LogPush sends HTTP request logs to Clarity .

One request does not equal one person or one visit. A bot can request the same page repeatedly or fetch images, JSON, XML, and other resources alongside the page text. A request alone does not establish successful content delivery. Clarity distinguishes successful, redirected, and failed requests , but this dataset does not include counts by response status.

The direct meaning of 72,578 is therefore the volume of recorded bot access attempts. It cannot be converted into the number of AI-answer citations or human visits referred by AI services.

Seven days of bot requests in Clarity’s categories#

AI Crawler was the largest category, with 32,924 requests and a 45.36% share. AI Search followed with 14,925 requests, and AI Assistant with 12,637. Together, the three categories accounted for 60,486 requests, or 83.34% of all bot requests.

The charts and tables retain Clarity’s category names and membership. GoogleBot and BingBot are included in AI Crawler, while OAI-SearchBot is in Search Engine Crawler. Official role descriptions are compared separately and are not used to recalculate the category totals.

Bot requests by original Clarity category 72,578 bot requests from November 4–10, 2026, compared using Clarity's original categories. The denominator is bot requests rather than all site traffic. Source: 247COMPASS Clarity–Cloudflare bot operator and activity data. Original categories and membership retained.
Requests and share of all bot requests by original Clarity category
Clarity categoryRequestsShare of all bot requests
AI Crawler32,92445.36%
AI Search14,92520.56%
AI Assistant12,63717.41%
Search Engine Crawler7,0119.66%
Search Engine Optimization2,5503.51%
Advertising & Marketing2,0702.85%
Automation & Tooling2950.41%
Monitoring & Analytics830.11%
Page Preview540.07%
Security210.03%
Academic Research5Under 0.01%
Aggregator3Under 0.01%
Total72,578100%

83.34% is the combined share of the three categories with AI in their names. It does not mean that AI accounted for 83.34% of site visits or that 83.34% of the site’s content was used for AI training. A category groups records; the ultimate use of an individual request requires separate verification.

Different roles within one operator: the OpenAI example#

OpenAI was the largest operator label in the dataset, with 8,285 requests across four bots. OpenAI’s official crawler documentation distinguishes search, model development, user requests, and advertising checks.

The four bots behind OpenAI's 8,285 requests OAI-SearchBot: 2,125; ChatGPT-User: 2,101; OAI-AdsBot: 2,067; GPTBot: 1,992. Search, user-requested fetching, advertising checks, and model development are separate roles, independently of request volume. Request counts: 247COMPASS bot data · Roles: OpenAI official documentation
OpenAI bot requests, original Clarity categories, and official roles
Original bot nameRequestsClarity categoryOfficially documented role
OAI-SearchBot2,125Search Engine CrawlerCollection for ChatGPT search results
ChatGPT-User2,101AI AssistantFetching a page for a specific user-requested action
OAI-AdsBot2,067Advertising & MarketingChecking the safety and relevance of pages submitted for ChatGPT advertising
GPTBot1,992AI CrawlerCollecting content that may be used to train foundation models

The role determines what needs checking. OAI-SearchBot relates to the search-crawling route, ChatGPT-User to access on a user’s behalf, and GPTBot to policies governing training-related content use. OpenAI describes independent controls for its search and training bots.

OAI-AdsBot’s stated purpose is to check pages submitted as advertisements; its collected data is not used to train foundation models. Its 2,067 requests should be distinguished from search-visibility outcomes. The counts alone cannot establish whether 247COMPASS ran advertising. Submission history, request paths, and verification status would need to be compared.

OpenAI’s total request count cannot explain these four functions. Adding bot-specific roles to an operator total makes it possible to distinguish which function the recorded access relates to.

Role distinctions observed at Anthropic and Perplexity#

Anthropic recorded 6,225 requests, and Perplexity 4,194. Anthropic distinguishes ClaudeBot, Claude-SearchBot, and Claude-User. Perplexity distinguishes its search crawler PerplexityBot from the user-requested Perplexity-User.

Anthropic and Perplexity: requests and roles by bot
Original bot nameRequestsClarity categoryRole description
Claude2,130AI CrawlerOfficial ClaudeBot collects material for potential model training. The source display name requires comparison with the full User-Agent
Claude-SearchBot2,010AI SearchCollection to improve Claude search quality
Claude-User2,085AI AssistantFetching pages in response to user requests
PerplexityBot2,182AI SearchCollection for Perplexity search; official documentation distinguishes it from foundation-model training collection
Perplexity-User2,012AI AssistantFetching pages in response to user actions

ChatGPT-User, Claude-User, and Perplexity-User together accounted for 6,198 requests, or 8.54% of the total. This records access by bots associated with user requests. A person can generate multiple requests, and operator testing may also be included. The total does not represent 6,198 readers or an equivalent number of unsolicited recommendations.

Observed activity from search, AI search, and preview bots#

Among search crawlers, GoogleBot recorded 5,047 requests, the most of any individual bot, and BingBot recorded 2,792. Their combined 7,839 requests represented 10.80% of the total.

Clarity places both bots in AI Crawler. Separately, Google documents Googlebot’s search-crawling role , and Bing describes Bingbot’s index-crawling role . Access for a search index should be distinguished from evidence of a citation in a particular AI answer.

NAVER’s Yeti recorded 351 requests, classified by Clarity as Search Engine Crawler. NAVER requests appeared only in this category during the period. NAVER’s official guidance describes Yeti as a robot that collects web documents for search.

The following table compares the roles of major bots observed across search, AI, and public-data collection routes.

Major bot roles across search, AI, and public-data collection
Original bot nameRequestsClarity categoryOfficially described role
GoogleOther988AI CrawlerMay support general-purpose crawling and research and development by Google product teams
Amazonbot2,419AI CrawlerBroad purposes including Amazon service improvement and model training
ShapBot2,378AI SearchBuilding a search index for Parallel web APIs
Applebot2,317AI SearchSupporting Apple search experiences; collected data may also be used for AI training and answer context
DuckAssistbot2,074AI AssistantCollecting information for AI-assisted search answers; official documentation states it is not used for model training
CCBot2,032AI SearchCollection for the Common Crawl public web-crawl archive
KimiBot2,124AI CrawlerCollecting content that may be used to train Kimi foundation models
Kimi-SearchBot1,941AI SearchCollection for the Kimi search index
Yeti (NAVER)351Search Engine CrawlerCollecting web documents for NAVER search; recorded only as Search Engine Crawler during this period

Sources: Google , Amazon , Parallel , Apple , DuckDuckGo , Common Crawl , Kimi , NAVER .

AI Search is Clarity’s classification for CCBot; collecting material for a public web-crawl archive is the operator’s stated role. These request records do not reveal which models or services later used the collected material. Some official descriptions, such as Applebot’s, cover multiple uses, so one bot should not automatically be assigned a single purpose.

Official documentation describes Google-Extended and Applebot-Extended as controls over the use of collected data through robots.txt. They are not separate web crawlers. Their absence from the observed list does not establish whether the related data uses occurred.

Meta: collection, user requests, search, and previews#

Meta’s official documentation distinguishes collection, user requests, AI search, and link previews.

Reading Meta's Clarity categories alongside its official roles
Original bot nameRequestsClarity categoryRole described by Meta
Meta-ExternalAgent2,510AI CrawlerCollection for foundation-model training, product indexing, and other uses
Meta-ExternalFetcher2,007AI AssistantFetching individual links requested by users
FacebookExternalHit37Page PreviewFetching titles, descriptions, and thumbnails for shared links
Meta-WebIndexer16Page PreviewSupporting Meta AI search quality, citations, and links

Meta-WebIndexer retains its original Page Preview classification. Its officially described AI-search role appears in a separate column. Category totals preserve the supplied data, while functional descriptions are compared against operator documentation.

What 247COMPASS’s seven-day dataset establishes#

Request distribution and concentration by operator label#

OpenAI, Google, Anthropic, Meta, and Perplexity were the top five operator labels. Their combined 29,599 requests represented 40.78% of all bot requests; the other 58 operator labels accounted for 59.22%.

The source labels include general-purpose tools and libraries alongside company names. Related services can also appear under different labels, as with Moonshot AI and Kimi. The 63 labels should therefore not be counted as 63 independent companies or AI services.

Request share of the top five operator labels OpenAI, Google, Anthropic, Meta, and Perplexity accounted for 29,599 requests, or 40.78%. The remaining 58 labels accounted for 59.22%. These are request shares, rather than company counts or AI market shares. Source: 247COMPASS bot data. Individual bots and operator totals are not double-counted.
Requests from the top five operator labels and all remaining labels
Operator labelRequestsShare of all bot requests
OpenAI8,28511.42%
Google6,3258.71%
Anthropic6,2258.58%
Meta4,5706.30%
Perplexity4,1945.78%
Other 58 operator labels42,97959.22%

Recorded requests are distributed across many operator labels, rather than concentrated entirely in one or two. Operational checks should compare requested URLs and response status across major search and AI routes. These shares do not measure platform market share or preference for 247COMPASS.

Distinguishing training, search, and user-requested access#

Different bot roles call for different operational checks:

  • Search access: whether public pages can be crawled, and error responses on important pages.
  • Training-related access: content-use policies and operator-specific controls.
  • User-requested access: availability of the requested information and the behavior of access controls.
  • Preview, SEO audit, and automation tools: accuracy of shared information and the operational impact of repeated requests.

ClarityBot/, classified as Search Engine Optimization, recorded 850 requests. seoClarity’s documentation describes a bot that performs customer-requested site audits; it is separate from Microsoft Clarity’s analytics tool. Semrush bots support backlink graphs, site audits, and page checks, while Majestic’s MJ12Bot collects links between sites. These records do not identify who commissioned an analysis.

General-purpose tool names such as curl, python-requests, and Go-http-client do not identify the actual operator or purpose. A name alone also cannot establish whether a request is useful or malicious. Where sender-verification methods are available, such as Google’s crawler verification guidance or Bing’s verification tool , they can be compared with the actual logs.

A single-period distribution and a trend are different findings#

The dataset contains one seven-day aggregate. Without a previous-week comparison or daily counts, it cannot verify claims that AI bots surged, that crawling increased after a new article, or that AI bots replaced search crawlers.

Without requested URLs or language breakdowns, collection volume cannot be attributed to individual articles. Without status codes and response sizes, collection success rates and server load cannot be calculated. Similar request counts across bots do not establish equal demand. Observed access distribution should be distinguished from trends and outcomes that require additional observation.

After the request: responses, citations, and visits#

247COMPASS’s seven-day record observes content access associated with operator and service functions. It includes automated collection and bot access that fulfills user requests. Whether those requests resulted in content delivery, AI-answer citations, or human visits must be checked using separate evidence.

Bot logs can help investigate errors in the content access and delivery process relevant to GEO. Analytics data classified as user traffic observes the content visitors reach and the choices and actions that follow. Treating direct verification of AI-answer citations as a separate step clarifies what each dataset measures.

Content delivery and selection: three distinct observations Bot requests, answer citations, and user visits require their own evidence.

1. Content access by a service

Bot traffic · Cloudflare → Clarity

Which bot requested the content, and was delivery successful?

  • Operator purposes: collection, search, and handling user requests
  • Requesting bot, URL, and timestamp
  • Response status, blocks, and repeated requests

Check errors along the content delivery route

2. Citations in AI answers

Direct observation of actual answers and sources

Which content appeared as a source in an actual answer?

  • Source URL displayed in the answer
  • Query, service, and observation date
  • Citation context and link format

Verify actual citations and their context

3. User selection and visits

User traffic · Analytics

Which content did visitors reach, and what did they do next?

  • Visited pages and identifiable referral sources
  • Sessions and configured behavior events
  • Measured conversions such as inquiries and downloads

Observe selected content and behavior after arrival

Bot requests, citations, and user visits are different metrics. Bot requests alone do not establish citations, and Analytics visits alone do not verify an actual citation. Compare identifiable AI-service referrals with the sources shown in actual answers.

Referral sources and landing pages in Analytics are distinguished using the definitions in Google’s traffic acquisition report and landing page report . Actual citations require direct observation and comparison of the answer and its sources.

Requested URLs and response status would allow checks of where major bots encountered errors or access restrictions. These detailed logs are absent from the seven-day aggregate, so this article does not calculate an error rate or assert that delivery failed on a particular page. It likewise does not estimate separate Analytics visit or conversion counts.

This also relates to the difference between AI-search visibility and actual referrals . Bot statistics describe routes of content access. Evaluating what information reached readers and what they did requires additional observation of cited pages, referral sources, and behavior after arrival.

Data and interpretation criteria#

  • Source: bot operator and activity data from 247COMPASS’s Cloudflare integration with Microsoft Clarity.
  • Period: November 4–10, 2026.
  • Official reference review date: November 11, 2026.
  • Coverage: 93 bot names, 63 operator labels, and 72,578 requests.
  • Percentage denominator: all 72,578 bot requests. Operator and individual-bot totals are not double-counted; percentages are rounded to two decimal places.
  • Role evidence: official operator documentation. Uncertain name mappings and roles are identified separately in the appendix.
  • Outside the dataset: individual response outcomes, actual AI-answer citations, and user visits or conversions.

Appendix: observed bots, classifications, and roles#

The directory compares bot request counts, Clarity categories, official roles, and the supporting evidence.

Classifications and roles of all 93 botsRequests by bot · Clarity category · Official roles and sources Open the full directoryClose the directory
Bot requests, Clarity categories, and roles: November 4–10, 2026
Bot name / Operator labelRequestsClarity categoryRole description and evidence
OAI-SearchBot
OpenAI
2,125Search Engine CrawlerCollection for ChatGPT search results. Requests alone do not establish citations or visibility.
OpenAI Crawlers · Official description and display-name comparison
ChatGPT-User
OpenAI
2,101AI AssistantPage fetching for a specific user request. It is not for automated web crawling and does not count users.
OpenAI Crawlers · Official description and display-name comparison
OAI-AdsBot
OpenAI
2,067Advertising & MarketingSafety and relevance checks on pages submitted for ChatGPT advertising. Not for foundation-model training. Submission history, IPs, and paths require comparison.
OpenAI Crawlers · Official description and display-name comparison
GPTBot
OpenAI
1,992AI CrawlerCollecting content that may be used for foundation-model training. Inclusion of any individual request in training is unverified.
OpenAI Crawlers · Official description and display-name comparison
GoogleBot
Google
5,047AI CrawlerCrawler for Google Search. The original AI Crawler classification is retained; these requests are not established as training-related.
Google common crawlers · Official description and display-name comparison
GoogleOther
Google
988AI CrawlerMay support general-purpose collection and internal research and development by Google product teams. The purpose of individual requests is unverified.
Google common crawlers · Official description and display-name comparison
Chrome-Lighthouse
Google
177Search Engine OptimizationClassified as SEO in the source. Bot-specific audit, backlink, and other operations have not yet been checked against operator documentation in this research.
Classification in supplied data · Detailed official role not yet verified
Google-Agent
Google
78AI AssistantClassified as AI Assistant in the source. The name alone does not establish an individual user request or a person count.
Classification in supplied data · Detailed official role not yet verified
Google Images
Google
22Search Engine CrawlerDisplay name associated with Google image search. Mapping to the official Googlebot-Image identifier requires comparison of the full User-Agent.
Google common crawlers · Official description and display-name comparison
HeadlessChrome
Google
10Automation & ToolingClassified as automation and tooling in the source. A general-purpose client name alone cannot identify the actual operator or purpose.
Classification in supplied data · Detailed official role not yet verified
Google-Apps-Script
Google
1Search Engine OptimizationClassified as SEO in the source. Bot-specific audit, backlink, and other operations have not yet been checked against operator documentation in this research.
Classification in supplied data · Detailed official role not yet verified
Google Site Verification
Google
1SecurityClassified as security in the source. The record does not establish a vulnerability or an attack.
Classification in supplied data · Detailed official role not yet verified
Google Image Proxy
Google
1Page PreviewClassified as page preview in the source. Exact behavior requires further comparison with official documentation.
Classification in supplied data · Detailed official role not yet verified
Claude
Anthropic
2,130AI CrawlerOfficial documentation describes ClaudeBot as collecting for potential model training. Mapping the source label Claude to the exact User-Agent needs verification.
Anthropic crawler controls · Official description and display-name comparison
Claude-User
Anthropic
2,085AI AssistantPage fetching in response to user requests. Cannot be converted into numbers of people or recommendations.
Anthropic crawler controls · Official description and display-name comparison
Claude-SearchBot
Anthropic
2,010AI SearchCollection to improve Claude search quality.
Anthropic crawler controls · Official description and display-name comparison
Meta-ExternalAgent
Meta
2,510AI CrawlerCollection for foundation-model training, product indexing, and other purposes. Not restricted to a single use.
Meta Web Crawlers · Official description and display-name comparison
Meta-ExternalFetcher
Meta
2,007AI AssistantFetches individual user-requested links to support product features.
Meta Web Crawlers · Official description and display-name comparison
FacebookExternalHit
Meta
37Page PreviewPreview fetching of titles, descriptions, and thumbnails for links shared in Meta apps.
Meta Web Crawlers · Official description and display-name comparison
Meta-WebIndexer
Meta
16Page PreviewSupports Meta AI search quality, citations, and links. The original Page Preview category is not reassigned to AI Search.
Meta Web Crawlers · Official description and display-name comparison
PerplexityBot
Perplexity
2,182AI SearchCollection for Perplexity search, distinguished from collection for foundation-model training.
Perplexity crawlers · Official description and display-name comparison
Perplexity-User
Perplexity
2,012AI AssistantFetching triggered by user actions. Not described as automated crawling or training collection.
Perplexity crawlers · Official description and display-name comparison
KimiBot
Moonshot AI
2,124AI CrawlerCollecting content that may be used to train Kimi foundation models.
Kimi Crawlers · Official description and display-name comparison
Kimi-SearchBot
Moonshot AI
1,941AI SearchCollection for the Kimi search index.
Kimi Crawlers · Official description and display-name comparison
BingBot
Microsoft
2,792AI CrawlerCollection of new and updated pages for the Bing index. Records alone do not establish Copilot citations.
Bing crawlers · Official description and display-name comparison
Amazonbot
Amazon
2,419AI CrawlerMay support Amazon service improvement, accurate information, and model training. Not to be confused with the separate Amzn-SearchBot or Amzn-User.
Amazon web search crawlers · Official description and display-name comparison
ShapBot
Parallel
2,378AI SearchBuilds a search index for Parallel web APIs. Signature and IP verification methods are published.
Parallel ShapBot · Official description and display-name comparison
Applebot
Apple
2,317AI SearchSupports search experiences including Spotlight, Siri, and Safari. Data may be used for AI training and answer context; individual request purposes remain unverified.
About Applebot · Official description and display-name comparison
ChatGLM-Spider
Other Operators
2,027AI CrawlerOperator and category labels are retained from the source platform. This research did not find first-party role and control documentation for this exact name. Inclusion in training cannot be established.
Cloudflare bot reference · Official description and display-name comparison
Operator
Other Operators
203AI AssistantClassified as AI Assistant in the source. The name alone does not establish an individual user request or a person count.
Classification in supplied data · Detailed official role not yet verified
ExaUser
Other Operators
39AI AssistantClassified as AI Assistant in the source. The name alone does not establish an individual user request or a person count.
Classification in supplied data · Detailed official role not yet verified
Google-NotebookLM
Other Operators
20AI AssistantClassified as AI Assistant in the source. The name alone does not establish an individual user request or a person count.
Classification in supplied data · Detailed official role not yet verified
Baiduspider
Baidu
2,097Search Engine CrawlerClassified as search collection in the source. The service-specific official purpose and User-Agent verification require further checking.
Classification in supplied data · Detailed official role not yet verified
DuckAssistbot
DuckDuckGo
2,074AI AssistantReal-time information collection for AI-assisted search answers, not model training. The original AI Assistant category is retained.
DuckAssistBot · Official description and display-name comparison
DuckDuckBot
DuckDuckGo
14Search Engine CrawlerCollection for DuckDuckGo search, distinct from DuckAssistBot for AI answers.
DuckDuckBot · Official description and display-name comparison
Hunyuan
Tencent
2,076AI CrawlerOfficial HySpider is described as potential training collection, but its mapping to the source label Hunyuan has not been verified.
Tencent Hunyuan crawler · Official description and display-name comparison
YouBot
You.com
2,074Search Engine CrawlerClassified as search collection in the source. The service-specific official purpose and User-Agent verification require further checking.
Classification in supplied data · Detailed official role not yet verified
MoonshotBot
Kimi
2,049AI CrawlerThe source operator label is Kimi. The official Kimi crawler documentation did not verify the role or mapping of this exact name. It is not merged with KimiBot.
Kimi Crawlers · Official description and display-name comparison
CCBot
Common Crawl Foundation
2,032AI SearchCollection for the Common Crawl public web-crawl archive. The original AI Search category is retained. This does not prove specific model training or direct citations.
Common Crawl CCBot · Official description and display-name comparison
QwenBot
Alibaba
2,024AI CrawlerOperator and category labels are retained from the source platform. This research did not find first-party role and control documentation for this exact name. Inclusion in training cannot be established.
Cloudflare bot reference · Official description and display-name comparison
cohere-ai
Cohere
2,017AI AssistantCurrent official documentation states that Cohere does not crawl with Cohere bots for foundation-model training. The display name alone does not establish the actual role.
Cohere web crawlers · Official description and display-name comparison
DeepSeekBot
DeepSeek
2,006AI CrawlerOfficial material describes DeepseekBot collection for model training and source cross-checking. Full User-Agent and IP authentication need further verification.
DeepSeek training-content summary · Official description and display-name comparison
Bravebot
Brave Software, Inc.
2,002AI SearchOfficial documentation states that the search crawler does not use a dedicated identifying User-Agent. The actual sender and role behind this label need verification.
Brave Search crawler · Official description and display-name comparison
Bytespider
ByteDance
2,000AI CrawlerOperator and category labels are retained from the source platform. This research did not find first-party role and control documentation for this exact name. Inclusion in training cannot be established.
Cloudflare bot reference · Official description and display-name comparison
ClarityBot/
Seoclarity
850Search Engine OptimizationPage and site audits requested or scheduled by seoClarity customers, separate from Microsoft Clarity analytics. This table does not identify the requesting party.
seoClarity ClarityBot · Official description and display-name comparison
SemrushBotBacklinks
Semrush
774Search Engine OptimizationOfficial SemrushBot collects a link graph for backlink analysis. The source combined display name needs comparison with the exact User-Agent.
Semrush Bot · Official description and display-name comparison
SemrushBotBacklinkAudit
Semrush
8Search Engine OptimizationOfficial SemrushBot-BA serves the Backlink Audit tool. The source display name and exact User-Agent need comparison.
Semrush Bot · Official description and display-name comparison
SiteAuditBot
Semrush
3Search Engine OptimizationSite audit bot for identifying technical and SEO issues.
Semrush Bot · Official description and display-name comparison
SemrushBotSI
Semrush
3Search Engine OptimizationOfficial SemrushBot-SI supports tools including On Page SEO Checker. Mapping to the source display name needs comparison.
Semrush Bot · Official description and display-name comparison
PetalBot
Huawei
635AI CrawlerOfficial search excerpts describe Petal Search indexing, Huawei recommendations, and AI Search support. The complete official text was not accessible. The original AI Crawler category is retained.
PetalBot (Official search excerpt only) · Official description and display-name comparison
Go-http-client
Golang
219Automation & ToolingClassified as automation and tooling in the source. A general-purpose client name alone cannot identify the actual operator or purpose.
Classification in supplied data · Detailed official role not yet verified
YandexBot
Yandex
216Search Engine CrawlerCrawler for the main Yandex search index.
Yandex robots verification · Official description and display-name comparison
SERankingBacklinksBot
SE Ranking
133Search Engine OptimizationCollects public links to support link-structure analysis.
SE Ranking Backlinks crawler · Official description and display-name comparison
SE Ranking Bot
SE Ranking
18Monitoring & AnalyticsOfficial SEBot-WA supports site audits and On-Page SEO checks. Mapping to this display name needs verification. The original Monitoring & Analytics category is retained.
SE Ranking crawler FAQ · Official description and display-name comparison
DataForSEO Bot
DataForSEO
139Search Engine OptimizationStores discovered links for backlink analysis. The official DataForSeoBot User-Agent needs comparison with the source display name.
DataForSEO Link Bot · Official description and display-name comparison
DataForSEO
DataForSEO
3Search Engine OptimizationOfficial documentation describes DataForSEO backlink collection, but the exact User-Agent and role of this standalone display name need verification.
DataForSEO Link Bot · Official description and display-name comparison
MJ12Bot
Majestic
133Search Engine OptimizationCollects links between sites to build the Majestic backlink index.
Majestic MJ12Bot · Official description and display-name comparison
AhrefsBot
Ahrefs
102Search Engine OptimizationOfficial search excerpts describe recording external links in a database and recrawling periodically. Only excerpts were accessible; the official full text was restricted.
AhrefsBot (Official search excerpt only) · Official description and display-name comparison
Barkrowler
Babbar
96Search Engine OptimizationClassified as SEO in the source. Bot-specific audit, backlink, and other operations have not yet been checked against operator documentation in this research.
Classification in supplied data · Detailed official role not yet verified
Chrome Privacy Preserving Prefetch Proxy
Datadome
88Search Engine OptimizationClassified as SEO in the source. Bot-specific audit, backlink, and other operations have not yet been checked against operator documentation in this research.
Classification in supplied data · Detailed official role not yet verified
NetAPI
Datadome
4Search Engine OptimizationClassified as SEO in the source. Bot-specific audit, backlink, and other operations have not yet been checked against operator documentation in this research.
Classification in supplied data · Detailed official role not yet verified
HappyWing
Datadome
3Search Engine OptimizationClassified as SEO in the source. Bot-specific audit, backlink, and other operations have not yet been checked against operator documentation in this research.
Classification in supplied data · Detailed official role not yet verified
Lightpanda
Grok-adjacent
85AI CrawlerClassified as AI Crawler in the source. The exact bot-specific role and ultimate use require further official documentation and original-request verification.
Classification in supplied data · Detailed official role not yet verified
nbot
NBot
63AI SearchClassified as AI Search in the source. Exact indexing and answer-use behavior remain unverified.
Classification in supplied data · Detailed official role not yet verified
KeenableBot
Keenable
41Search Engine CrawlerClassified as search collection in the source. The service-specific official purpose and User-Agent verification require further checking.
Classification in supplied data · Detailed official role not yet verified
Keenable-User
Keenable
1AI AssistantClassified as AI Assistant in the source. The name alone does not establish an individual user request or a person count.
Classification in supplied data · Detailed official role not yet verified
HubSpot Crawler
HubSpot
37Monitoring & AnalyticsClassified as monitoring and analytics in the source. The detailed request purpose remains unverified.
Classification in supplied data · Detailed official role not yet verified
curl
Haxx
34Automation & ToolingClassified as automation and tooling in the source. A general-purpose client name alone cannot identify the actual operator or purpose.
Classification in supplied data · Detailed official role not yet verified
ibou-search
ibou
27Search Engine CrawlerClassified as search collection in the source. The service-specific official purpose and User-Agent verification require further checking.
Classification in supplied data · Detailed official role not yet verified
HaloBot
Haloscan
26Monitoring & AnalyticsClassified as monitoring and analytics in the source. The detailed request purpose remains unverified.
Classification in supplied data · Detailed official role not yet verified
Moz dotbot
Moz
24Search Engine OptimizationClassified as SEO in the source. Bot-specific audit, backlink, and other operations have not yet been checked against operator documentation in this research.
Classification in supplied data · Detailed official role not yet verified
CensysInspect/
Censys
20SecurityClassified as security in the source. The record does not establish a vulnerability or an attack.
Classification in supplied data · Detailed official role not yet verified
Reflectionbot
Reflection AI
20AI CrawlerClassified as AI Crawler in the source. The exact bot-specific role and ultimate use require further official documentation and original-request verification.
Classification in supplied data · Detailed official role not yet verified
python-requests
python-requests
13Automation & ToolingClassified as automation and tooling in the source. A general-purpose client name alone cannot identify the actual operator or purpose.
Classification in supplied data · Detailed official role not yet verified
Cốc Cốc
Coccoc
12Search Engine CrawlerClassified as search collection in the source. The service-specific official purpose and User-Agent verification require further checking.
Classification in supplied data · Detailed official role not yet verified
Qwantbot
Qwant
11Search Engine CrawlerClassified as search collection in the source. The service-specific official purpose and User-Agent verification require further checking.
Classification in supplied data · Detailed official role not yet verified
SofyaBot
Sofya
9Search Engine CrawlerClassified as search collection in the source. The service-specific official purpose and User-Agent verification require further checking.
Classification in supplied data · Detailed official role not yet verified
ALittle Client
Udger
9Automation & ToolingClassified as automation and tooling in the source. A general-purpose client name alone cannot identify the actual operator or purpose.
Classification in supplied data · Detailed official role not yet verified
ZoomInfo
ZoomInfo
8Search Engine CrawlerClassified as search collection in the source. The service-specific official purpose and User-Agent verification require further checking.
Classification in supplied data · Detailed official role not yet verified
ICBot/
Ideasandcode
5Search Engine OptimizationClassified as SEO in the source. Bot-specific audit, backlink, and other operations have not yet been checked against operator documentation in this research.
Classification in supplied data · Detailed official role not yet verified
TurnitinBot
Turnitin
4Academic ResearchClassified as academic research in the source. The specific research project and actual use remain unverified.
Classification in supplied data · Detailed official role not yet verified
YisouSpider
Yisou
4Search Engine CrawlerClassified as search collection in the source. The service-specific official purpose and User-Agent verification require further checking.
Classification in supplied data · Detailed official role not yet verified
httpx
Python-httpx
4Automation & ToolingClassified as automation and tooling in the source. A general-purpose client name alone cannot identify the actual operator or purpose.
Classification in supplied data · Detailed official role not yet verified
GeedoShopProductFinder
Geedo SIA
3AggregatorClassified as an aggregator in the source. The exact collection purpose remains unverified.
Classification in supplied data · Detailed official role not yet verified
BrightEdge Bot
BrightEdge
3Advertising & MarketingClassified as advertising and marketing in the source. This does not establish an advertising campaign or its performance.
Classification in supplied data · Detailed official role not yet verified
Python-urllib
Python-urllib
3Automation & ToolingClassified as automation and tooling in the source. A general-purpose client name alone cannot identify the actual operator or purpose.
Classification in supplied data · Detailed official role not yet verified
okhttp
okhttp
2Automation & ToolingClassified as automation and tooling in the source. A general-purpose client name alone cannot identify the actual operator or purpose.
Classification in supplied data · Detailed official role not yet verified
Wordpress Namecheap
Namecheap
2Monitoring & AnalyticsClassified as monitoring and analytics in the source. The detailed request purpose remains unverified.
Classification in supplied data · Detailed official role not yet verified
Observer
Obsrvr
2Search Engine OptimizationClassified as SEO in the source. Bot-specific audit, backlink, and other operations have not yet been checked against operator documentation in this research.
Classification in supplied data · Detailed official role not yet verified
SeobilityBot
Seobility
2Search Engine OptimizationClassified as SEO in the source. Bot-specific audit, backlink, and other operations have not yet been checked against operator documentation in this research.
Classification in supplied data · Detailed official role not yet verified
trafilatura
Github
1Automation & ToolingClassified as automation and tooling in the source. A general-purpose client name alone cannot identify the actual operator or purpose.
Classification in supplied data · Detailed official role not yet verified
ResearchBot
Kaust
1Academic ResearchClassified as academic research in the source. The specific research project and actual use remain unverified.
Classification in supplied data · Detailed official role not yet verified
Yeti
NAVER
351Search Engine CrawlerCollection of web documents for NAVER search. All 351 NAVER requests in this period were recorded only in Search Engine Crawler.
Clarity counts · Official NAVER guidance

The 247COMPASS perspective#

AI citations and search visibility start with a technical foundation that allows bots to access and collect our content. After a bot’s activity, actual visibility and citations still depend on each platform’s indexing, algorithms, and policies.

This is why Technical SEO matters. Content sits on that technical foundation. Think of a pizza: Technical is the dough, and content is the topping.

Is your dough ready for great toppings?

Author


Founder & CEO

Founder and CEO of 247COMPASS. From content marketing to SEO, brand content strategy and media planning, he helps brands understand and strengthen their connections with customers.

Next step

Let’s talk about your next direction

Connect with 247COMPASS for search, content and media strategy.