GPTBot vs ChatGPT-User vs OAI-SearchBot: OpenAI's Three Crawlers, Explained

"Should I block GPTBot?" is the wrong question, because OpenAI doesn't run one bot. It runs three, with three different jobs - and the robots.txt decision that's right for one is often wrong for the others. OpenAI's own bot documentation spells out the split, but almost nobody reads it before writing blocking rules.

Here's what GPTBot, ChatGPT-User, and OAI-SearchBot each actually do, how to tell them apart in your traffic, and what blocking each one really costs.

The three OpenAI bots at a glance

The key mental model: GPTBot is about the future, ChatGPT-User is about right now, and OAI-SearchBot is about being findable. They're separate user-agents with separate robots.txt tokens precisely so you can treat them differently.

GPTBot: the training crawler

GPTBot is the one that made headlines. It reads public web pages and the content may be used to train OpenAI's models. It's also the heaviest of the three by volume - Vercel measured GPTBot at 569 million requests per month across its network back in 2024, and training crawling has only grown since: Cloudflare's 2026 breakdown puts training at nearly half of all AI bot activity.

Blocking GPTBot is a legitimate choice - publishers who don't want their content in training data do it deliberately. Just be clear about the trade: models trained after your block may know less about your product, your docs, and your point of view.

ChatGPT-User: a real person is asking about you

ChatGPT-User is the most misunderstood - and, for most businesses, the most valuable. It fires when a live ChatGPT conversation needs your page: someone asks "what does this tool cost?" and ChatGPT fetches your pricing page to answer.

That means every ChatGPT-User visit in your logs is a real user, mid-question, with your brand already in play. When we tracked our own site's bot visits, these were the hits we cared about most - they're as close as bot traffic gets to a warm lead.

Blocking ChatGPT-User while wanting AI visibility is self-sabotage: ChatGPT will simply answer from what it remembers (which may be stale) or from your competitor's page (which is worse).

OAI-SearchBot: the index behind ChatGPT search

OAI-SearchBot crawls to build and refresh the index that ChatGPT's search feature draws from. It's the closest analog to Googlebot in OpenAI's lineup: it doesn't train models, it makes you findable.

If ChatGPT search matters to your acquisition at all, this is the last bot you want to block. Google's own robots.txt documentation applies the same way here: rules are per user-agent token, so you can allow OAI-SearchBot explicitly even while disallowing GPTBot.

A robots.txt that treats them differently

Most sites that think about this end up somewhere like:

# Allow live answering and search indexing
User-agent: ChatGPT-User
Allow: /

User-agent: OAI-SearchBot
Allow: /

# Opt out of training only
User-agent: GPTBot
Disallow: /

That keeps you visible in live answers and ChatGPT search while opting out of training. Generate and validate yours with our robots.txt tool if you're editing by hand.

Common mistakes

Blocking "GPTBot" and assuming ChatGPT can't see you. The three bots are independent. A GPTBot disallow does nothing to ChatGPT-User or OAI-SearchBot - and vice versa.

Judging by robots.txt instead of actual traffic. robots.txt is a request, and it only tells you your policy - not what's actually visiting. The bot traffic tracker shows which OpenAI bots hit your site, which pages they read, and how often - the ground truth your policy should be based on.

Forgetting the other vendors. Anthropic, Perplexity, Google, and ByteDance each run their own crawlers with their own tokens. The same allow-answering/block-training logic applies, per vendor - the 2026 numbers on who crawls how much are a good place to calibrate.

FAQ

Does blocking GPTBot remove my content from ChatGPT? No. It stops future training collection. ChatGPT can still reference you via live fetching (ChatGPT-User) and search (OAI-SearchBot), and existing models keep whatever they already learned.

How do I know which OpenAI bot visited my site? Each sends a distinct user-agent string containing its name. You'll see them in raw server logs, or classified automatically with category labels in DidYouSEO's bot traffic dashboard.

Do these bots respect robots.txt? OpenAI states all three honor robots.txt rules for their specific user-agent tokens. Changes can take up to 24 hours to be picked up, and robots.txt doesn't stop bad actors impersonating these bots - verification does.

Is ChatGPT-User traffic counted in Google Analytics? No. Like all bots, it fetches raw HTML without executing JavaScript, so browser-based analytics never records it. Only server logs or server-side tracking can see it.