My prompts
Do not paste names, addresses, passwords, health details, legal problems, children’s information or confidential work data unless the tool and account are approved for that use.
Private person guide
This is for normal people first: your blog, photos, CV, family documents, private notes, school work, client files, social posts and prompts. Learn what you can block, what you can only reduce, and what you should never upload without thinking.
Quick answer
The safest move is not only a crawler block. It is knowing what not to publish, what not to paste, which account settings to check and when a file should stay out of AI tools completely.
Do not paste names, addresses, passwords, health details, legal problems, children’s information or confidential work data unless the tool and account are approved for that use.
Redact CVs, contracts, bank files, medical letters, school documents and client PDFs. If the data belongs to someone else, treat it as higher risk.
Public photos can contain faces, locations, names, license plates and private events. Keep sensitive albums private and think before uploading images to AI tools.
If you run a blog or portfolio, robots.txt and server rules can request that compliant AI crawlers stay away. They do not erase old copies.
Private person checklist
Ask these questions before uploading or pasting anything. If one answer feels uncomfortable, use placeholders, remove the file, or use a more controlled account.
Does this identify me? Name, address, phone, email, signature, birthday, passport number, face, voice or exact location.
Does it identify someone else? Children, partner, customer, patient, student, colleague, tenant, employee or client.
Would I publish it online? If no, do not treat a random AI prompt as a safe private notebook.
Can I replace details? Use “Person A”, “Company B”, fake dates, broad locations and shortened excerpts when the exact data is not needed.
Do I know the account settings? Consumer, team, API and enterprise accounts may handle data differently. Check current settings.
Technical layers
Use robots.txt, server logs, firewall rules and a visible content policy. This reduces compliant crawling but does not remove old copies or stop bad actors.
Use the provider’s data controls, chat history settings, team plans, API terms, retention options and opt-outs. This is separate from crawler blocking.
Use client permission, company policy, redaction, approved tools and access control before uploading customer files, HR records or confidential documents.
Decision map
Start with the place where your data becomes available. A prompt, public blog, private PDF, photo album, cloud drive and work document are different risk surfaces.
A useful privacy habit is boring but powerful: upload less, redact more, keep private albums private, check settings and use crawler blocks only where they fit.
Crawler matrix
| User agent | Company | Why it matters | Careful note |
|---|---|---|---|
GPTBot | OpenAI | Commonly used for OpenAI web crawling controls. | Block if you do not want this crawler to fetch your public pages for AI-related use. Verify current OpenAI docs. |
ChatGPT-User | OpenAI | User-triggered browsing/fetching agent in many discussions. | Blocking can stop ChatGPT users from asking it to read your page. This is not the same intent as broad training crawling. |
Google-Extended | A control Google documents for some generative AI use cases. | This is not the same as blocking Google Search. Do not block Googlebot unless you want search visibility affected. | |
ClaudeBot | Anthropic | Anthropic crawler name seen in crawler-control discussions. | Use only after checking Anthropic’s latest documentation. |
Claude-SearchBot | Anthropic | Search-related Claude crawler name seen in public guidance. | Consider the visibility tradeoff before blocking search-style agents. |
PerplexityBot | Perplexity | Perplexity crawler/user-agent discussed by publishers. | Blocking may affect how your content is discovered or cited by Perplexity-style answer engines. |
CCBot | Common Crawl | Public web crawler used for large web datasets. | Often blocked by publishers who do not want pages copied into public crawl corpora. |
Bytespider | ByteDance | Crawler associated with ByteDance. | Useful to review if you want a broad AI-crawler blocklist. |
Static tool
Select the crawlers you want to request not to crawl. This generator runs in your browser only. It does not send data anywhere. Before deploying, verify the current official user-agent names.
# AI crawler controls: verify current user-agent names before deploying.
# This is voluntary. robots.txt is not a legal lock.
User-agent: GPTBot
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: ClaudeBot
Disallow: /
User-agent: PerplexityBot
Disallow: /
User-agent: CCBot
Disallow: /Warning: blocking user-triggered or search-related agents can reduce AI-assisted discovery, previews or citations.
Official references
Check the current OpenAI crawler and user-agent guidance.
Google crawler docs developers.google.com/search/docs/crawling-indexing/overview-google-crawlersVerify Google crawler names, indexing behavior and AI-related controls.
Anthropic support support.anthropic.com/Check current Claude account, data and crawler guidance.
Perplexity www.perplexity.ai/Review current Perplexity product and publisher information.
Common Crawl commoncrawl.org/Understand one major public web crawl dataset used in research.
Robots Exclusion Protocol www.rfc-editor.org/rfc/rfc9309The formal robots.txt protocol reference.
Related guides
Compare AI models before choosing a workflow.
ChatGPT Alternatives chatgpt-alternatives.comFind neutral alternatives when one AI provider is not enough.
AI Subscription Guide aisubscriptionguide.comLearn which AI subscription fits your real work.
KIWerkzeuge kiwerkzeuge.comGerman AI tool directory and privacy-focused AI tool guides.
BoiteAIA boiteaia.comFrench AI tool directory for chatbots, image tools and business use.
MultipleChat AI multiplechat.aiUse several AI models in one interface and keep workflows separated by project.
FAQ
You can reduce exposure, but you cannot guarantee full control once photos are public. Keep sensitive photos private, review platform settings, avoid public high-resolution uploads and use crawler controls for your own website.
Only after removing details you do not need for the task. Names, addresses, phone numbers, birth dates, signatures, employer secrets and references can often be replaced with placeholders.
Be careful. Medical letters, passports, school documents, legal papers, bank files and family records can contain sensitive data about other people. Use approved privacy settings or do not upload them.
It depends on the product, account type and settings. Some providers offer controls, team plans or API terms that treat data differently. Check the current settings before typing sensitive information.
Not with one technical switch. robots.txt can request that compliant crawlers stay away, but it is voluntary and does not remove copies that already exist elsewhere.
robots.txt is a technical convention, not a contract by itself. It can support your policy position, but legal questions depend on jurisdiction, terms, copyright, contracts and enforcement.
Blocking AI-specific agents should not be treated the same as blocking Googlebot. If you block normal search crawlers, search visibility can be harmed. Check each user agent carefully.
Only if you understand the tradeoff. User-triggered agents may fetch pages when a person asks an AI tool to read your site. Blocking them can reduce AI-assisted discovery.