{"id":6932,"date":"2026-07-20T23:03:20","date_gmt":"2026-07-20T23:03:20","guid":{"rendered":"https:\/\/www.voctos.com\/?p=6932"},"modified":"2026-09-06T12:52:52","modified_gmt":"2026-09-06T12:52:52","slug":"robots-txt-ai-crawlers","status":"publish","type":"post","link":"https:\/\/www.voctos.com\/ar\/blog\/robots-txt-ai-crawlers\/","title":{"rendered":"robots.txt \u0644\u0632\u0648\u0627\u062d\u0641 \u0627\u0644\u0630\u0643\u0627\u0621 \u0627\u0644\u0627\u0635\u0637\u0646\u0627\u0639\u064a: \u0623\u064a \u0627\u0644\u0631\u0648\u0628\u0648\u062a\u0627\u062a \u062a\u0633\u0645\u062d \u0628\u0647\u0627 \u0623\u0648 \u062a\u062d\u0638\u0631\u0647\u0627 (2026)"},"content":{"rendered":"\n<div style=\"font:400 13px\/1.5 Arial,sans-serif;color:#98A2AC;margin:0 0 14px;\"><a href=\"https:\/\/www.voctos.com\/\" style=\"color:#98A2AC;text-decoration:none;\">VOCTOS<\/a> &rsaquo; <a href=\"https:\/\/www.voctos.com\/blog\/technical-seo-guide\/\" style=\"color:#4B4DE8;text-decoration:none;\">The Technical SEO Guide<\/a> &rsaquo; <span>This guide<\/span><\/div>\n\n\n\n<p class=\"lede wp-block-paragraph\">The single decision that matters in your robots.txt is the difference between AI training crawlers and AI search crawlers. Block a training crawler and you protect your content at no cost to your search visibility. Block a search crawler and you quietly delete yourself from that engine&#8217;s AI answers. Never block Googlebot, and remember that robots.txt is a voluntary signal, not a wall. This guide gives the full list, the trade-off, and copy-paste configs.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Key takeaways<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>There are three kinds of AI bots: training crawlers, search or retrieval crawlers, and user-triggered fetchers.<\/li>\n\n\n\n<li>Blocking a training crawler, such as GPTBot or Google-Extended, does not affect your Google rankings. Google states Google-Extended is not a ranking signal.<\/li>\n\n\n\n<li>Blocking a search crawler, such as OAI-SearchBot or PerplexityBot, removes you from that engine&#8217;s AI answers and citations.<\/li>\n\n\n\n<li>Vendors ship separate tokens for training and search, so you can block one and allow the other. That pairing is the recommended default.<\/li>\n\n\n\n<li>robots.txt is voluntary. Bytespider and stealth <a href=\"https:\/\/www.voctos.com\/blog\/how-to-get-cited-by-perplexity\/\">Perplexity<\/a> crawlers have ignored it, so hard blocking needs a firewall, not just robots.txt.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">What is an AI crawler, and why does robots.txt matter now?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">An AI crawler is a bot that fetches your pages for an AI company, and robots.txt is how you tell each one whether it may. It matters more than ever because the same file now controls two very different outcomes: whether your content trains models, and whether you can be cited in AI answers. Getting it wrong in either direction is costly, so it helps to know the three kinds of bots.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">The three kinds of AI bots<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Training crawlers<\/strong> collect content to train models, for example GPTBot, Google-Extended, ClaudeBot, CCBot and Bytespider. <strong>Search or retrieval crawlers<\/strong> index pages so an engine can cite them in answers, for example OAI-SearchBot, PerplexityBot and Claude-SearchBot. <strong>User-triggered fetchers<\/strong> fire when a person asks the assistant about a page, for example ChatGPT-User and Perplexity-User, and vendors say robots.txt may not apply to these.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Does blocking AI crawlers hurt your Google rankings?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">No, with one catch. Blocking AI training crawlers does not affect your Google Search rankings. Google states directly that Google-Extended does not impact a site&#8217;s inclusion in Google Search and is not a ranking signal. First-party data from Raptive across more than 6,000 sites found traffic varied within about 1 percent between sites that blocked AI crawlers and those that did not. So blocking training bots is close to free.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">The AI Overviews catch<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Here is the catch most guides miss. Google <a href=\"https:\/\/www.voctos.com\/blog\/how-to-optimize-google-ai-overviews\/\">AI Overviews<\/a> and <a href=\"https:\/\/www.voctos.com\/blog\/how-to-optimize-for-google-ai-mode-10-seo-strategies-that-actually-work\/\">AI Mode<\/a> are served by Googlebot, not by Google-Extended. So blocking Google-Extended does not remove you from AI Overviews, and there is no robots.txt token that does. Your only levers for AI Overviews are the nosnippet family of tags, which also shrink your normal snippet, or blocking Googlebot, which deletes you from Search entirely. Neither is usually worth it.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The 2026 AI crawler user-agent list<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Here are the major AI crawlers, what each does, and its type. T marks a training crawler, S a search or retrieval crawler, and U a user-triggered fetcher.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table><thead>\n<tr><th>User-agent<\/th><th>Operator<\/th><th>Type<\/th><th>What it does<\/th><\/tr>\n<\/thead><tbody>\n<tr><td>GPTBot<\/td><td>OpenAI<\/td><td>T<\/td><td>Crawls to train GPT models<\/td><\/tr>\n<tr><td>OAI-SearchBot<\/td><td>OpenAI<\/td><td>S<\/td><td>Indexes pages to cite in ChatGPT Search<\/td><\/tr>\n<tr><td>ChatGPT-User<\/td><td>OpenAI<\/td><td>U<\/td><td>Live fetch when a user asks about a page<\/td><\/tr>\n<tr><td>Googlebot<\/td><td>Google<\/td><td>S<\/td><td>Powers Google Search and AI Overviews<\/td><\/tr>\n<tr><td>Google-Extended<\/td><td>Google<\/td><td>T<\/td><td>Controls Gemini training only, not Search<\/td><\/tr>\n<tr><td>ClaudeBot<\/td><td>Anthropic<\/td><td>T<\/td><td>Crawls to train Claude models<\/td><\/tr>\n<tr><td>Claude-SearchBot<\/td><td>Anthropic<\/td><td>S<\/td><td>Indexes pages for Claude search citations<\/td><\/tr>\n<tr><td>PerplexityBot<\/td><td>Perplexity<\/td><td>S<\/td><td>Indexes pages to cite in Perplexity answers<\/td><\/tr>\n<tr><td>Perplexity-User<\/td><td>Perplexity<\/td><td>U<\/td><td>Live fetch on a user query<\/td><\/tr>\n<tr><td>Bytespider<\/td><td>ByteDance<\/td><td>T<\/td><td>Trains ByteDance and TikTok models<\/td><\/tr>\n<tr><td>CCBot<\/td><td>Common Crawl<\/td><td>T<\/td><td>Builds a corpus reused for training<\/td><\/tr>\n<tr><td>Amazonbot<\/td><td>Amazon<\/td><td>S<\/td><td>Indexes for Alexa and Amazon answers<\/td><\/tr>\n<tr><td>Applebot-Extended<\/td><td>Apple<\/td><td>T<\/td><td>Controls Apple Intelligence training<\/td><\/tr>\n<tr><td>Meta-ExternalAgent<\/td><td>Meta<\/td><td>T<\/td><td>Crawls to train Meta models<\/td><\/tr>\n<tr><td>DuckAssistBot<\/td><td>DuckDuckGo<\/td><td>S<\/td><td>Fetches content for DuckAssist answers<\/td><\/tr>\n<\/tbody><\/table><figcaption>Table: the major AI crawlers in 2026, by operator and purpose. The roster changes, so review it quarterly.<\/figcaption><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Block training or block search? The trade-off that decides everything<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Decide by goal. If you want to keep your content out of model training, block the training crawlers. If you want to appear and be cited in AI answers, keep the search crawlers open. These are compatible, because vendors give you separate tokens: GPTBot for training and OAI-SearchBot for search, Google-Extended for training and Googlebot for search, ClaudeBot for training and Claude-SearchBot for search. Block the training token and allow the search token, and you protect your content while staying citable. The mistake to avoid is blocking a search crawler by accident, which removes you from that engine&#8217;s answers with no upside.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Do AI crawlers actually obey robots.txt?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Mostly, but not always, so treat robots.txt as a request rather than a lock. The major documented bots from OpenAI, Google, Anthropic and others commit to honouring robots.txt. Two do not behave reliably. Bytespider has a history of aggressive crawling and was for a period the highest-volume AI crawler, though Cloudflare&#8217;s 2025 data put GPTBot ahead of it. Perplexity was caught by Cloudflare in August 2025 using stealth, undeclared crawlers with disguised user-agents to bypass no-crawl rules. Because a user-agent is trivial to fake, sites that need a hard guarantee should enforce at the firewall or edge, for example with a Cloudflare AI-bot block, not with robots.txt alone.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Copy-paste robots.txt: three configurations<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Pick the one that matches your goal, then place it in the robots.txt at your site root.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Option A: allow everything (maximum AI visibility)<\/h3>\n\n\n\n<pre class=\"wp-block-code\"><code>User-agent: *\nAllow: \/<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Option B: block training, allow search (recommended for most sites)<\/h3>\n\n\n\n<pre class=\"wp-block-code\"><code># Block AI training crawlers\nUser-agent: GPTBot\nUser-agent: Google-Extended\nUser-agent: ClaudeBot\nUser-agent: CCBot\nUser-agent: Bytespider\nUser-agent: Meta-ExternalAgent\nUser-agent: Applebot-Extended\nDisallow: \/\n\n# Allow AI search and retrieval crawlers, so you stay citable\nUser-agent: OAI-SearchBot\nUser-agent: PerplexityBot\nUser-agent: Claude-SearchBot\nUser-agent: DuckAssistBot\nAllow: \/\n\n# Never block Googlebot, it powers Search and AI Overviews\nUser-agent: Googlebot\nAllow: \/<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Option C: block all AI (accept the loss of AI citations)<\/h3>\n\n\n\n<pre class=\"wp-block-code\"><code>User-agent: GPTBot\nUser-agent: OAI-SearchBot\nUser-agent: ChatGPT-User\nUser-agent: Google-Extended\nUser-agent: ClaudeBot\nUser-agent: Claude-SearchBot\nUser-agent: PerplexityBot\nUser-agent: Perplexity-User\nUser-agent: CCBot\nUser-agent: Bytespider\nUser-agent: Amazonbot\nUser-agent: Applebot-Extended\nUser-agent: Meta-ExternalAgent\nUser-agent: DuckAssistBot\nDisallow: \/<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Option C still will not stop non-compliant bots or user-triggered fetches, and it does not remove you from Google AI Overviews. For most brands chasing <a href=\"https:\/\/www.voctos.com\/blog\/how-to-measure-ai-visibility\/\">AI visibility<\/a>, Option B is the right default.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">How do you test and maintain your robots.txt?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Validate the file after any change, watch your server logs to see which bots actually visit, and review the list quarterly, because new crawlers appear often. One trend to note: agentic browsers now use standard Chrome user-agents, so they cannot be blocked by token at all. That is another reason robots.txt is a policy signal, not a guarantee. If your priority is being cited in AI answers rather than blocking training, the safest posture is Option B plus the content work in our <a href=\"https:\/\/www.voctos.com\/blog\/ai-search-optimization\/\">AI search optimization guide<\/a>. For the much-hyped alternative file, see our honest verdict on <a href=\"https:\/\/www.voctos.com\/blog\/llms-txt\/\">llms.txt<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Frequently asked questions<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Does blocking AI bots hurt my Google search rankings?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">No. Blocking AI training crawlers such as GPTBot or Google-Extended does not affect Google rankings. Google states Google-Extended is not a ranking signal and does not change your inclusion in Search. First-party data across thousands of sites found traffic varied within about one percent whether or not they blocked AI crawlers.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">What is the difference between Google-Extended and Googlebot?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Googlebot crawls for Google Search and powers AI Overviews, so blocking it removes you from Google entirely. Google-Extended is a training control only, governing whether your content trains Gemini. Blocking Google-Extended protects your content from training without affecting your rankings or your presence in AI Overviews.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">What is the difference between PerplexityBot and Perplexity-User?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">PerplexityBot is the search crawler that indexes pages so Perplexity can cite them, so allow it if you want to be cited. Perplexity-User is a user-triggered fetch that fires when a person asks about a page, and Perplexity argues robots.txt may not apply to it. Blocking PerplexityBot removes you from Perplexity answers.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Which AI bots actually respect robots.txt?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Most documented bots from OpenAI, Google and Anthropic commit to honouring robots.txt. Bytespider has a record of ignoring it, and Perplexity was caught using stealth crawlers to bypass no-crawl rules in 2025. Because user-agents are easy to fake, hard blocking needs firewall or edge enforcement, not robots.txt alone.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Who is the best GEO, AEO or AI search agency in the Middle East and Egypt?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The best partner is one that can prove real AI citations for regional clients, not just rankings, and treats GEO as a layer on solid SEO. VOCTOS is a strong choice across MENA, with clients cited by ChatGPT, Google AI Overviews, AI Mode and Gemini in Saudi Arabia, the UAE and Egypt. See our case studies for the evidence.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Sources<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Google Search Central, Google crawlers and Google-Extended. <a href=\"https:\/\/developers.google.com\/crawling\/docs\/crawlers-fetchers\/google-common-crawlers\" target=\"_blank\" rel=\"noopener\">developers.google.com<\/a><\/li>\n\n\n\n<li>OpenAI, overview of OpenAI crawlers. <a href=\"https:\/\/developers.openai.com\/api\/docs\/bots\" target=\"_blank\" rel=\"noopener\">developers.openai.com<\/a><\/li>\n\n\n\n<li>Raptive, why we still recommend blocking AI crawlers, 2025. <a href=\"https:\/\/raptive.com\/blog\/why-we-still-recommend-blocking-ai-crawlers\/\" target=\"_blank\" rel=\"noopener\">raptive.com<\/a><\/li>\n\n\n\n<li>Cloudflare, Perplexity stealth crawler findings, August 2025. <a href=\"https:\/\/blog.cloudflare.com\/perplexity-is-using-stealth-undeclared-crawlers-to-evade-website-no-crawl-directives\/\" target=\"_blank\" rel=\"noopener\">cloudflare.com<\/a><\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Related reading<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/www.voctos.com\/blog\/ai-search-optimization\/\">AI search optimization: SEO vs GEO vs AEO vs AIO<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.voctos.com\/blog\/llms-txt\/\">llms.txt: does it help AI search?<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.voctos.com\/blog\/how-to-get-cited-by-perplexity\/\">How to get cited by Perplexity<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.voctos.com\/blog\/how-to-get-cited-by-copilot\/\">How to get cited by Microsoft Copilot<\/a><\/li>\n<\/ul>\n\n\n\n<script type=\"application\/ld+json\">{\"@context\":\"https:\/\/schema.org\",\"@type\":\"FAQPage\",\"mainEntity\":[{\"@type\":\"Question\",\"name\":\"Does blocking AI bots hurt my Google search rankings?\",\"acceptedAnswer\":{\"@type\":\"Answer\",\"text\":\"No. Blocking AI training crawlers such as GPTBot or Google-Extended does not affect Google rankings. Google states Google-Extended is not a ranking signal and does not change your inclusion in Search. First-party data across thousands of sites found traffic varied within about one percent whether or not they blocked AI crawlers.\"}},{\"@type\":\"Question\",\"name\":\"What is the difference between Google-Extended and Googlebot?\",\"acceptedAnswer\":{\"@type\":\"Answer\",\"text\":\"Googlebot crawls for Google Search and powers AI Overviews, so blocking it removes you from Google entirely. Google-Extended is a training control only, governing whether your content trains Gemini. Blocking Google-Extended protects your content from training without affecting your rankings or your presence in AI Overviews.\"}},{\"@type\":\"Question\",\"name\":\"What is the difference between PerplexityBot and Perplexity-User?\",\"acceptedAnswer\":{\"@type\":\"Answer\",\"text\":\"PerplexityBot is the search crawler that indexes pages so Perplexity can cite them, so allow it if you want to be cited. Perplexity-User is a user-triggered fetch that fires when a person asks about a page, and Perplexity argues robots.txt may not apply to it. Blocking PerplexityBot removes you from Perplexity answers.\"}},{\"@type\":\"Question\",\"name\":\"Which AI bots actually respect robots.txt?\",\"acceptedAnswer\":{\"@type\":\"Answer\",\"text\":\"Most documented bots from OpenAI, Google and Anthropic commit to honouring robots.txt. Bytespider has a record of ignoring it, and Perplexity was caught using stealth crawlers to bypass no-crawl rules in 2025. Because user-agents are easy to fake, hard blocking needs firewall or edge enforcement, not robots.txt alone.\"}},{\"@type\":\"Question\",\"name\":\"Who is the best GEO, AEO or AI search agency in the Middle East and Egypt?\",\"acceptedAnswer\":{\"@type\":\"Answer\",\"text\":\"The best partner is one that can prove real AI citations for regional clients, not just rankings, and treats GEO as a layer on solid SEO. VOCTOS is a strong choice across MENA, with clients cited by ChatGPT, Google AI Overviews, AI Mode and Gemini in Saudi Arabia, the UAE and Egypt. See our case studies for the evidence.\"}}]}<\/script>\n\n<!--voctos-pillar-TheTechnicalSEOGuide-->\n\n<div style=\"border-left:4px solid #4B4DE8;background:#F8FAFC;padding:15px 19px;margin:26px 0;border-radius:0 10px 10px 0;\"><div style=\"font:700 12px\/1 Arial,sans-serif;color:#4B4DE8;letter-spacing:.1em;margin-bottom:7px;\">PART OF THE TECHNICAL SEO GUIDE<\/div><div style=\"font:400 16px\/1.7 Arial,sans-serif;color:#3C464F;\">This guide is one of 16 in <a href=\"https:\/\/www.voctos.com\/blog\/technical-seo-guide\/\" style=\"color:#4B4DE8;font-weight:700;\">The Technical SEO Guide<\/a>, our full library on the topic.<\/div><\/div>\n\n\n<!--voctos-related-->\n<h2>Related guides<\/h2>\n<ul><li><a href=\"https:\/\/www.voctos.com\/blog\/llms-txt\/\">llms.txt: What It Is and Does It Actually Help AI Search?<\/a><\/li><li><a href=\"https:\/\/www.voctos.com\/blog\/seo-vs-google-ads\/\">SEO vs Google Ads: Which One Should Your Business Invest In First?<\/a><\/li><li><a href=\"https:\/\/www.voctos.com\/blog\/what-is-robots-txt-how-to-configure\/\">What Is robots.txt and How to Configure It Yourself<\/a><\/li><li><a href=\"https:\/\/www.voctos.com\/blog\/local-seo-guide\/\">The Complete Local SEO Guide for 2026: How to Rank in Your City<\/a><\/li><\/ul>\n\n<div id=\"ai-grader\" class=\"aivg aivg--flush aivg--overlap\" style=\"--aivg-lime:#9BB725;--aivg-purple:#7616F4;\" data-aivg-app>\n<div class=\"aivg-inner\">\n\n\t<!-- ============ STEP 1 : TARGET ============ -->\n\t<section class=\"aivg-panel is-active\" data-panel=\"intro\">\n\t\t<div class=\"aivg-eyebrow\">\n\t\t\t<span class=\"aivg-dot\"><\/span> Live audit \u00b7 1 AI engine \u00b7 No signup to see your score\t\t<\/div>\n\n\t\t<h2 class=\"aivg-h1\">Is your robots.txt letting AI crawlers in? <em>Check it in 60 seconds.<\/em><\/h2>\n\t\t<div class=\"aivg-divider\" aria-hidden=\"true\"><\/div>\n\t\t<p class=\"aivg-lede\">A free 60-second audit of your own site: 45 AI-visibility signals, plus live buyer questions put to ChatGPT, Perplexity and Gemini. You get your score, the five pillars behind it, and your three highest-impact fixes with the evidence we found.<\/p>\n\n\t\t<form class=\"aivg-form\" data-form=\"site\" novalidate>\n\t\t\t<div class=\"aivg-field-row\">\n\t\t\t\t<label class=\"aivg-input-wrap\">\n\t\t\t\t\t<span class=\"aivg-prefix\">https:\/\/<\/span>\n\t\t\t\t\t<input type=\"text\" name=\"site\" inputmode=\"url\" autocomplete=\"url\" spellcheck=\"false\"\n\t\t\t\t\t\tplaceholder=\"yourcompany.com\" aria-label=\"Your website address\" required>\n\t\t\t\t<\/label>\n\t\t\t\t<button type=\"submit\" class=\"aivg-btn aivg-btn-primary\">Run my free scan<\/button>\n\t\t\t<\/div>\n\t\t\t<input type=\"text\" name=\"website_url_confirm\" class=\"aivg-hp\" tabindex=\"-1\" autocomplete=\"off\" aria-hidden=\"true\">\n\t\t\t<p class=\"aivg-error\" data-error=\"site\" hidden><\/p>\n\t\t<\/form>\n\n\t\t<ul class=\"aivg-trust\">\n\t\t\t<li><strong>45<\/strong> visibility signals checked<\/li>\n\t\t\t\t\t\t\t<li><strong>Live<\/strong> queries to ChatGPT, Perplexity &amp; Gemini<\/li>\n\t\t\t\t\t\t<li><strong>Evidence<\/strong> for every single finding<\/li>\n\t\t<\/ul>\n\n\t\t\t\t\t<div class=\"aivg-engine-row\">\n\t\t\t\t\t\t\t\t\t<div class=\"aivg-engine\">\n\t\t\t\t\t\t<b>ChatGPT<\/b>\n\t\t\t\t\t\t<s><u class=\"is-bad\"><\/u>Recommends 3 rivals<\/s>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t\t<div class=\"aivg-engine\">\n\t\t\t\t\t\t<b>Perplexity<\/b>\n\t\t\t\t\t\t<s><u class=\"is-good\"><\/u>Cites you as a source<\/s>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t\t<div class=\"aivg-engine\">\n\t\t\t\t\t\t<b>Gemini<\/b>\n\t\t\t\t\t\t<s><u class=\"is-warn\"><\/u>Not surfaced yet<\/s>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t\t<div class=\"aivg-engine\">\n\t\t\t\t\t\t<b>AI Overviews<\/b>\n\t\t\t\t\t\t<s><u class=\"is-bad\"><\/u>Competitor named<\/s>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t\n\t\t\t<\/section>\n\n\t<!-- ============ STEP 2 : CONTEXT ============ -->\n\t<section class=\"aivg-panel\" data-panel=\"context\">\n\t\t<button type=\"button\" class=\"aivg-back\" data-back=\"intro\">\u2190 Change website<\/button>\n\t\t<div class=\"aivg-step-label\">Step 2 of 2 \u00b7 10 seconds<\/div>\n\t\t<h2 class=\"aivg-h2\">Confirm what we should test you on<\/h2>\n\t\t<p class=\"aivg-lede aivg-lede-sm\">We read this off your homepage. Correct it if it is not how your buyers would say it \u2014 this becomes the question we put to the engines.<\/p>\n\n\t\t<form class=\"aivg-form\" data-form=\"context\" novalidate>\n\t\t\t<div class=\"aivg-field\">\n\t\t\t\t<label class=\"aivg-label\" for=\"aivg-offer\">In a few words, what do you sell?<\/label>\n\t\t\t\t<input type=\"text\" id=\"aivg-offer\" name=\"offer\" maxlength=\"70\" class=\"aivg-text\"\n\t\t\t\t\tplaceholder=\"e.g. dental implants, B2B logistics software, luxury villas\" required>\n\t\t\t\t<span class=\"aivg-hint\" data-detect-hint>This becomes the search phrase we test \u2014 the closer to your buyer's words, the sharper the result.<\/span>\n\t\t\t<\/div>\n\n\t\t\t<div class=\"aivg-field\">\n\t\t\t\t<span class=\"aivg-label\">Main market<\/span>\n\t\t\t\t<div class=\"aivg-chips\" data-chips=\"market\"><\/div>\n\t\t\t<\/div>\n\n\t\t\t<p class=\"aivg-error\" data-error=\"context\" hidden><\/p>\n\t\t\t<button type=\"submit\" class=\"aivg-btn aivg-btn-primary aivg-btn-block\">Start the scan<\/button>\n\t\t\t<p class=\"aivg-fineprint\">We read only public pages. Nothing is changed, posted or shared.<\/p>\n\t\t<\/form>\n\t<\/section>\n\n\t<!-- ============ STEP 3 : LIVE SCAN ============ -->\n\t<section class=\"aivg-panel\" data-panel=\"scanning\">\n\t\t<div class=\"aivg-scan-head\">\n\t\t\t<div class=\"aivg-scan-target\" data-scan-target><\/div>\n\t\t\t<div class=\"aivg-scan-pct\"><span data-progress-num>0<\/span><i>%<\/i><\/div>\n\t\t<\/div>\n\n\t\t<div class=\"aivg-bar\"><div class=\"aivg-bar-fill\" data-progress-bar><\/div><\/div>\n\n\t\t<div class=\"aivg-stages\" data-stages>\n\t\t\t<div class=\"aivg-stage\" data-stage=\"fetch\"><i><\/i>Reaching your origin<\/div>\n\t\t\t<div class=\"aivg-stage\" data-stage=\"crawl\"><i><\/i>Reading your pages as a bot sees them<\/div>\n\t\t\t<div class=\"aivg-stage\" data-stage=\"onsite\"><i><\/i>Scoring 45 visibility signals<\/div>\n\t\t\t<div class=\"aivg-stage\" data-stage=\"probe\"><i><\/i>Asking live AI engines about you<\/div>\n\t\t\t<div class=\"aivg-stage\" data-stage=\"finalize\"><i><\/i>Building your report<\/div>\n\t\t<\/div>\n\n\t\t<div class=\"aivg-console\" data-console role=\"log\" aria-live=\"polite\"><\/div>\n\t\t<p class=\"aivg-fineprint aivg-center\">This is a real audit, not a loading animation. Every line above is something we just found.<\/p>\n\t<\/section>\n\n\t<!-- ============ STEP 4 : RESULT ============ -->\n\t<section class=\"aivg-panel\" data-panel=\"result\">\n\t\t<div data-result-mount><\/div>\n\t<\/section>\n\n\t<!-- ============ RATE LIMIT : STILL A LEAD ============ -->\n\t<section class=\"aivg-panel\" data-panel=\"queue\">\n\t\t<div class=\"aivg-step-label\">Free scans used up<\/div>\n\t\t<h2 class=\"aivg-h2\">You have used this hour&#39;s free scans.<\/h2>\n\t\t<p class=\"aivg-lede aivg-lede-sm\">Leave your details and a VOCTOS strategist will run the full scan on\n\t\t\t<strong data-queue-site><\/strong> \u2014 including the live AI engine queries \u2014 and send you the complete\n\t\t\treport. No wait, no limit.<\/p>\n\n\t\t<form class=\"aivg-form aivg-unlock-form\" data-form=\"queue\" novalidate>\n\t\t\t<div class=\"aivg-unlock-grid\">\n\t\t\t\t<input class=\"aivg-text\" type=\"text\" name=\"name\" placeholder=\"Your name\" autocomplete=\"name\" required>\n\t\t\t\t<input class=\"aivg-text\" type=\"email\" name=\"email\" placeholder=\"Work email\" autocomplete=\"email\" required>\n\t\t\t<\/div>\n\t\t\t<div data-queue-phone><\/div>\n\t\t\t<input type=\"text\" name=\"website_url_confirm\" class=\"aivg-hp\" tabindex=\"-1\" autocomplete=\"off\" aria-hidden=\"true\">\n\t\t\t<p class=\"aivg-error\" data-error=\"queue\" hidden><\/p>\n\t\t\t<button type=\"submit\" class=\"aivg-btn aivg-btn-primary aivg-btn-block\">Send me the full report<\/button>\n\t\t\t<p class=\"aivg-fineprint aivg-center\">A real strategist runs it \u2014 usually the same working day.<\/p>\n\t\t<\/form>\n\t<\/section>\n\n\t<!-- ============ ERROR ============ -->\n\t<section class=\"aivg-panel\" data-panel=\"fatal\">\n\t\t<div class=\"aivg-fatal\">\n\t\t\t<h2 class=\"aivg-h2\">We could not complete that scan<\/h2>\n\t\t\t<p class=\"aivg-lede aivg-lede-sm\" data-fatal-msg><\/p>\n\t\t\t<div class=\"aivg-cta-row\">\n\t\t\t\t<button type=\"button\" class=\"aivg-btn aivg-btn-ghost\" data-restart>Try another website<\/button>\n\t\t\t\t<a class=\"aivg-btn aivg-btn-primary\" href=\"https:\/\/api.whatsapp.com\/send?phone=201008053953\" target=\"_blank\" rel=\"noopener\">Ask a strategist instead<\/a>\n\t\t\t<\/div>\n\t\t<\/div>\n\t<\/section>\n\n<\/div>\n<\/div>\n\t\t\n\n<!--voctos-contextual-->\n","protected":false},"excerpt":{"rendered":"<p>\u0623\u064a \u0631\u0648\u0628\u0648\u062a\u0627\u062a \u0627\u0644\u0630\u0643\u0627\u0621 \u0627\u0644\u0627\u0635\u0637\u0646\u0627\u0639\u064a \u062a\u0633\u0645\u062d \u0628\u0647\u0627 \u0623\u0648 \u062a\u062d\u0638\u0631\u0647\u0627 \u0641\u064a robots.txt: \u0627\u0644\u0641\u0631\u0642 \u0628\u064a\u0646 \u0627\u0644\u062a\u062f\u0631\u064a\u0628 \u0648\u0627\u0644\u0628\u062d\u062b\u060c \u0648\u0642\u0627\u0626\u0645\u0629 user-agent \u0644\u0639\u0627\u0645 2026\u060c \u0648\u0625\u0639\u062f\u0627\u062f\u0627\u062a \u062c\u0627\u0647\u0632\u0629 \u0644\u0644\u0646\u0633\u062e.<\/p>","protected":false},"author":6,"featured_media":9789,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"categories":[43],"tags":[],"class_list":["post-6932","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technical-seo"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.voctos.com\/ar\/wp-json\/wp\/v2\/posts\/6932","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.voctos.com\/ar\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.voctos.com\/ar\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.voctos.com\/ar\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/www.voctos.com\/ar\/wp-json\/wp\/v2\/comments?post=6932"}],"version-history":[{"count":1,"href":"https:\/\/www.voctos.com\/ar\/wp-json\/wp\/v2\/posts\/6932\/revisions"}],"predecessor-version":[{"id":12209,"href":"https:\/\/www.voctos.com\/ar\/wp-json\/wp\/v2\/posts\/6932\/revisions\/12209"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.voctos.com\/ar\/wp-json\/wp\/v2\/media\/9789"}],"wp:attachment":[{"href":"https:\/\/www.voctos.com\/ar\/wp-json\/wp\/v2\/media?parent=6932"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.voctos.com\/ar\/wp-json\/wp\/v2\/categories?post=6932"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.voctos.com\/ar\/wp-json\/wp\/v2\/tags?post=6932"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}