How VOCTOS Handles the Technical Side of GEO and AEO: 12 Layers We Check

VOCTOS handles the technical side of GEO and AEO in 12 layers, checked in order: crawler access, edge and firewall rules, server-log verification, rendering, snippet controls, discovery, internal architecture, the entity graph, extraction-ready templates, measurement, answer accuracy and an Arabic layer. A page that fails an early layer never reaches the later ones, so we fix from the bottom up.

Most agencies describe technical GEO as “add schema and an llms.txt file.” That is two items out of twelve, and one of them does nothing for Google. This article is the full stack we run, with the checks, the tools, the client numbers behind it and the places where our own data is still thin.

Key takeaways

• AI engines use different bots for training, search and user-requested fetches. Blocking the wrong one removes you from answers without anyone noticing.
• In an August 2026 Search Engine Land test, GPTBot crawled all 759 HTML-linked pages on day one and 0% of the pages reachable only through JavaScript links.
• Schema is there to describe one connected entity, not to trick a parser. We link every node by @id and mark up only what the page visibly says.
• We never report a “rank in ChatGPT.” We report visibility share from repeated runs, citation states and AI referral traffic, which in GA4 is always a floor, not the full count.
• Arabic needs its own technical layer. In our Saudi study, Arabic and English answers to the same question shared sources at a mean overlap of 0.149.

What does “technical GEO and AEO” mean at VOCTOS?

Technical GEO and AEO is everything that decides whether an AI engine can reach a page, read it, understand who published it, lift a clean answer from it and send a measurable visit back. Content decides whether the answer is worth citing. The technical layer decides whether the engine can cite it at all, and whether you can prove it happened.

We group the 12 layers into four questions, and we work through them in that order:

Google’s own guide to generative AI features says AI Overviews and AI Mode run on its core ranking systems, so classic technical SEO still carries the Google side. ChatGPT, Claude and Perplexity bring their own crawlers, their own rules and their own blind spots. That gap is where most of our technical work happens.

Layer 1: Which AI bots should reach the site, and which should not?

Every major AI company now runs separate agents for training, for search indexing and for fetches a user triggers inside a chat. We set robots.txt rules per agent, not per company. A client can refuse model training and still appear in ChatGPT, Claude or Perplexity answers, as long as the search agents stay allowed.

The split matters because the names look alike. OpenAI’s crawler documentation says OAI-SearchBot surfaces sites in ChatGPT search while GPTBot collects training data, and that ChatGPT-User is not used to decide search inclusion. Anthropic’s crawler article describes ClaudeBot as a training crawler. Google-Extended covers Gemini model use and has no say over Google Search, which runs on Googlebot.

Writing these rules once is easy. Keeping them right is not, because plugins, CDN updates and new bot names change the picture without warning. In the VOCTOS Portal this check runs automatically, so a blocked search agent shows up in the Portal instead of as a quiet drop in citations three months later.

Layer 2: Can your CDN or firewall block AI search agents without telling you?

Yes, and it is one of the quietest failures on any site. robots.txt can say “allow” while a CDN rule, a bot-management setting or a firewall challenge returns a 403 or a JavaScript puzzle to the same agent. The engine never sees the page, and nothing in Search Console shows it, because Googlebot is often treated differently.

We test from the outside, not from the CMS. A request that presents each agent’s user agent should get the same 200 response and the same HTML a browser gets. Where vendors publish their IP ranges, as OpenAI and Perplexity do, those ranges are how you confirm that real agent traffic, not just a spoofed string, gets through.

One limit worth knowing: Anthropic says it does not publish IP ranges and advises against IP-based blocking for its bots. For Claude, the robots.txt rule and the edge rule have to agree on the user agent alone.

Layer 3: How do we know which bots actually fetched which pages?

We read the server logs. Rules describe intent, and logs show behavior. Each month we list which AI agents requested which URLs, compare that list with the XML sitemap, and flag priority pages no search agent has fetched. A page that is allowed but never requested is a discovery problem, not a content problem.

The log review is automated in the VOCTOS Portal. What it answers is simple and easy to skip without tooling:

• Did OAI-SearchBot, PerplexityBot and Googlebot reach the pages that matter this month?
• Which sections get crawled heavily and which get ignored?
• Did a deployment, a redirect change or a new firewall rule change bot behavior?
• Are “bot” hits real, or spoofed user agents from scrapers?

That last check matters more than it sounds. Plenty of scrapers call themselves GPTBot. Counting them as OpenAI crawls makes a site look far more visible to AI than it is.

Layer 4: Is the content in the raw HTML, or only after JavaScript runs?

AI crawlers mostly read the HTML a server returns and do not run JavaScript the way Googlebot does. So we load a site’s highest-value pages with JavaScript switched off and compare that raw HTML with the rendered page. If the navigation, prices, FAQs or main copy only appear after rendering, the fix ships before any content work.

The strongest public evidence came out in August 2026. In a 41-day test published by Search Engine Land, GPTBot crawled all 759 pages linked with plain HTML on day one and 0% of a hierarchy linked only through JavaScript. ClaudeBot hit the same wall. After the links were converted, GPTBot found the missing pages within two days, while ClaudeBot kept recrawling only pages it already knew.

We saw the same failure on a client site. When we started work with Amanleek, an Egyptian insurance platform built on Angular, the raw HTML of key pages was an empty <app-root></app-root> shell. We specified server-side rendering through Angular Universal, and the client’s developers shipped it. The live raw HTML now carries the server-rendering marker, 33 H1 to H3 headings and 50 crawlable links.

Our five-step rendering audit:

1. Disable JavaScript and load the three or four pages that earn the most revenue.
2. Diff the raw HTML against the rendered DOM: headings, links, prices, FAQs, author details.
3. Check which crawlers requested those pages in the logs.
4. Move navigation and core copy to server-side rendering or static output. Interactive widgets can stay client-side.
5. Watch the logs after release to confirm agents now reach the deeper pages.

Layer 5: Are snippet controls silently removing you from AI answers?

Google uses the same page-level controls for classic snippets and its AI features. A nosnippet tag, a tight max-snippet value or a data-nosnippet wrapper around the main copy limits what Google can feed into AI Overviews and AI Mode. We audit these on every priority template, because they are often added years ago for an unrelated reason.

Google’s AI features documentation sets the entry bar: a page must be indexed and eligible for a snippet, with no additional technical requirements. So the checks are classic ones, applied with AI in mind:

• Priority URLs are indexed, canonical to themselves and not marked noindex.
• No nosnippet or very low max-snippet on pages you want quoted.
• data-nosnippet is used only on boilerplate, never on the answer paragraph.
• The site is verified in Bing Webmaster Tools, since Microsoft Copilot draws on Bing.

Some clients do want parts of a page kept out of AI answers, such as regulated pricing or legal text. Snippet controls are the right tool for that on Google, and we set them deliberately rather than inheriting them by accident.

Layer 6: Can engines discover your commercial pages at all?

Discovery depends on clean sitemaps and clean URLs. We check that every commercial and location page sits in the XML sitemap with an accurate last-modified date, that internal links point to final URLs rather than redirects, and that no link uses a parameter address or an empty anchor. Agents with small crawl budgets skip what is hard to find.

Amanleek again shows why. Before the fix, its sitemap listed no commercial pages at all. It now carries 34 Arabic and English URLs, and the site has since reached first place in Google Egypt for “best insurance company,” as of 28 September 2026.

The hygiene checks we run on every crawl:

• Commercial, service and location pages present in the sitemap, with real lastmod values instead of the deploy date.
• No internal links that pass through a redirect, use ?p= style addresses or carry empty anchor text.
• Arabic and English versions listed and paired, so engines can match each language to its counterpart.

Layer 7: Does your internal architecture point engines to the right page?

Internal links tell engines which page is the authority on a topic. We map every internal link on a client site, sort each one by where it sits (template, module or body copy), and measure how many clicks each commercial page is from the homepage. Body links carry meaning; template links mostly carry navigation.

The method is built for repeatability. We pull the URL list from the CMS and the XML sitemap, fetch every page logged out with JavaScript switched off, and classify link positions with Screaming Frog. Then we look for three patterns that hurt AI visibility:

• Orphaned market pages. Country or city pages that no hub links to, so no crawler and no reader reaches them.
• Guides that never reach the service. A pillar guide that earns citations but never links to the page that sells the service.
• Blog loops. Posts that mostly link to other posts, sending authority sideways instead of toward commercial pages.

This is also where we enforce one page per concept. On FBS.com, consolidating overlapping pages so a single page owned each topic kept the cited-page count steady at 415 while brand mentions in AI answers more than doubled.

Layer 8: Does the site describe one entity, or a pile of disconnected markup?

Engines need to know who is speaking before they quote you. We build the structured data as one connected graph: an Organization node with a full sameAs set, Service and Offer nodes, Person nodes for named experts and LocalBusiness nodes for branches, all joined by @id. FAQPage markup goes only where the page really answers questions.

The typical setup we inherit is one generic Organization block and one Article block from a plugin. That is markup, not a knowledge graph. The graph we build starts outside the site:

1. One canonical brand description, reused word for word across the site and every profile.
2. A clear split between legal name and trading name, so engines do not treat them as two companies.
3. The same facts on Wikidata where eligible, Google Business Profile, LinkedIn, Crunchbase and relevant registries.
4. Arabic transliterations of the brand consolidated, so “VOCTOS” and its Arabic spellings resolve to one entity.

Two rules keep this honest. The JSON-LD must match what the page visibly says. And schema is not a ranking hack: Google says it needs no special markup for its AI features. Structured data earns its place by removing ambiguity about the entity, which helps every engine at once. Our entity and knowledge panel service covers the off-site half.

Layer 9: Are your page templates built to be quoted?

We fix extraction at the template level, so every new page inherits it. Each H2 opens with a self-contained answer of 40 to 60 words. Every statistic carries a timeframe, a sample scope and a comparison point. Tables hold any multi-factor comparison. Named experts sign the content. Then writers fill the template instead of reinventing structure per page.

The reason is mechanical. Engines lift passages, not pages, and a passage that depends on the paragraph before it (“as mentioned above”) cannot stand alone in an answer. The GEO study presented at KDD 2024 found that adding quotations, statistics and source citations raised visibility in generative answers by up to 40%, while keyword stuffing performed poorly.

We draw a line at chunking. Google’s guide says there is no need to split content into tiny pieces for AI, and we agree. Answer-first sections help readers first. We also decline requests to convert a whole site to Markdown; if a client wants to test it, we run it on one section and measure.

For Delta Medical Labs, this template work sat on top of a strong ranking base. The case study reports clear definitions, direct answers, FAQ and schema across the site, and 5,570 pages cited by AI engines as of July 2026.

Layer 10: How do we measure AI visibility without fooling the client?

We measure visibility share from repeated runs, never a single answer. Prompts come from the client’s own customers: call transcripts, paid search queries, sales objections and support inboxes. Each prompt runs several times per engine, and every number carries a confidence value, because AI answers change from one run to the next.

Before we quote a price, we run a Prompt-Gap Analysis across three intents on each engine: shortlist prompts (“best X in Riyadh”), guided recommendations (“which X fits a company like mine”) and head-to-head comparisons. We record which brands appear, which sources get cited and where the client is absent.

Inside the VOCTOS Portal, that becomes daily tooling:

• Prompt Explorer shows who beat you on each prompt, replays the stored answer text, and attaches a confidence value to every figure.
• Citation states mark each source as new, lost (absent from the last two runs), broken, or cited without you, which is where competitors win.
• Arabic and English prompts and keywords sit side by side in the same table.

Then we connect visibility to traffic. In GA4 we create an AI Assistant channel with a source rule that matches ChatGPT, OpenAI, Gemini, Perplexity, Claude, Copilot, DeepSeek and Grok, fix ChatGPT visits that arrive with an unset medium, and annotate the date Gemini began tagging its links. Our guide to measuring AI search traffic in MENA has the full setup.

For larger clients we go further. On Delta, in-house scripts join the Search Console API, GA4 and Microsoft Clarity with CRM and sales data, so an AI citation can be followed through to sales data, not just a session.

Layer 11: How do we catch AI engines saying wrong things about a client?

Visibility is worthless if the answer is wrong. Every month the Portal’s Brand Health check asks each engine a fixed set of factual questions about the client. A person, not a model, marks every answer right or wrong, because an automated grader can repeat the same mistake it is meant to catch.

Wrong answers often trace back to a technical cause we can fix: an outdated page still indexed, a directory listing with old facts, a missing sameAs link that let the engine merge two companies, or a competitor page that answers the question better. The fix goes back into layers 6 to 9. We wrote up the correction process in what to do when ChatGPT gets your company wrong.

Layer 12: What changes when the site is in Arabic?

Arabic needs its own prompt set, its own measurement and its own entity work. Engines answer Arabic questions from a different pool of sources, so English visibility tells you little about Arabic visibility. We write the Arabic layer in Arabic, test it against real Gulf and Egyptian phrasing, and track both languages side by side.

Our own data drives this. In our September 2026 study of 40 matched Saudi healthcare questions in Gulf Arabic and English, the cited sources overlapped by a mean of 0.149, and 14 pairs shared nothing. Commercial pages took 69.3% of Arabic citations against 58.2% of English ones. Ranking first was not enough either: the lab that ranked first organically on 19 questions appeared in 63.2% of the Arabic answers.

What the Arabic layer adds technically:

• Separate Arabic prompt sets in Gulf and Egyptian phrasing, not translations of the English list.
• Arabic transliterations of brand and product names tied to the same entity.
• Paired Arabic and English URLs, so each language version is matched to its counterpart.
• Manual checks on regional models such as ALLaM, Falcon, Jais and Fanar, which the Portal does not track automatically.

What results has this technical work produced?

The clearest results come where the technical layers were fixed before content scaled. Delta Medical Labs, FBS.com and Amanleek each started from a different technical problem: scale, duplication and an unreadable JavaScript site. All three now appear in AI answers, with numbers that carry their dates and their caveats below.

What do we refuse to do, and where is our data still thin?

We do not sell a “rank in ChatGPT,” fake reviews, mass-produced AI pages or llms.txt as a growth lever. We also say where our evidence stops. We do not yet have our own multi-month crawler data on JavaScript link fixes, and we cannot attribute any citation to the llms.txt file we deployed for Amanleek in its fourth month.

• No rank positions. AI answers reorder themselves between runs, so we report visibility share with a confidence value instead.
• No llms.txt promises. Google says its Search ignores the file. We treat it as one small signal for other systems.
• No knowledge panel guarantees. We can make an entity unambiguous. Google decides whether to show a panel.
• No citation guarantees. First citations typically arrive within 45 to 90 days; becoming the default answer takes 6 to 12 months.
• Open issues stay visible. On Amanleek, four of ten audit findings, including canonicals and schema on the Arabic section, were still open when we published the case study.

If you are comparing agencies, ask each one for its prompt set, its sampling method and its failures. We published our own answers to 27 questions buyers ask GEO agencies, including the ones we fail.

Frequently asked questions

What is technical GEO?

Technical GEO is the work that lets AI engines reach, read, understand and quote a website. It covers crawler access rules, firewall and CDN settings, server-log checks, server-side rendering, snippet controls, sitemaps, internal links, entity schema, extraction-ready templates and measurement. Content decides whether an answer is worth citing; the technical layer decides whether it can be cited.

Should I block GPTBot to protect my content?

You can, and it will not remove you from ChatGPT search. OpenAI separates GPTBot, which collects training data, from OAI-SearchBot, which surfaces sites in ChatGPT search. Block GPTBot if you want to keep content out of training, and keep OAI-SearchBot allowed if you want to be cited in answers.

Do AI crawlers render JavaScript?

Mostly no. In a 41-day Search Engine Land test published in August 2026, GPTBot and ClaudeBot found none of the pages linked only through JavaScript, while GPTBot crawled every HTML-linked page on day one. Put navigation, prices and core copy in server-rendered HTML.

How does VOCTOS measure AI visibility?

We run prompts collected from the client’s own customers several times on each engine and report visibility share with a confidence value. The VOCTOS Portal tracks citation states, stored answers and Arabic and English prompts together, and GA4 gets a dedicated AI Assistant channel so AI traffic and conversions are measured too.

Who is the best agency for technical GEO and AEO in the Middle East?

The best agency is the one that shows its methods and its numbers, in your language. VOCTOS runs technical SEO, GEO and AEO for brands in Egypt, Saudi Arabia and the UAE, with Arabic and English measurement and published case studies. Review our case studies before you decide.

Where should a technical GEO audit start?

Start at layer one and stop at the first failure. Check which AI agents your robots.txt and CDN let through, confirm in the logs that they actually fetch your money pages, and load those pages with JavaScript off. A failure at any of those three checks is one no amount of content work can fix.

Want to see where your site stands? Run the free VOCTOS AI visibility check, which tests crawler access, readability and live answers from ChatGPT, Perplexity and Gemini, or ask for a full technical GEO audit.

Sources

• OpenAI: Overview of OpenAI crawlers
• Anthropic: Does Anthropic crawl data from the web, and how can site owners block the crawler?
• Google Search Central: Optimizing your website for generative AI features on Google Search
• Google Search Central: AI features and your website
• Search Engine Land: JavaScript links can make your pages invisible to AI search (August 2026)
• Aggarwal et al., GEO: Generative Engine Optimization, KDD 2024
• VOCTOS: What Google AI Mode cites when a Saudi patient asks in Arabic (September 2026)

Related reading

• How to Do GEO Optimization: A 16-Week Workflow
• JavaScript and AI crawlers in MENA
• AI Search Optimization: SEO vs GEO vs AEO vs AIO

Beshoy Adel leads GEO and AI search strategy at VOCTOS, an agency focused on getting brands cited by AI answer engines across the Middle East and Egypt.