Retrieval or Memory? How to Tell Why an AI Got You Wrong

Every wrong AI answer about you has one of three causes, and the test that separates them takes ninety seconds. Ask the same question with search on and with search off, then ask it in Perplexity. The three answers tell you whether you are dealing with a live source you can fix, a training memory you cannot, or a gap the engine filled by inventing something.

Get this wrong and you will spend a quarter publishing content against a problem that content cannot reach. It is the first thing we do on every engagement and the single most useful diagnostic in the discipline.

The two ingredients in every answer

Training memory is what the model absorbed before its cutoff, compressed into weights. There is no document to edit and no cache to clear. It behaves like recall rather than lookup.

Retrieval is what the engine fetched from the live web during your conversation. It reflects the world as of now, and it is the part you can influence today.

Answers blend the two, and which one dominates changes by engine, by question and sometimes by session.

The test

Pick one specific false or outdated claim. Not a general impression, one sentence you can quote.

1. ChatGPT, search off. Turn off web search and ask. This is pure memory.
2. ChatGPT, search on. Same prompt, new session, search enabled.
3. Perplexity. Same prompt. It retrieves for nearly every query and numbers its citations, which hands you the source list.

Use fresh sessions with no memory of earlier chats. A personalized session tells you what the engine says to you rather than to a stranger.

Reading the result

Pattern A: retrieval

Wrong with search on. Perplexity cites a specific page. Often correct with search off, because the model had no strong prior and went looking.

This is the good outcome. A live document is producing the claim and you can name it. Correct it if you control it. If you do not, publish something clearer, more current and more directly on the question, so retrieval has a better option. Expect movement in weeks, and expect Perplexity to show it first.

Pattern B: training memory

Wrong with search off. Right, or better, with search on. Perplexity gets it right and cites current pages.

Nothing you publish today removes this. The claim sits in weights until a model release. What you can do is make sure that when the engine does retrieve, and when the next model is trained, the correct version is unmissable. This is also the pattern that explains why free-tier users see worse answers about you than your team does, since free sessions currently run on a model whose knowledge ends in October 2023.

Report this honestly. An agency that promises a fixed date for a memory-driven claim is describing a calendar it does not control.

Pattern C: a gap

Wrong both ways. Perplexity cites nothing relevant, or cites pages that do not contain the claim.

The engine had nothing solid and produced something plausible. Invented specifics live here: a case that never happened, a certification never held, a figure nobody published. There is no wrong page to correct, which is why this feels hardest.

The fix is to make the correct answer exist, clearly, on a page an engine can reach, in a form that answers the question completely in one passage. You are competing against absence rather than against a rival source.

The fourth pattern, which the test will not show you

Absence. The engine is not wrong about you. It does not mention you, and it names a competitor instead.

This will never appear in a monitoring alert, because there is nothing to alert on. It is found only by asking the unnamed recommendation question, and it is often the most commercially expensive of the four. Whether an engine searches at all is itself a signal: Claude answering from memory without searching usually means your entity is not strong enough to trigger a lookup.

Running it across six engines

The pattern is rarely the same everywhere, and the differences are informative.

• Perplexity is your tracing tool. Numbered citations, retrieval on nearly every query.
• ChatGPT is your memory test, because you can toggle search.
• Google AI Overviews draws from pages already ranking, so a problem here is usually a ranking or structure problem.
• Gemini and AI Mode ground on the Google index. If competitors appear and you outrank them in the blue links, the gap is passage structure rather than authority.
• Claude tells you about entity strength by whether it bothers to search.
• Copilot follows Bing, a different ranking set. A wide Bing-to-Google gap is visibility nobody is measuring.

Wrong on one engine and right on another is a retrieval problem. Wrong on all six is an entity problem, and the fix is entity repair rather than content.

Frequently asked questions

What if the answer changes every time I ask?

Normal. Generation is not deterministic and retrieval reaches different pages on different days. Run each prompt several times and judge the substance rather than the wording.

Can I turn search off in every engine?

No. ChatGPT gives you the clearest control, which is why the memory test runs there. Elsewhere you infer from whether citations appear and what they point at.

Does a pattern B claim ever go away on its own?

With a model release, if the corrected version of you was well represented in the material available before training. Which is the argument for publishing the correction now rather than waiting for the release.

How often should I re-run the test?

Monthly, on a frozen prompt set, with the answers stored beside last month’s. Rewrite the questions and you lose the ability to compare, which is the only thing that makes this a measurement.

Where this leads

This diagnosis decides everything that follows in an AI reputation management engagement. The prompt set we run it against is published in full in our guide to checking what AI says about your brand.

Related guides

• Google’s Search Generative Experience (SGE) and AI Overviews: What Egyptian Brands Need to Know in 2026
• GEO: Generative Engine Optimization – Your Guide to Ranking on ChatGPT
• Generative Engine Optimization (GEO) Services in Egypt: The 2026 Playbook
• Answer Engine Optimization (AEO) Services in Egypt: How to Win Zero-Click Search