Skip to main content

AI Visibility | 9 min read

AI Visibility Checker: Running One, and Reading It Properly

By ยท Updated ยท 9 min read

What an AI visibility checker is for

An AI visibility checker answers one question: when somebody asks an assistant about your category, does your name come up. That is a genuinely useful thing to know and it is not something you can find in your analytics, because most people who read a recommendation never click anything.

The tools vary enormously in what they do underneath. Some ask a handful of generated questions to one engine and give you a percentage. Some run a large question set across four engines on a schedule and track the trend. The output looks similar in both cases, which is precisely the problem.

This page is about how to run a check that tells you something true, whether you use a tool or do it by hand. Doing it by hand once is genuinely worth the hour, because it shows you what the tools are compressing.

The short version

The check itself is easy. The questions are the hard part, and a checker that writes the questions for you has made the most important decision on your behalf.

How these tools actually work

Every AI visibility checker does the same four things, and knowing them lets you interrogate any tool in about two minutes.

It takes your brand. Usually a domain and a name, sometimes a category. Some tools try to infer your category from your homepage, which works well for a focused store and badly for one that sells several unrelated things.

It builds a question set. This is where the tools diverge most. Some generate questions from your site's content, which tends to produce questions shaped like your marketing rather than like a buyer's actual uncertainty. Some let you write your own, which is more work and much better.

It asks the engines. One engine or several. Once, or repeatedly. A tool asking a single engine once is producing an anecdote and presenting it as a measurement.

It records whether you appear. Present or absent, sometimes with position. The better ones keep the full text of the answer, which matters more than it sounds, because being mentioned dismissively and being recommended both count as present.

What an AI visibility checker actually does Four stages from brand input to a recorded result, with two failure points marked below. 1. Takes your brand Domain, brand name, sometimes a category 2. Builds questions Either generated for you or written by you 3. Asks the engines One engine, or several, once or repeatedly 4. Records the result Named or not named, and in what position FAILURE 1 Questions no real buyer asks. The answer is about nothing. FAILURE 2 One run treated as a result. Answers vary between runs.
The two places a check goes wrong. Both produce a confident number that means nothing.

What the free ones can and cannot tell you

Free checkers are genuinely useful for one job and misleading for another, so it is worth being clear about which is which.

What a free check is good for: a first look. If you have never checked, running a free tool will tell you whether you are anywhere at all. Finding out that you are absent from every answer in your category is a real and actionable result, and it costs nothing.

What a free check cannot do: establish a trend. A number from a single run on a single day carries no information about direction. Answers vary run to run for reasons that have nothing to do with you, and a second run a week later showing a different number tells you about that variance rather than about your progress.

What free tools usually get wrong: the questions. A generated question set tends to reflect the words on your website. Your buyers use different words, because they do not know your product exists yet. This is the single largest source of confidently wrong results.

The reasonable use is to run a free check to see whether there is a problem, then do the question-writing work yourself before you invest in tracking anything over time.

One more thing worth knowing about the free tier of anything in this space. A free checker is usually a lead magnet, which is fine, but it shapes the product. It is built to produce a result that feels alarming enough to prompt a conversation, so it tends to ask broad questions where almost nobody is named rather than the specific ones where a smaller business plausibly is. That is not dishonest. It does mean the picture skews pessimistic, and that a business quietly doing well on the narrow questions its buyers actually ask can come away believing it is invisible.

Running a proper check by hand

An afternoon and a spreadsheet gets you a better baseline than most tools, and it teaches you what matters.

Write twenty questions in a buyer's words. Not your product name. Not keywords. The questions somebody asks when they have the problem and do not yet know the solution. If you sell insulated bottles, that is "what water bottle keeps ice all day", not "best insulated bottle brand".

Include the shape of question that actually gets asked. Real buyers ask for shortlists ("what should I get for..."), comparisons ("is X or Y better for..."), and constraints ("what works if I have..."). A set made entirely of "best X" questions misses most of how people talk to assistants.

Ask each question to at least two engines. They disagree constantly, and the disagreement is informative. Being present in Perplexity and absent from ChatGPT points at a different problem from being absent everywhere.

Record every brand named, not just whether you appeared. This turns a pass or fail into a picture of your category, and it tells you who you are actually competing with for the mention.

Repeat the identical set monthly. Same wording. Changing the questions resets your history, so write them carefully once.

What a single check proves, and what it does not

This is where most people over-read the result, in both directions.

A check showing you are absent everywhere is reliable. If twenty questions across two engines never name you, that is not noise. You are not in the consideration set, and you can act on it.

A check showing you appear once is not. A single appearance in a single answer can easily be run-to-run variation. Do not build a strategy on it, and do not report it as a win.

A check cannot tell you why. Absence has at least three separate causes: an AI crawler that cannot read your site, pages that never surface in the results the assistant reads, and pages that are read but contain nothing quotable. Those need three different fixes and the check does not distinguish them. The optimization guide works through how to tell them apart.

A check cannot tell you about money. Being named is upstream of being chosen. Whether the mention converts depends on your price, your reviews and your product.

What to do with the result

Three outcomes, three different next moves.

If you are absent everywhere, start with the crawler check. Confirm GPTBot, ClaudeBot, OAI-SearchBot, PerplexityBot and Google-Extended can all read your site, which comes down to what your robots.txt allows. This takes ten minutes and it is binary. A site that blocks them is ineligible no matter what else it does, and a surprising share of sites block them without anyone having decided to.

If you appear occasionally, look at which questions. The pattern usually tells you something. Appearing on narrow, specific questions and vanishing on broad ones is the normal shape for a smaller business and is not a failure. Appearing on questions that do not describe what you sell suggests the assistant has misunderstood your category, which is a content problem you can fix directly by building a proper topic cluster around what you actually do.

If you appear consistently, start reading the sentences. Presence is not the whole story once you have it. Being listed fourth among similar options is different from being the recommendation, and a single presence-or-absence tally will not show you the difference. Read the actual answers.

In all three cases the underlying constraint is usually the same one: whether your pages are in the search results the assistant reads. That is a question of topical authority more than of any individual page, and it is the slowest thing to move.

Bottom line

Run a free check to find out whether there is a problem. Write your own questions before you track anything over time. Treat a single run as an anecdote and a repeated set as a measurement.

What to record, beyond present or absent

Most people record a tick or a cross and lose the information that would have told them what to do. Four extra columns cost nothing and change what the check is worth.

Position in the answer. Named first in a list of three is a different outcome from named fifth in a list of six. Over months, movement within the answer is usually visible before movement into it.

The words used about you. Assistants describe brands as well as listing them. Being called the budget option, the specialist one, or the one with slow shipping is information you will not get anywhere else, and it is often the first sign that a review problem or a shipping complaint has become part of how your category talks about you.

Everyone else who was named. This turns your check into a picture of your category. You find out who you are genuinely competing with for the mention, which is frequently not who you assumed.

Whether the engine searched. Some answers are written from what the model already knew and some from a live search. Where you can tell, note it. A category answered without searching is one where your content cannot reach the answer at all, and that is worth knowing early rather than after six months of writing. AI Overviews behave differently again, because they sit directly on top of Google rankings.

All four fit in a spreadsheet. The habit that makes them useful is filling them in every time rather than only when something looks interesting.

One column people wish they had added earlier: the date, obviously, but also which version of the question you asked. Question wording drifts when a set is maintained by more than one person, and a set that has quietly changed over six months cannot be compared with its own beginning. Freeze the wording, write it down somewhere separate from the tracker, and treat any edit to it as starting a new measurement rather than continuing the old one.

Why the answer changes when nothing changed

The first thing that unsettles people running these checks is that the same question, asked twice in an hour, produces different brands. Nothing is broken. Four things cause it, and knowing them stops you chasing changes that are not real.

The model is not deterministic. Assistants generate answers with a degree of randomness by design. Two runs of an identical prompt can take different paths through the same underlying material and surface different examples.

The search behind the answer moves. When an assistant searches, it reads whatever the results are at that moment. Results shuffle constantly, so a page that was fourth in the morning and seventh in the afternoon can drop out of what gets read.

The wording it searches with is its own. The assistant writes its own query from your question, and that translation is not fixed either. A slightly different phrasing retrieves a slightly different set of pages.

Engines update without announcing it. Retrieval behaviour changes, sometimes noticeably, with no changelog you can read.

The practical response is to establish your own noise floor before drawing any conclusion. Run your whole set twice in one afternoon and see how much the result moves on its own. Whatever that gap is, any change smaller than it carries no information. Most people are surprised by how large it is, and it usually settles the argument about how often the check is worth running.

If you would rather buy than build

Five questions that separate a useful tool from a confident-looking one.

Can I write my own questions? If the answer is no, the tool has made the most important decision for you and you cannot audit it.

How many engines, and can I see them separately? A blended number hides the split, and the split is usually the actionable part.

Does it keep the full answer text? Presence alone loses the difference between a recommendation and a dismissal.

How does it handle variance? A tool that samples once and reports a precise number is reporting noise with false precision. Ask how often it runs and whether it shows a range.

What does a zero mean? There is a large difference between genuinely absent and every engine call failing that day. If both show as zero, you will eventually spend a week investigating an outage.

Worth knowing before you compare vendors: scores are not comparable between tools, because each is asking different questions with different weights. Switching vendor resets your history, so the first choice matters more than it looks.

Frequently asked questions

What is an AI visibility checker?

An AI visibility checker is a tool that asks AI assistants a set of questions and records whether your brand is named in the answers. It exists because this is not visible in normal analytics: most people who read an AI recommendation never click a citation, so being recommended produces no measurable referral.

Are free AI visibility checkers accurate?

They are accurate enough to tell you whether you are anywhere at all, which is a genuinely useful first result. They are not reliable for tracking change over time, because a single run on a single day carries no information about direction, and because generated question sets tend to reflect the words on your website rather than the words your buyers use.

How often should I check AI visibility?

Monthly is a sensible cadence for most businesses. Answers vary between runs for reasons unrelated to anything you did, and that run-to-run variation is often larger than a week of genuine progress. Checking weekly mostly means reacting to noise. Use the identical question set each time, because changing the wording resets your history.

Why does my brand appear in one AI engine but not another?

Because they retrieve differently. Perplexity leans heavily on live search results, ChatGPT retrieves through Bing, and Google AI Overviews sit on top of Google rankings. A brand can be well established in the results one engine reads and absent from another. The split is informative: it usually points at where the underlying retrieval problem is.

What should I do if the check shows I am not visible at all?

Start with crawler access, because it is instant and binary. Confirm GPTBot, ClaudeBot, OAI-SearchBot, PerplexityBot and Google-Extended can all read your site. A site that blocks them is ineligible regardless of content quality, and many sites block them without anyone having decided to. Only after that is confirmed is it worth looking at retrieval and content.

MG
Written by

Matt is the founder of RunOctopus. He built All Angles Creatures from zero to page-1 rankings in reptile feeder insects using exactly this method. Turning a hard, entrenched niche into RunOctopus's proof store for programmatic SEO and AI search citation.

Connect on LinkedIn →

Ollie builds this for your store automatically

A complete launch build . 8 expert guides, 6 collection pages, and an interactive tool. Structured for both Google and AI search. Live on your store in 48 hours.

See What Ollie Builds →

See what Ollie builds before you pay. Cancel anytime.

Trusted by store owners in 20+ niches