AI Visibility Score: What a Single Number Can Carry

What a visibility score actually is

A visibility score is a summary of how often you were named across a set of questions somebody chose, and it is a property of that question set at least as much as it is a property of your business. Change the questions and the number changes without anything about you having changed at all, which is the single most important thing to understand before anybody puts one in a report.

This is not an argument against having one. A fixed set of questions, asked consistently, produces a number whose movement genuinely means something, and having one number rather than forty rows is what makes the work legible to people who are not going to read forty rows. The problem is not the summarizing. The problem is that the summary gets detached from the method that produced it, travels into a slide, and starts being treated as a measurement of the business rather than a measurement of a chosen sample.

The test for whether a score is being used well is simple. If somebody can state, without looking anything up, how many questions it covers, how many times each was asked, and on which assistants, then it is being used as a measurement. If they cannot, it has become a number, and numbers detached from their method are argued about rather than acted on.

The four layers underneath it

Every score is built from four decisions stacked on each other, and only the top one is usually visible. Working down through them is how you find out whether a particular number deserves any weight.

The question set is the foundation and the only genuinely difficult judgment in the whole apparatus. It is a model of what your buyers ask. Built from your own marketing vocabulary it will flatter or damn you for reasons unrelated to your actual standing, because the phrasings a company uses about itself are rarely the phrasings anybody uses to look for it. Built from support tickets, sales notes and the words customers actually used, it becomes a reasonable proxy for demand.

The number of askings decides whether the reading is a measurement or a coin flip. The same question can answer differently between runs, so asking once and recording the result is recording a single draw from a distribution. Five askings per question per reading is a reasonable floor and almost nothing in the market does it.

The counting rules decide what presence means. Does a passing mention count the same as a recommendation. Does being named third count the same as being named first when the answer has no order. Reasonable people differ here and the differences move the number substantially, which is why two tools can report different scores for the same business on the same day.

The weighting decides whether all questions count equally. Weighting the broad category question heavily will produce a low, flat number for years. Weighting long-tail questions heavily will produce an encouraging one that overstates your position. Neither is wrong and both should be disclosed.

How to read a score that moved

A number that changed is a prompt to investigate rather than a result, and the investigation should run in a fixed order because the boring explanations are also the most common ones.

First, check whether the question set changed. This sounds absurd and it is the leading cause of unexplained movement. Somebody edited a question that seemed unfair, a vendor updated its defaults, an assistant was added or dropped from the panel. All of those move the number without anything about your business having altered, and none of them are usually announced.

Second, check whether the movement is concentrated or spread. One question flipping from named to not named is exactly the variation you would expect from an unchanged state. Several questions moving in the same direction in the same reading is much harder to explain as chance and is worth taking seriously.

Third, look for a cause in the right window. A change three weeks after an access rule was corrected is plausible and probably yours. A change with nothing behind it is more likely the assistant than you, which happens regularly and is not something you can act on however much you would like to.

Fourth, wait for the next reading before restructuring anything. Two consecutive readings agreeing is a state. One is a sample. The cost of waiting a month is almost always lower than the cost of reorganizing a content program around noise, and the temptation to act immediately is strongest exactly when the movement is largest.

Why you cannot compare your score with anybody else

Scores from different sources are not comparable, and treating them as though they were produces confident nonsense that is very hard to walk back once it is in a board pack.

Two vendors will use different question sets, different numbers of askings, different counting rules and different weightings. Four independent variables, none of them standardized, all of them moving the output. A business scoring forty with one tool and sixty with another has learned something about the tools rather than about itself.

The same applies to competitor scores computed by a vendor. Those are produced by asking the same questions and counting who appears, which is a legitimate exercise, and the resulting league table is a ranking of performance against that vendor's chosen questions. It is genuinely useful for spotting who dominates a category and genuinely useless as an absolute statement about anybody's marketing.

What is comparable is your own number against itself, over time, from an unchanged set. That is a real measurement and it is the only one on offer. Everything else in this area that looks like a benchmark is a comparison of methods wearing the clothes of a comparison of businesses.

When to ignore the score entirely

There are three situations where the score is actively misleading and the underlying readings should be looked at instead.

When you have just started, the score is a baseline rather than a grade, and reacting to its absolute value is reacting to where you happened to begin. A low first score is the normal condition of a business that has never done this work, and it tells you nothing about whether the work will succeed.

When the category has entrenched sources, the score will be flat for a long time while genuine progress is happening in the questions you can actually win. A single number that averages the winnable questions with the unwinnable ones will move very slowly and will be read as failure. Splitting the reading into questions that name you directly, specific long-tail questions and the broad category question shows the progress that the average is hiding.

And when something structural has changed, such as an access rule being fixed, the score jumps for a reason that has nothing to do with the content program it will be credited to. Attributing a step change to whatever was happening at the time is how organizations learn the wrong lesson and repeat it for a year. The defense against this is a written log of structural changes kept alongside the readings, so that anybody reading the history later can see that the jump in month three sat three weeks after a firewall rule was corrected rather than three weeks after a campaign launched.

What to report instead of a score

A score is a convenience for the reader and a hazard for the person presenting it, and there is a reporting format that keeps the convenience without the hazard.

Lead with the denominator attached. Named in eleven of forty-five askings across nine questions and three assistants is a sentence somebody can interrogate, and it survives being questioned by somebody who was not in the room when the method was agreed. Twenty-four percent is the same fact with the evidence stripped off, and it invites the one question you cannot answer, which is twenty-four percent of what.

Show the three question groups separately rather than blended. Direct-name questions, long-tail questions, and the broad category question behave completely differently and move on different timescales. Blending them produces a number that moves slowly and explains nothing.

Show who else was named. A reader who sees that the same four sources dominate every reading understands the strategic position immediately, in a way that no number about your own presence conveys. This is consistently the most useful line in the whole report and it is the one most often omitted because it is about somebody else.

And publish every reading, including the flat ones and the bad ones. A program that only reports its good months will not be believed in the month it most needs to be, and credibility here is the only currency available given that none of it can be attributed.

Building your own score

If you are going to have a score, building it yourself is usually better than buying one, because the value is in the method being yours and understood rather than in the arithmetic being sophisticated.

Write six to ten questions in customer language, taken from support tickets, sales notes and reviews rather than from your own site. Cover the three groups deliberately: one or two that name you, several long-tail questions matching pages you care about, and the broad category question you probably will not win.

Ask each question five times, on the same day each month, across at least two assistants including one that shows its sources. Record the proportion that named you, the other names, the credited sources and the wording. Date every row and record the assistant.

Compute the score as the simplest thing that could work, which is the proportion of askings that named you. Resist weighting until you have a year of data, because a weighting scheme invented at the start encodes assumptions you have not tested and it will be defended long after it should have been revised.

Our page on AEO monitoring covers the sampling discipline, and benchmarking covers choosing a comparison set if you want one.

The honest trade-off

The case for a score is that forty rows of readings do not travel and one number does. Work that cannot be summarized does not get funded, and a program with no headline figure will lose its budget to one that has an impressive-looking chart, regardless of which is doing better.

The case against is that a single number invites exactly the misuse it is most vulnerable to. It gets compared with other companies' numbers, it gets treated as a grade rather than an index, and it gets explained month to month when most of its movement has no explanation. Every one of those failures is predictable and none of them are prevented by better arithmetic.

The resolution that actually holds is to keep the score and never let it travel without its denominator. It is more words on the slide and it is the difference between a measurement and a rumor.

Bottom line

A score summarizes the questions somebody chose. Its movement over a fixed set is meaningful. Its absolute value is not comparable with anybody else's and barely comparable with your own from a different set.

When it moves, check the question set first, then whether the movement is spread across several questions, then whether there is a cause in the right window, then wait for a second reading before acting.

Report the denominator, split the three question groups, show who else was named, and publish the flat months. Those four habits are what turn a number into evidence, and none of them require better tooling.

If you inherit a score from a vendor rather than building one, ask the four questions that reveal its foundations before you put it in front of anybody. How many questions, in whose words, asked how many times, on which assistants. A vendor who answers those crisply is worth listening to. A vendor who treats them as implementation detail is selling a number rather than a measurement, and the number will be the first thing questioned in the meeting where it matters.