Skip to main content

AI Visibility | 12 min read

AI Visibility Monitoring: Continuous, Not a Snapshot

By ยท Updated ยท 12 min read

Three products that get sold as one

A checker, a monitor and an analytics view answer three different questions, and the category sells all three under one label. Knowing which you are buying prevents paying monthly for something you needed once.

A checker answers where you stand today. It asks a set of questions, reports whether you were named, and stops. It is the right thing to buy before any work starts and the wrong thing to buy repeatedly.

Monitoring answers whether it is moving. It requires a fixed question set, repeated asking, and enough history for a change to be distinguishable from variation. It is worth paying for once you are actually changing something.

Analytics answers what the readings mean in aggregate, which is a reporting layer on top of monitoring rather than a separate capability. It is where the score, the trend and the competitive comparison live.

Most disappointment in this category comes from buying the second when you needed the first, then cancelling four months later because the number did not move. It did not move because nothing was being done to it.

Before you buy

Ask what you would do differently if the number moved. If there is no answer, you need a check rather than a monitor.

Check against monitor What each one answers and what it costs. A one-off check Answers where you stand now Cheap, sometimes free Cannot show movement Right before any work starts Wrong as a standing purchase Continuous monitoring Answers whether it is moving Ongoing cost, ongoing value Needs a fixed question set Right once work is underway Premature if nothing is changing
Buying monitoring before you have anything to move is paying monthly to watch a number that is not being acted on.

When monitoring genuinely earns its cost

Monitoring is worth paying for in three specific situations and is premature in most others.

When work is actually underway. If pages are being rewritten and access is being corrected, monitoring is how you find out whether it worked. Without it the program is unfalsifiable and will be judged on opinion.

When something can break silently. Access rules change without announcement, because they are changed by people solving unrelated problems. A monitor that watches whether you are still reachable catches a regression that would otherwise cost months before anybody noticed.

When you need to report to somebody. If a budget depends on evidence, a consistent monthly reading against a fixed set is the only defensible artifact available in this channel.

Outside those three, a check twice a year tells you most of what monitoring would, at a fraction of the cost and attention.

What to require of a monitor

The category varies enormously in quality and the differences are checkable before you commit.

Multiple askings per question per reading. This is the single most important requirement and the most commonly missing. A tool that asks once and reports a result is reporting a coin flip.

A fixed question set that you control. If the vendor changes the questions, your history becomes uncomparable and you will not be told.

Raw readings, not only a score. A computed number is convenient and lossy. You want the underlying answers so you can see who else was named and how the wording changed.

Coverage of more than one assistant. They behave differently and a change can surface in one long before another. Watching one and generalising produces confident wrong conclusions.

The other names, not just yours. The stability of the competing set is the most strategically useful thing in a reading and many tools discard it.

The costs nobody quotes

The subscription is the smallest cost of monitoring, and the others are worth planning for.

The first is attention. A monthly reading that nobody reads is money spent on a file. Somebody has to own it, look at it, and be willing to say that nothing moved this month, which is a duller job than it sounds and the first thing to lapse.

The second is the temptation to react. A number that updates invites explanation, and most month-to-month movement has no explanation. Teams that watch closely tend to act on noise, which is worse than not watching at all.

The third is the reporting relationship. Because none of this is attributable, whoever presents the number is asking to be trusted on a measurement they collected. That is survivable only if the readings are published consistently, including the bad ones.

None of these are arguments against monitoring. They are arguments for deciding in advance who owns it, how often it is looked at, and what size of change is worth reacting to.

Doing it yourself

A perfectly serviceable monitor is a fixed list of questions, a schedule, and a spreadsheet, and there is a real case for starting there.

Write your six to ten questions in buyer language. Ask each one several times on a set day each month across more than one assistant. Record the proportion that named you, the other names, the credited sources where shown, and the wording. Date every row.

The whole exercise is an hour or two a month. What it costs is consistency, and consistency is exactly what a tool sells. A team that will genuinely do this every month for a year does not need to buy anything. Most teams will not, which is a fair reason to pay somebody.

Starting manually has one advantage no tool provides, which is that you see the variation with your own eyes. A team that has watched a question answer three ways in ten minutes never again treats a single reading as a fact. Our walkthrough on tracking mentions by hand is enough to run the first readings.

What to do when the number moves

A reading that changes is a prompt to investigate rather than a result, and the investigation has a sensible order.

First, check whether the question set changed. This sounds absurd and it is the commonest explanation, because somebody edited a question that seemed unfair or a tool updated its defaults.

Second, check whether the change is across several questions or one. One question flipping is expected. Several moving together is worth explaining.

Third, look for a cause in the right window. A change three weeks after an access fix is plausible. A change with nothing behind it is probably the assistant rather than you, which happens regularly and is not something you can act on.

Fourth, wait for the next reading before acting. Two consecutive readings agreeing is a state, one is a sample, and the cost of waiting a month is almost always lower than the cost of restructuring a program around noise.

Four questions that sort the vendors

The category is young enough that capability varies enormously behind similar marketing, and four questions separate them quickly.

How many times do you ask each question in a reading? If the answer is once, the product reports a single draw from something that varies, and every trend line it draws is decoration. This one question eliminates a lot of the market.

Can I see the raw answers? A vendor that shows only a computed score has discarded the most useful part of the reading, which is who else was named and how you were described.

Who controls the question set? If the vendor can change it, your history can become uncomparable without anybody telling you, and the number will move for reasons that have nothing to do with your site.

What happens when an assistant changes? These systems change on their own and a reading can move with nothing on your side altering. A vendor who has thought about this will say so plainly. One who has not will attribute every movement to your performance, which is flattering and useless.

What to do in the first month

The first month of monitoring is different from every month afterwards and it is worth treating deliberately.

Take more readings than you intend to sustain. Three or four in the first month establishes how much your numbers move on their own, which is the single most useful thing you can learn and is impossible to recover later.

Resist acting on any of it. The first month is calibration, not measurement. A team that restructures its content plan after its first reading has acted on a sample and will spend the next quarter unwinding it.

Write down what you expect before you look. Predictions that turn out wrong are the fastest way to correct a mistaken model of your own position, and most businesses are wrong in the same direction, which is assuming they are better known than the readings show.

And fix the question set at the end of the first month rather than the start. One month of readings will show you which questions are useless because everybody wins them or nobody does, and fixing the set after that is the last chance to change it without breaking comparability.

The honest trade-off

Monitoring converts an unfalsifiable program into a measurable one, and it does so by adding a monthly cost and a monthly obligation to look at something that usually has not changed.

The strongest argument for it is regression rather than progress. Access rules break silently, and a business that was visible and quietly stopped being visible will not find out any other way. That alone can justify it.

The strongest argument against buying it early is that it measures a thing you are not yet changing. A number watched for six months while nothing is being done to it is a subscription funding a chart.

The sensible sequence is a check first, then work, then monitoring once the work is underway. Buying in that order costs less and produces a number that means something when it arrives.

The exception worth naming is a business that is currently visible and cares about staying so. There the argument inverts. You are not watching for progress, you are watching for a silent regression, and the value of the monitor is entirely in the month it catches something. That is insurance rather than measurement, it should be priced and judged as insurance, and it justifies a much lighter setup than a program trying to demonstrate improvement. One question set, one reading a month, one alert if presence drops across several questions at once.

The three signals worth alerting on

Most monitoring produces more data than anybody acts on, so it is worth deciding in advance which three changes are worth an interruption.

Presence dropping across several questions at once. A single question flipping is expected. Several dropping together in one reading almost always means something structural, and the first thing to check is whether an access rule changed rather than anything about content.

A new name appearing consistently. When a source you have never seen starts being named across several questions, somebody has done something that worked. That is the most useful competitive intelligence available in this channel and it is free.

The wording around your name changing. Going from being recommended to being listed as an alternative is a real change that a presence count will not show at all. This is why the raw answers matter more than the score, and why a tool that discards them has thrown away the interesting part.

Everything else is worth recording and not worth interrupting anybody about. A monitor that alerts on ordinary variation trains its audience to ignore it, and then the alert that mattered is ignored too. This is the same failure that makes people filter their own error emails, and it is worth designing against from the first month rather than after it has already happened.

Bottom line

A check tells you where you stand. Monitoring tells you whether it is moving. Buy the first before any work and the second once work is underway, not the other way round.

Require multiple askings per question, a fixed set you control, the raw readings rather than only a score, and the other names alongside yours. A tool that asks once and reports a number has automated the central mistake in this discipline, and no amount of interface quality compensates for it.

And decide before you subscribe what you would do differently if the number moved. If there is no answer to that, you are buying a chart rather than a measurement, and the subscription will be cancelled in month four for reasons that were true on day one.

If you are already visible and want to stay that way, treat this as insurance instead. One question set, one reading a month, one alert if presence drops across several questions at once. That is a much lighter commitment than a program trying to prove improvement, and it catches the failure that would otherwise cost you a quarter before anybody noticed.

Frequently asked questions

What is the difference between an AI visibility checker and monitoring?

A checker answers where you stand today and stops. Monitoring answers whether that is changing, which needs a fixed question set, repeated asking and enough history to separate a real change from ordinary variation.

Do I need monitoring if I am not doing any work yet?

Usually not. A number watched while nothing is being done to it does not tell you anything actionable. A check twice a year is a better fit until work is underway.

How many assistants should be monitored?

More than one. They behave differently and a change can surface in one well before another, so watching a single assistant and generalising produces confident wrong conclusions.

What is the most important feature to insist on?

Multiple askings of each question in each reading. A tool that asks once and reports a result is reporting a single draw from something that genuinely varies.

Can monitoring show me whether AI visibility drove revenue?

No. Someone who reads your name and then searches for you arrives as direct traffic with no trace of the assistant. Monitoring measures presence, which is not the same as return.

What is the strongest reason to monitor?

Catching a silent regression. Access rules change without announcement, and a business that quietly stops being reachable has no other way of finding out until a quarter has passed.

MG
Written by

Matt is the founder of RunOctopus. He built All Angles Creatures from zero to page-1 rankings in reptile feeder insects using exactly this method. Turning a hard, entrenched niche into RunOctopus's proof store for programmatic SEO and AI search citation.

Connect on LinkedIn →

Ollie builds this for your store automatically

A complete launch build: 8 expert guides, 6 collection pages, and an interactive tool. Structured for both Google and AI search. Live on your store in week one.

See What Ollie Builds →

See what Ollie builds before you pay. Cancel anytime.

● Trusted by store owners in 20+ niches