Tracking Brand Mentions in ChatGPT, by Hand

The whole method, in five steps

You can establish whether an assistant names your business, how often, and in what terms, in about an hour, with a spreadsheet and no budget. The method is not sophisticated and the sophistication is not what makes it useful. Consistency is, and consistency is free.

Write six to ten questions in the language a buyer would use. Open a clean session with no history attached to you or your company. Ask each question five times, each in a separate fresh conversation. Record five things about each asking. Then do exactly the same thing on the same day next month, and the month after.

That is the entire method. Businesses spend a great deal on tools that do this, and the tools are worth it for one reason which is that they keep doing it when everybody is busy. The measurement itself is not hard and it is not proprietary.

What makes the difference between a useful record and a wasted hour is four traps, all of which are easy to avoid once named and all of which are extremely common in manual attempts.

Writing the questions

The question set is the only genuinely difficult judgment in the whole exercise, and getting it wrong makes everything downstream misleading rather than merely imprecise.

The trap is using your own vocabulary. Businesses test with the category name they use internally, which is frequently a term the industry adopted and customers never did. Asking about a phrase nobody uses returns answers written by whoever else uses that phrase, which tends to be other suppliers rather than anything a buyer would find, and the reading says almost nothing about your visibility to real customers.

Build the set from language you did not write. Open your support tickets and read how people describe the problem. Read the subject lines of inbound inquiries. Read a handful of reviews, including of competitors. Listen to how a salesperson repeats back what a prospect just said. Any of those produce phrasings that differ from marketing vocabulary, usually more than expected.

Cover three kinds deliberately. One or two that name your company directly, which are the easiest to win and the first to move. Several specific questions matching the problems your best pages address. And the broad category question with the most obvious phrasing, which you will probably not win for a long time and which is worth watching because it tells you who dominates.

Then fix the set and do not improve it. A set that keeps getting better is a set whose history cannot be compared with itself, and comparability is the only thing this exercise produces that opinion could not.

The four traps

Each of these makes a manual reading worthless, each is invisible while you are doing it, and three of the four flatter you rather than damaging you, which is why nobody catches them.

Asking once. The same question can return different names between runs. One asking recorded as a fact is a coin flip written into a spreadsheet, and a year of single askings is a year of coin flips with a trend line drawn through them. Five per question is a reasonable floor.

Asking in one conversation. Answers are shaped by what came earlier in the same thread. Ask your ten questions in sequence and question seven inherits everything from questions one to six, including any mention of your own company. Every question needs a fresh conversation.

Using your own account. An account with history, or any stored context about you or your work, is primed and will produce readings that do not reproduce anywhere else. This is the most common contamination and the hardest to notice because the result is flattering.

Paraphrasing between readings. Typing the question from memory each month means the question drifts, and by month six you are measuring a different thing and comparing it with the old numbers. Store the questions and copy them verbatim every time.

The five things to record

Recording only whether you appeared throws away most of what the reading contains, and the extra columns cost seconds because you are already reading the answer.

Presence, as a count out of the askings. Three of five, not yes. The proportion is the number that moves and the only one worth trending.

Every other name that appeared. Including ones that are not competitors. Publications and communities show up constantly, and the composition of that list tells you whether your category is contested between suppliers or dominated by third parties, which is the single most strategically useful thing in the whole record.

The wording around your name. Recommended, listed among several, mentioned as the cheaper option, mentioned with a caveat. These are different commercial positions that all count as present, and the drift in this language usually moves before the presence proportion does.

How many names the answer contained. Being absent from a one-name answer and absent from a six-name answer are different findings, and without this column your proportions are not comparable across question types.

The date and the question verbatim. Obvious, routinely skipped, and the omission that makes an archive uninterpretable a year later. Record which assistant too, even if you only watch one today, because the day you add a second the old rows need to be attributable to the first.

Keep the raw answers as well as the summary for the first few months. You will not know what matters until you have read a few dozen, and rereading old answers with a better idea of what to look for is only possible if you kept them. Text is cheap to store and impossible to recover.

Reading your first result

The first reading is a baseline rather than a grade, and reacting to its absolute value is reacting to where you happened to start.

Almost every business that has never done this work comes back low, and low is the normal condition rather than a verdict. What matters is the composition of what you found, not your own number.

If publications and community discussion dominate the names, your own pages are not the binding constraint and a content program will not fix it. The realistic work is being referenced in those places, which has no schedule and is uncomfortable to plan.

If specialist sites dominate, the category rewards depth and your effort can change your position. This is the most encouraging finding available.

If suppliers including your competitors dominate, the category is commercially contested and being named is a function of being reachable, quotable and deep. That is a fair fight, winnable with ordinary work, and the reading has just told you the work is worth doing. Our page on why a brand is not in ChatGPT works through the causes in the order worth checking.

Making it survive month three

Manual monitoring does not fail because it is inaccurate. It fails because it stops, and it stops in month three when somebody is busy.

Give it a named owner rather than assigning it to whoever suggested it. Put it on a fixed day of the month in a calendar. Keep it to an hour, because a process that takes a morning becomes quarterly and then stops.

Store the questions, the method and the reason each question was chosen in one place next to the data. That last field is the one that prevents the slow death, because a question whose purpose has been forgotten is the first one somebody edits when it produces an uncomfortable answer.

And publish the flat months. A record that only appears when the news is good is a record nobody trusts when it matters, and credibility is the only currency available here given that none of this can be attributed.

When a tool is finally worth paying for

There are three honest triggers and none of them are about accuracy.

When you have skipped two readings. That is the signal that the manual process will not survive, and paying somebody to be consistent is a reasonable response to a demonstrated fact about your own team rather than an admission of anything.

When you need to watch more than three assistants. The arithmetic compounds fast. Ten questions asked five times across four assistants is two hundred askings a month, which stops being an hour and starts being a job.

When somebody outside the team needs to trust the number. A record kept by the person whose work it measures is awkward, however honestly kept, and an external source removes an argument you should not have to have.

What is not a trigger is wanting a nicer chart. Our page on visibility monitoring covers what to require of a tool if you do buy one, and the first requirement is that it asks each question more than once.

What to do about awkward results

Three results come up often enough to be worth having an answer ready for, because each one produces a meeting where somebody wants to act immediately.

A competitor named consistently and you never. The instinct is to study their pages. Resist it until you have checked your own access, because if nothing can fetch your site then their page structure teaches you nothing you can use. Once access is confirmed, their pages become genuinely useful, particularly if you can see which passage was quoted somewhere that shows sources.

Something factually wrong about you. This is uncomfortable and it is usually traceable. It is drawn from material somewhere, frequently your own site, sometimes an old page or a third-party description that has gone stale. The response is to find the source and correct it rather than to argue with the reading, and businesses are often surprised to discover the wrong statement is one they published years ago.

Named, but described in terms you would not choose. Being called the budget option or the one for small teams is a positioning fact rather than a visibility problem, and it is arriving in front of buyers whether or not you monitor for it. It is one of the more valuable things this exercise surfaces and it cannot be fixed by writing more pages.

In all three cases the discipline is the same. Confirm it across several readings before acting, because a single answer is a sample, and reorganizing a quarter around one screenshot is the commonest expensive mistake in this work.

The honest trade-off

Doing this by hand costs an hour a month and produces a record that only you have, collected by the person whose work it reflects. Both of those are real weaknesses.

Against that, running it manually for the first few months teaches you something no dashboard conveys, which is how much the answers vary on their own. A team that has watched the same question answer three different ways in ten minutes never again treats a single reading as a fact, and that instinct is worth more than the readings themselves.

The sensible path is to start by hand, learn what the variation looks like, and buy consistency later if the manual process demonstrates that it needs buying. Starting with a tool skips the education and produces a team that trusts a number it does not understand, which is a worse position than having no number at all because it feels like knowledge.

Bottom line

Six to ten questions in buyer language, five askings each, a clean account, a fresh conversation every time, five columns recorded, same day every month. That is the whole method and it costs an hour.

Avoid the four traps. Asking once, asking in one thread, using a primed account and paraphrasing the questions all produce readings that are wrong in the flattering direction, which is why they survive.

Read the composition of names before you read your own number. What else appears tells you which strategy is even available to you, and that is worth more than knowing where you currently stand.

And give it a named owner with an hour booked in the calendar. This exercise does not fail on method. It fails in month three when everybody is busy, and the only defense against that is somebody whose job it explicitly is.