Measuring AI visibility without fooling yourself

The temptation, when you first look into this, is to open ChatGPT and type your company name. You get a flattering answer, you feel reassured, you move on.

That measurement is worth nothing, for three separate reasons.

The three traps of testing by hand

You are asking the wrong question. Typing "what do you think of Dupont & Sons" already puts your name in the question. The model only has to elaborate. Your customers do not know your name: they ask "which plumber in Lyon", "which payroll software for 20 staff", "who does laser cutting in the region". Those are the questions you need to be in.

Your session is not neutral. Conversation history, memory turned on, location, a signed-in account: your answer is personalised. Your prospect's is not. A reading taken from your usual account describes your bubble, not the market.

One answer is not a measurement. Ask again ten minutes later and the wording changes, sometimes the names too. A single reading is a draw, not a statistic.

What to fix before measuring

A useful measurement needs four things settled in advance.

A stable question set. Between fifteen and forty depending on the size of the business, written the way a customer would ask, without your name in them. That list must then stop moving: change it at every reading and you are no longer comparing anything.

The engines, named. Three is enough to start, provided at least one of them searches the web.

A cadence. Monthly suits most businesses. Weekly makes sense when the market moves fast, or during active work when you want to see the effect. Daily is pointless: noise will swamp the signal.

And answers kept in full, dated, with the engine named. Not a score, not a summary: the text. In six months, that is the only thing that will let you say what changed.

The three numbers that actually matter

A well-run reading produces three, and they say different things.

Citation rate: across all answers, in how many are you named. That is the headline indicator. It moves slowly.

Share of voice: among all the company names cited, what proportion is yours. A 30% citation rate means something different depending on whether answers name two competitors or twelve.

Sources: which domains the engine cites. That number is not about you, it is about the terrain. It is the most actionable of the three, and the most often ignored.

I am wary of "AI visibility scores" that blend all of this into a mark out of a hundred. The blend sells well and cannot be steered: when it drops, you do not know what to do.

Reading a change without over-reading it

Two readings a few points apart say nothing. That is noise, and it is normal: these engines are not deterministic.

What counts is the direction over several consecutive readings, and sharp changes. A competitor going from absent to present in half the answers is worth looking at. A new domain appearing all at once in the sources is worth looking at too.

And you have to accept the part that is not up to you. An engine updates its model, changes how it searches, and your numbers move without you having done anything. Which is why it pays to track two or three competitors as well: if they drop at the same time you do, the problem is not yours.

How long before anything moves

Asked every time, so here is the honest answer: it depends on the engine.

On engines that search at question time, work on third-party sources can show within a few weeks. On those answering from training, the delay runs into months, sometimes longer, and there is no lever to speed it up.

Which is why measuring early has value even before anything has been done. The first reading is not there to judge you. It is there as the zero point.