We have to prove our worth in this world so whenever a new bit of marketing technology comes along we have to stick a number on it. SEO has rankings. Social media has followers and engagement rates. PR had reach, impressions and, yes, if we go back far enough, the dreaded AVE.

Now we have AI visibility. Ta dah!

And, almost inevitably, we have plethora of dashboards showing percentages, scores, rankings and share of voice designed to tell us exactly how visible an organisation is in ChatGPT, Gemini, Claude or whatever else comes along next. But there’s a problem in this world. AI answers are like shifting atoms.

Try this. Ask the same question on the same platform, but use slightly different wording or at a different point in time. Compare the answers. Are they the same? Not always. The sources used can change. The people or places the LLM recommends can change.

Steph and I were chatting through this as the vagaries make me worry and she agreed – well kind off. On the one hand she pointed out that traditional media measurement was dealing with a tangible, in that a story actually exists in a publication and the coverage was either positive, or it wasn’t.

In the world of AI we’re seeking answers in a world that’s far more sketchy. At times, measuring AI feels a little like trying to nail jelly to a wall. But that doesn’t mean we should give up.

It means two things:

1. We should be much more sensible about what we are looking at
2. We need to take our rose-tinted glasses off when we are looking at traditional measurement

Taking bearings rather than pretending we know our exact position

My dad is a sailor and he told me something about navigation that has stuck with me. Forgive me dad if I get this wrong, but he says when you’re heading home you don’t necessarily point the boat directly at the harbour. You give yourself some room to windward.

That way you can still make your way safely into port even if you get it slightly wrong – in fact Mother Nature will give you a helping hand. What you don’t want to do is be left fighting the prevailing wind to get back.

That feels like a reasonable way of thinking about AI measurement. We don’t need to pretend we can calculate the precise reputation of an organisation inside every possible AI answer. Right now, everything I have read and all the experts I have spoken to suggest that anyone who can give you a definitive number is probably not to be trusted.

What a brand needs is reliable bearings. Take the measure and you’ll understand the direction of travel and be able to take some tactical course correction if needed. Do not change course every time one reading moves. So if a brand appears consistently in a set of AI answers for 11 months and then disappears for one, it would be rash to conclude that its reputation had collapsed. It could simply be normal model variation.

AI measurement is best looked at as something that helps inform judgement rather than something that dictates strategy on its own. Or to put it another way (and this will melt your brain!):

The sea doesn’t have to stop moving before you can navigate it.

This is also where AMEC comes in

We are not making excuses for imperfect data. AMEC, the International Association for Measurement and Evaluation of Communication, and its approach to evaluation informs how both of us think about PR.

And here this year’s AMEC GEO Principles are particularly relevant. They say sensible stuff like AI-led discovery should be measured against communications objectives and stakeholder information needs; that AI outputs should be treated as indicators and tested across tools, prompts, markets and time; and that trustworthy, current evidence matters more than simply generating more visibility.

AMEC is clear on what we said earlier. It cautions against the single score. You need to look at the bigger picture. That makes sense in an ephemeral world.

Viva’s own measurement ethos has been built on the AMEC principles and forget about celebrating outputs and  start asking sensible questions about what the evidence means for organisational impact.

We think GEO needs this same approach. Our view on good measurement is clear. It requires a methodology that is structured, transparent and replicable, and that lets you show your working if somebody challenges the conclusion. So, for us the question shouldn’t be:

What is our AI visibility score?

It should be:

• What are we trying to understand
• What does the evidence tell us
• And what, if anything, should we do differently?

Share of voice only gets you so far

What interests some of the comms leaders we speak to is not pretty graphs and share of voice, but what AI is actually saying about their brand. And more than that, can having a handle on AI awareness help a communications team understand where misinformation, outdated information or weakly supported claims were being picked up?

That is a spark to a very interesting corporate communications question.

Do you know if an AI system is giving somebody a wrong or misleading answer about your client? Should you? What would you want to know? We reckon it’s something along these lines:

• What is it saying?
• Is it genuinely wrong?
• Is it outdated?
• What evidence has it used?
• Is the answer based on your own website, credible independent media, a government source, a customer, Reddit, an old PDF or something else entirely?

And then:

What should communications do about it?

For us that’s where the PR industry needs to get to sharpish. This isn’t about pretty dashboards per se – they have their place – but strategic advice based on sensible insights is where we come in.

Follow the evidence

We’d urge every PR consultant and in house professional to look to the AMEC GEO framework . It describes three different but connected areas.

What is happening upstream:  Reputation shaped by the likes of earned media, reviews, expert commentary, public records, owned material and all the other evidence AI can encounter.

What is accessible: Whether good information is available, current, understandable and supported by credible sources.

Then the downstream outputs: This is where we asked what the LLMs are saying, including presence, framing, sources, omissions, inaccuracies and risk.

We’ve been refining our own tool and it is based on the EMC principles because we started from a point of view that said look, we’re not really interested in saying: ‘Your AI visibility is 28.4%.’ It fails the ‘so what?’ test.

Instead, we have created something which gives a communications team something to work with. And we’ve grounded it in AMEC thinking as Steph has helped us build it and been out conscience.  So it looks more at stuff like this:

• AI consistently connects you with this capability, but not with another area you regard as strategically important.
• AI repeats this claim about you, but the source it cites doesn’t actually substantiate it.
• Almost all the evidence supporting this aspect of your reputation comes from your own website. There is very little independent corroboration.
• A competitor appears to own this subject because there is substantially more credible third-party evidence connecting them with it.

We will not reinvent the wheel

There are already plenty of platforms gathering this data. We’ve tried some.

Some have come from the world SEO, others created specifically for generative search. And of course, the existing names in the monitoring and measurement space have all been quick to add AI functionality.

They are seriously good. What they can do is automate the repetitive work of asking questions, capturing answers and identifying sources. That bit needs deep pockets and a good process.

But we think collecting the data and advising an organisation what to do with it are two different jobs. That is where our thinking at Viva has settled.

We are not an IT firm. We are not interested in building an enormous AI visibility dashboard.

But we are very keen on learning what evidence is out there and then and turning it into something a communications director can understand:

• What does AI think about us?
• Where is it getting that picture from?
• What is wrong, weak or missing?
• What should we do next?

This is guiding our thinking and giving our clients something meaningful in a complex world. We can help them navigate complexity and deliver last results.