How Much Does a Number Have to Move Before the Change Is Real?

AI platforms are probabilistic. Ask the same question twice and the answer can change, even with identical inputs. The SparkToro/Gumshoe.ai study measured exactly how much: across 2,961 runs of 12 prompts, the odds of getting the same brand list twice were under 1 in 100. The odds of getting the same list in the same order were roughly 1 in 1,000.

One number survived that variability. Visibility percentage, the share of runs where a brand appeared at all, held up across repeated runs. Rank position did not. A single AI visibility reading, taken once, reflects one draw from that distribution rather than a fixed position for the brand.

When a visibility number shifts between two reads, there are three possible explanations:

All three look identical in a simple before/after comparison. The only way to tell them apart is knowing, in advance, how much a metric moves on its own when nothing else has changed.

That known range of natural movement is the variance floor. AIVO establishes it per dimension (presence, sentiment, accuracy, and others) during the baseline read, before any strategy work begins. A later reading counts as real movement, up or down, only when it falls outside that floor. Anything inside it is treated as the same result.

Running one query a day builds a long historical record, but it does not establish where a brand stands today with confidence. Every observation is tied to the specific day it was taken. A single daily draw mixes brand-driven movement and platform-driven movement into the same number, on the same axis, with no way to separate the two from that one data point.

Presence tolerates this reasonably well, since it is a stable rate across many runs. Accuracy and sentiment are more sensitive to which specific run gets sampled, which is why both need a variance floor and same-day sample sizes large enough to place a confidence interval around the result, rather than a daily trend line alone.