Only 40% of AI-Quoted Stats Trace to a Real Study, Trakkr Finds

Ask five AI assistants what share of salespeople give up after one follow-up, and all five will tell you 44%. Trakkr, a company that tracks how AI tools cite brands, followed that number back through two links and found no study behind it. It’s one of four statistics every assistant in its new test repeated with nothing underneath.

What happened

On 28 October, Trakkr published Study 013, written by Mack Grenfell. The team asked ChatGPT, Perplexity, Gemini, Claude and Google’s AI Overviews 200 questions that call for a number. Then it followed 1,285 of the cited statistics back to where they came from, up to five pages deep.

The results split almost evenly. 40% reached a study that says the same thing. Another 8% reached the study, but the number or its context had shifted on the way. A further 40% stopped at a source nobody could check, because it was locked, gone or silent on the figure. The last 11% led nowhere.

ChatGPT did best, with 57% of its statistics sourced, and it quoted the fewest at 1.3 per answer. Claude quoted the most, 6.8 per answer, and landed at 38% alongside Gemini. Perplexity scored 46% and AI Overviews 39%. Trakkr says those four sit too close together to rank.

Why it matters for anyone who writes with AI

The assistants mostly aren’t inventing numbers. The page they cited contained the figure 82% of the time. The trouble is one step back: only 21% of cited pages were the research itself, and more than half were company blogs.

That’s a trap for students drafting essays and freelancers dropping stats into client posts. A citation looks like proof. It only shows where the AI found a number, not where the number was born.

Agreement doesn’t help either. On the 10 questions where all five assistants gave the same figure, Trakkr traced five trails. Two were sourced. Of the 29 statistics quoted by three or more assistants, 11 held up.

The 44% trail shows how this happens. AI Overviews cited a sales firm’s stats page from 2020, whose “source” link pointed to a freelance writer’s sample article, online by 2017, with the number in its headline. After that, nothing. “It’s a number still walking around with nothing underneath it,” Grenfell writes. Only ChatGPT told users the evidence was unclear.

A writing sample became a fact.

If you sell writing for a living, catching the errors AI states with total confidence is now part of what clients pay for.

How a number drifts

Real studies get bent too. Of the statistics that reached a source, 17% had changed on the way. The digits moved only about one time in four. Usually it was the context: what was measured, the year, or who was asked.

The clearest case began in 2004, when Sillence and colleagues studied how 15 British women judged health websites. Of their comments about rejecting a site, 94% concerned design. Twenty years of retelling turned that into “94% of first impressions are design-related,” which AI Overviews now states as fact for every website.

Some zombies are older still. The claim that 80% of sales need five or more follow-ups is credited to a sales association, and its successor says the closest thing in its archive is a 1942 survey of fewer than 40 members of one local chapter.

The assistants don’t even agree with each other. Asked on 25 October what share of marketers use automation tools, AI Overviews said 49 to 51%, ChatGPT 62%, Gemini 70 to 79%, Perplexity 81% and Claude 96%.

This isn’t the first warning. An EBU and BBC study found 45% of AI answers to news questions had at least one significant issue, with sourcing the most common failure. NPR was one of 22 public broadcasters that took part.

Trakkr’s study has limits worth stating. The company sells AI citation tracking, so the finding suits its business. It asked each assistant once, through the company’s API with web search on and a US location, and the consumer apps can answer differently. Its AI judge alone rated just 23% of trails sourced. Hand checks on 150 trails lifted that to 40%.

What to watch next

Trakkr’s advice for readers is simple. Follow the citation, then follow it again. Check who was asked, how many, and when. Don’t treat five matching answers as five sources.

For publishers, the study asks for the method next to the number and findings kept out from behind sign-up forms, since gated reports were a common dead end.

The study carries no response from OpenAI, Google, Anthropic or Perplexity. The test to watch is whether any assistant starts flagging thin evidence the way ChatGPT did on the 44% claim, instead of quoting a 2017 writing sample as settled fact.