Can You Trust the Statistics on Trust?

The ZDNet headline is striking: “Americans trust Amazon and Google more than Oprah (and Trump).”

Wow! Ring the alarm bells, right?

The article goes on to cite the underlying study, from Morning Consult, called Most Trusted Brands 2020. Those brands range from the US Post Office to Hershey and Cheerios, from “religious leaders” and labels on food packaging to Oprah and Warren Buffett, from extreme weather warnings to Tom Hanks.

Both make a big deal about the validity of the study, averaging 16,700 interviews covering 2,000 brands. With such an impressive load of statistics, who could doubt the findings?

Me, for one. And so should you, after a minute’s reflection.

In fact, these ‘findings’ are about as meaningful as the results of a poll asking, “Which is nicer: a rhinoceros or a tricycle?”

Blurred Lines

The problem doesn’t lie in the statistics – it lies in the question being asked.

In this particular survey, the single question asked was, “How much do you trust each brand to do what is right?” The answer range was a lot, some, not much, not at all, or don’t know.

Whenever you encounter a study that offers to compare trust, you should ask yourself – trust to do what? The more specific the answer to that question, the more informative it is. The vaguer the answer, the less meaningful it is.

For example, “I trust Cheerios to avoid food contamination” would be fairly informative. You could compare the Cheerios score to Wheaties’ score. But you couldn’t compare it to Oprah or the Post Office, simply because neither has much to do with food contamination.

In this case, the question is “to do what is right.” But what does that even mean? Is there any “right thing” that covers both Warren Buffett and a weather forecast?

Comparing “the right thing” for religious leaders with “the right thing” for food packaging labels is not just apples and oranges: it’s apples and Sherman tanks. Any definitional overlap is at such a high level of abstraction as to render it nearly meaningless.

Proper Stats

Statistics like these do have two uses.

First, they are great clickbait. But, that’s the problem.

More seriously, they actually are good for tracking comparisons over time. If there is a decline from 2018 to 2020 in people’s ratings of how likely Tom Hanks is to “do the right thing,” that reflects a real shift in people’s perceptions of “America’s dad.” But comparing Hanks to Hershey? That’s just silly.

The ways people actually use words is an anthropological fact, one we can’t change. But that’s no reason responsible researchers shouldn’t use words with care. And this is not a thoughtful or careful use of the word ’trust.’

In this case, they’d be far better off talking about ‘brand image,’ or ‘reputation,’ or simply ‘positive feelings.’

For example, the ZDNet article says, “There was but one [brand] that was trusted ahead of Amazon and Google: the United States Postal Service.”

But – to do what?

If the answer is “to deliver packages” – a pretty core mission of the Postal Service – sorry, I give the nod to Amazon. Yet the article chooses to focus instead on Amazon’s connections to home surveillance and connection to police forces, suggesting that the Post Office is more ethical than Google.

If you can’t be precise in defining “trust to do what,” then it’s like any weak syllogism: from a false premise, any conclusion follows.

Sorry, this is just sloppy thinking. It’s akin to bar arguments about the greatest rock ’n roll band, or the all-time NBA dream team. Actually, it’s worse: it’s like arguing whether Tiger Woods or Serena Williams is the greater athlete.

Again, it all depends on answering “trust to do what?” The more vague the answer, the less useful the statistic – no matter how many decimal points you can point to in the data.