Have you encountered news online that seems biased but isn’t labelled as opinion?

My dissertation was born out of a realization that much news content online was being presented without indication of whether it is opinion. Clearly indicating whether an article is an opinion piece has been a cornerstone of good journalism for decades, but this critical piece of information has often been missing online, and from the most popular place for news: Google. (Yes – more than even Facebook).

Imagine you are searching for something online. You type “US Politics” into the search bar and hit enter:And you see one of the top results: “America’s culture wars distract from what’s happening beneath them.”

If you only saw this article on Google would you think the Guardian is biased?

(This article comes from the Guardian’s opinion section, which is separate from their news department.)

I hypothesized that if Google were to clearly label which articles were opinion, they would improve the perceived credibility of news sites, and thereby improve trust in the news media as a whole – very important for healthy democracies. (Granted, bias is not a monolithic thing, and many news organizations exhibit bias even in their factual reporting).

I made two online experiments to test my hypotheses. In the first, I presented participants with an imitation Google search page. Participants were either shown a news feed with opinion labels:

Or without:

And the headlines were either biased toward the political-left:

Or the political-right:

I also randomized the news source: CNN or Fox News (image not shown).

For the second experiment, I was curious to know what effects these ‘opinion labels’ would have when they are presented on something more like a feed where there are multiple sources present – a Google News feed to be exact.

After running the studies and collecting and cleaning the data, I was ready to begin my analysis using fancy statistical techniques like MANOVA/ANCOVA, multiple regression and structural equation modeling. Anywho, here are my main findings:

  1. Opinion labels significantly increased how credible participants found news brands to be in study 1 (the search engine study) but not in study 2 (the news feed study).
  2. The reason why perceived credibility went up in study 1 was for the reason I hypothesized: opinion labels helped participants distinguish news from opinion.
  3. This finding was consistent across political identity (liberals + Democrats v conservatives + Republicans), news source (Fox v. CNN), age and gender, etc.

Interesting, right?

My theory for why opinion labels increased credibility in search engines but not in news feeds is a fairly simple one. It has to do with people’s motivations for using platforms. If you think about it, most people use search engines to find something specific (this is called way-finding). They want to find a piece of information in the most efficient manner possible, so they tend to be more critical about where information comes from. In contrast, most people use feeds to passively scroll down a list, so they are less likely to pay attention to where a particular source of information comes from.

There’s much more to the dissertation that I left out, so if you’re curious about this research and want to learn more, you can read the full dissertation here (will update – not yet on ProQuest’s servers).

(If you don’t have institutional access you can also read it for free here)