Equality by lot

An interesting, and worrying, article by Robert Epstein on what he calls the “Search Engine Manipulation Effect (SEME)” appeared the other day on the site Aeon.

We already knew that the order of results on web search engines, particularly Google, can influence consumers’ choices. It’s not surprising that it also has an effect on political choices. What is surprising is the degree. Epstein and his team conducted experiments which show a very large effect indeed. In one case the proportion of people favouring the search engine’s top-ranked candidate increased by 48.4 per cent, in another by an average of 37.1 percent, and by as much as 80 per cent in some groups.

We also learned in this series of experiments that by reducing the bias just slightly on the first page of search results – specifically, by including one search item that favoured the other candidate in the third or fourth position of the results – we could mask our manipulation so that few or even no people were aware that they were seeing biased rankings. We could still produce dramatic shifts in voting preferences, but we could do so invisibly.

Could voting preferences be shifted with real voters in the middle of a real campaign? We were skeptical. In real elections, people are bombarded with multiple sources of information, and they also know a lot about the candidates. It seemed unlikely that a single experience on a search engine would have much impact on their voting preferences.

To find out, in early 2014, we went to India just before voting began in the largest democratic election in the world – the Lok Sabha election for prime minister. The three main candidates were Rahul Gandhi, Arvind Kejriwal, and Narendra Modi. Making use of online subject pools and both online and print advertisements, we recruited 2,150 people from 27 of India’s 35 states and territories to participate in our experiment. To take part, they had to be registered voters who had not yet voted and who were still undecided about how they would vote.

Participants were randomly assigned to three search-engine groups, favouring, respectively, Gandhi, Kejriwal or Modi. As one might expect, familiarity levels with the candidates was high – between 7.7 and 8.5 on a scale of 10. We predicted that our manipulation would produce a very small effect, if any, but that’s not what we found. On average, we were able to shift the proportion of people favouring any given candidate by more than 20 per cent overall and more than 60 per cent in some demographic groups. Even more disturbing, 99.5 per cent of our participants showed no awareness that they were viewing biased search rankings – in other words, that they were being manipulated…

SEME’s near-invisibility is curious indeed. It means that when people – including you and me – are looking at biased search rankings, they look just fine… Even I have trouble detecting bias in search rankings that I know to be biased (because they were prepared by my staff). Yet our randomised, controlled experiments tell us over and over again that when higher-ranked items connect with web pages that favour one candidate, this has a dramatic impact on the opinions of undecided voters, in large part for the simple reason that people tend to click only on higher-ranked items. This is truly scary…

[I]f Google set about to fix an election, it could first dip into its massive database of personal information to identify just those voters who are undecided. Then it could, day after day, send customised rankings favouring one candidate to just those people…”

[the full article is well worth reading]

In fact, does it make much difference if it’s deliberate bias by Google not living up to its “do no evil” slogan? If the bias is introduced by the algorithm, without any evil intention on Google’s part, are we any better off? In practice, it will probably be the pages that are most frequently referred to that appear at the top of a search. “Search Engine Optimisation” is a well-established black art which is surely being used in the current US election, no doubt by all candidates. In this, the advantage will presumably go (as always) to the candidate with the most money, unless Google deliberately fiddles with page rankings, which Epstein seems to think is likely. Still, even if no candidate had an advantage, so that the rankings, at least on the first page, were an unpredictable or random result of the algorithm, is the effect anodyne?

Food for thought. My first reaction was “Damn, I’ll have to rewrite my book to add a twenty-fifth failing of elections, but at least it won’t affect sortition” (You could argue that it isn’t elections that are at fault, it’s a brainless public, or that evil Google, but since elections are vulnerable to this effect, that’s not much of an argument.)

My second reaction was: is sortition really exempt? How would the SEME affect the model I have proposed? (available here: [PDF epub mobi]. Please don’t refer to the early six-part draft on this site; important changes have been made since then.)

There would be no effect at all on the selection of Assembly members, or of committee members, so in this respect sortition is exempt. But what about Assembly members’ votes on proposals for law? What about the opinions expressed by policy committee members in their reports to the Assembly? What about the opinions expressed to the policy committees, and what about the birth of the proposals themselves?

I think it’s reasonable to guess that by the time the policy committees’ reports have been read to or by the Assembly members, they have debated a proposal, and are ready to vote they would no longer be using Google for that proposal. And while members of policy committees might do a fair bit of Googling (to find terms they did not understand, for instance) in the early days of their researches, by the time they have been harangued by the various experts and interested parties, their Google-inspired prejudices will probably have been overwritten several times, so that page rankings should be irrelevant. It does seem quite likely that the SEME might affect which bright ideas come to the minds of the public, and so the birth of proposals might be affected. But then, a news item or documentary, or even a fiction could inspire a new proposal, and where’s the harm, so long as there is adequate informed deliberation before the proposal gets to the vote?

This last paragraph is no more than a guess, and is something that could usefully be investigated by someone with a research grant.

Filed under: Elections, Experiments, Sortition |