Are there measurable benefits in using a lottery to select leaders? A scientific experiment

Short answer: Yes, and no!

Longer answer:

Hubris is a tendency of leaders to hold an overly confident view of their own capabilities and to abuse power for their own selfish goals, sometimes with disastrous consequences for organizations. A major reason for hubris is the rigorous selection process leaders typically undergo. This study proposes a governance mechanism used successfully in history to tackle hubris: partly random selections, which combine competitive selections by competence with lotteries. A frequently voiced concern about the use of lotteries is that it takes no account of the competence of the leader chosen. We propose that partly random selections can mitigate the disadvantages of both competitive selections alone and lotteries alone and reduce hubris in leaders. We conduct a test of this governance mechanism by means of a computerized laboratory experiment. Our results show that partly random selections significantly reduce the hubris of group leaders. [my emphasis]

This is the Abstract from the Report. The full citation is: Joël Berger; Margit Osterloh; Katja Rost; Thomas Ehrmann (2020, May 13) ‘How to prevent leadership hubris? Comparing competitive selections, lotteries, and their combination’ The Leadership Quarterly, ISSN: 1048-9843 (paywall)

In order to test their theory, this group of Swiss and German scientists conducted an experiment, using a method instantly recognisable to experimental economists (and others, but they are the ones I’m familiar with). Their hypothesis was that a lottery could play a useful part in limiting hubris when selecting leaders.

We conducted a computerized laboratory experiment   ….  864 students of the University of Zurich and ETH Zurich, were randomly selected from a pool of students who had volunteered to participate in behavioral experiments for monetary compensation. Participants on average gained USD 30 for 45 min……The 864 participants were randomly selected into groups of six and randomly assigned to one of three treatment conditions.

Wow! As you can see, this sort of experiment is not cheap, so well done to the guys in Zurich to obtain the funds from the Swiss government to conduct an experiment on lotteries-for-jobs. Note, too, the use of a randomly selected sample and sub-samples. Ok, so it’s students, it generally is in these scientific tests, but for obvious practical reasons.

Briefly, the experiment proceeded thus: A leader for each group were produced by one of three methods. 1. Using a general knowledge test and appoint the top scorer; or 2. Same test, but select at random from the top three scorers; or 3. A simple lottery where every member of the group has an equal chance.

How ‘hubris’ of the selected leaders was measured was complicated, and if you want know, you’ll have to read the article, but it did involve the well-known ‘Prisoner’s Dilemma Game’. It was from this, and 11 pages of statistical analysis of regression models that the conclusion was reached.

Our study follows a pioneering approach to investigate an unusual selection method for appointing leaders in organizations, partly random selection. This selection method has been extensively used in history but has nearly been forgotten. Today, random decisions are considered by many people to be “irrational”. Our study shows that purposeful random selection, in particular combining competitive selections with a random component, is a rational and promising way of recruiting leaders that tackles hubris in overconfident leaders. Our proposal to “draw your CEO by lot” is provocative but may be promising.

Most of the members of this group engage in philosophical discussion, where the merits of a proposal are a matter of persuasive rhetoric. Elsewhere, exhortations to ‘follow the science’ abound, and mere rhetoric is treated with caution. Even calling in aid ‘common-sense’ can be suspect.

This is, I believe, the first time any hypothesis of us Kleroterians has been subject to what has been described as ‘The gold standard of science’. I have another example from Levitt of Freakonomics fame which almost constitutes Science, which I will post about later.