Lotteries instead of point-scores on exams: A great quote from Peter Stone

A story in the Irish Times (25th Feb 2021). This is a paywalled link. The full text of the article appears below.

The Leaving Cert is not fair. Why not just replace it with a lottery?
Joe Humphreys
Unthinkable: We can no longer plead ignorance of the inner workings of our State exams

‘I think recognizing the wider role luck plays in society is very important,’ says TCD political scientist Peter Stone.

The Leaving Cert has had an untouchable status in Irish life. It may be a brutal memory test but it is our brutal memory test – a rite of passage nearly as old as the State itself.

In the past 12 months, however, the bonnet has been lifted on this national treasure and we can no longer plead ignorance of its inner workings. The attempt to manufacture a distribution of grades under pandemic conditions equivalent to those produced by the annual exams has spotlighted long-running questions of fairness.

As a test of ability, the Leaving Cert is fair in the narrow sense that a bobsleigh race between Jamaica and Norway is fair. Contestants do not start with the same advantages, and the format – which lends itself to a parallel grinds industry – gives an extra edge to students from better-off families.

However, there’s a second matter of fairness surrounding the appropriateness of using the Leaving Cert to determine who gets what college places. This must be considered against the backdrop of stark figures showing that, on average, a third-level graduate earns much more over her or his lifetime than a worker who doesn’t have a degree – at least €100,000 more, according to one conservative estimate.
Continue reading

Only two days left to vote!

The poll for the change to the subtitle of this blog ends on Tuesday, yet currently only .02% of the “electorate” have recorded their preferences. This might seem like a trivial matter, but it crucially affects the range and scope of the posts submitted. The blog was founded by Conall Boyle and others some ten years ago in order to discuss the work of those with an interest in lotteries for equal distribution and social justice — see for example Barbara Goodwin’s Justice by Lottery. However the blog soon became dominated by those (like Yoram and myself) exploring the political potential of sortition in reforming (or replacing) electoral democracy. This change of focus seems to meet the needs of most contributors and readers but it would be a tragedy if those working on other aspects of sortition felt excluded by an over-prescriptive sub-title. If you look at the book series Sortition and Public Policy you’ll see that around half of the titles are devoted to the non-political use of lot. And many theorists dealing with the political potential of sortition, for example Oliver Dowlen and Peter Stone are unpersuaded regarding the use of sortition for democratic representation (they focus more on the Blind Break as an arational prophylactic against factionalism). So it would be good if the new subtitle reflected the full range of interest in sortition. If you want to vote, just go to the Online Poll, look at the list of “candidates”, choose your preference(s) and post a comment, it’s that easy!

Covid Treatment Lottery

“For patients with similar prognosis, who cannot be separated in other ways, a random allocation, such as a lottery, may be used”, says the protocol.

So says a Report in the Daily Telegraph (UK) of 5th January by Paul Nuki titled “Covid ‘lottery’: Doctors draw up triage protocol in the event treatment has to be rationed” (Telegraph usually paywalled, but this seems open-access)

It refers to a paper in J Medical Ethics “Development of a structured process for fair allocation of critical care resources in the setting of insufficient capacity: a discussion paper” also accessible f.o.c.

This is circulating in NHS hospitals as a proposed protocol.

The protocol – drafted by medical, legal and palliative care specialists at the Royal United Hospital Bath NHS Trust – is the most sophisticated attempt yet to devise an ethical system for rationing care in the event that there are insufficient resources to treat everyone.

Now this is exciting! But it is not new. Right from the start of organ transplantation (1960s) such moralistic contentions were weighed up.

In Seattle the so-called ‘God committee’ was set up to make these difficult choices (reported in Calabresi & Bobbit (1978) Tragic Choices). The committee eventually found that it was too agonising to make these choices, and passed the task back to the medical practitioners. In the end it was felt that only medical  factors should be taken into account. Even if no overt rules on social merit were in place, we should not be surprised if the doctor, genuinely uncertain on medical grounds,  was to pick the ‘nicer’ of the two patients.

A secret lottery?

Elster (1989) in his masterly ‘Solomonic Choices’ gives the example of child custody cases, where the judge is frequently unable (in his own mind) to give a clear-cut decision. Yet decide he must, so he goes ahead, dressing up the verdict with trappings of rationality.

This, claims Elster, satisfies both parties, the winner praising the wisdom of the judge, the loser cursing his bias. No doubt a similar process might go on when a medical doctor decides, even if partly randomly and in secret, between her two patients: So long as both patients believe that their case is decided clinically by an expert, then both winner and loser may find it acceptable.

The doctor herself may even be a bit cognitively dissonant—convincing herself that she is doing the right thing for the right reason, exercising judgement based on intuition  rather than validated knowledge. This form of fudging may be acceptable all round, but it is fraught with dangers.

If fakery is suspected, patients rapidly lose their trust in their professionals. Unwitting discrimination seems inevitable. True expertise will fail to develop unless its limits are acknowledged.

Against a lottery is Greely (1977) who suggests that if recipients can argue about any allocation, they feel more satisfied. Anand was also interested in what is called ‘voice’—that one of the reasons a coin-toss was thought to be unfair is that it deprived customers of a say in the decision.

In favour of a visible act of coin-tossing Calabresi & Bobbit explain that it draws attention to the fact that resources are limited. Edgeworth (1888) suggested another benefit would be that the public, seeing a random drawing take place, would be alerted to the ‘aleatory nature’ of the decision. Bureaucrats might not like having such attention focussed on this shortage of resources and their uncertain knowledge.

[This was part of my 2006 thesis Who Gets The Prize. It can be viewed in full on my website]

Subtitle change vote

Following the call for proposals for changing the subtitle of this blog, we have the following proposals:

  1. The blog of the Kleroterians (i.e., keep the subtitle as is.)
  2. The political potential of sortition
  3. Sortition as a democratic tool
  4. No democracy without sortition
  5. Because you can’t have democracy when you don’t have sortition
  6. The democratic potential of sortition
  7. Sorting out sortition
  8. A blog to sort out sortition
  9. Maximus in minimis
  10. Put the man in the street in the catbird seat
  11. Democratic lotteries and the potential of sortition
  12. Selection by lot
  13. Selecting political decision-makers the way we select jurors
  14. More democracy by random selection of citizens
  15. Better democracy through sortition
  16. Renewing democracy through sortition
  17. More democracy by haphazardly selected citizens
  18. Sortition: next step for democracy
  19. The political potential of democratic lotteries and sortition
  20. More democracy via sortition
  21. Democracy and the potential of sortition
  22. Sortition is the future of democracy
  23. Better politics through sortition
  24. Sortition, impartiality, equality, People’s rule
  25. Democracy through sortition
  26. Sortition for democracy, fairness and good governance

(I tried to include no more than two proposals by each person. If you feel that there are fewer than 2 of your proposals on the list, or if you otherwise feel that your proposals were unfairly excluded, please let me know as soon as possible.)

Ideally, I would go with proportional representation, so that each subtitle would be used part of the time, where the part is determined by the proportion of the votes it got. However, I am afraid this is technically difficult. (Maybe we can consider changing the subtitle every year?)

As we all know, there are no good voting schemes, so we are left with using a bad one. I suggest then that we use ranked choice. Please respond in the comments below with exactly one ordered list of subtitles from the list above representing your order of preference. Voting closes in a week.

2020 review – statistics

Below are some statistics about the 11th year of Equality-by-Lot. Comparable numbers for last year can be found here.

2020 Page views Posts Comments
Jan 3,223 7 28
Feb 3,008 6 21
Mar 3,562 8 41
Apr 4,368 10 106
May 4,507 7 156
June 3,481 13 67
July 3,828 11 100
Aug 3,898 12 123
Sept 4,773 21 201
Oct 4,733 16 106
Nov 4,005 15 165
Dec (to 26th) 1,989 10 54
Total 45,375 135 1,168

Note that page views do not include visits by logged-in contributors – the wordpress system does not count those visits.

Posts were made by 15 authors during 2020. (There were, of course, many other authors quoted and linked to.)

There are currently 449 email and WordPress followers of this blog. In addition there are 483 Twitter followers (@Klerotarian) and 67 Facebook followers.

Searching for “distribution by lot” (with quotes) using Google returns Equality-by-Lot as the 2nd result (out of “about 307,000 results”). Searching for “sortition” does not show Equality-by-Lot until the 6th results page (out of “about 253,000 results”) – a dramatic demotion compared to previous years.

Happy holidays and a happy new year to Equality-by-Lot readers, commenters and posters. Keep up the good fight for democracy!

Call for 2020 review input

This is the yearly call for input for the year’s end review. As in previous years, I would like to have a post or two summarizing the ongoings here at Equality-by-Lot and notable sortition-related events over the passing year. Any input about what should be included is welcome – either through comments below or via email. You are invited to refresh your memory about the events of the passing year by browsing Equality-by-Lot’s archives.

For previous years’ summaries see: 2019, 2018, 2017, 2016, 2015, 2014, 2013, 2012, 2011, 2010.

Using focal random selection to close the gender job- and pay-gaps

Those well funded Swiss researchers have just produced another Report on the benefits of using a lottery as part of the job-appointment process.

By ‘focal’ they mean a two stage process (focussed?) with all applicants undergoing an ability test and the top three being entered into a draw, so the winner is selected at random.

The alternatives were: to select entirely on ability, or else entirely at random (from a pool of well-qualified applicants).

Their conclusion

“Our findings suggest that the pool of high-performing women who apply for top jobs can be substantially enlarged by the introduction of focal random selection. Consequently, the pipeline for women to leadership positions can be made less leaky without lowering candidates’ performance. Moreover, focal random selection closes the gender pay gap among high performers. In addition, differences between men and women in entering competition caused by gender stereotypes are completely eliminated by randomness. Our findings, therefore, point to the relevance of gender stereotypes as an underlying mechanism of gender gap in competitiveness.”

Not bad! Fix the ‘glass ceiling’ and the gender pay gap with the judicious use of lotteries!

You can read the paper here (no paywall)

Voting is a Community Right

In an election day post, I discuss how traditional voting and sortition can be viewed as aspects of the same right. Unifying both is the need to reverse the burden of action: while voting requires citizens to decide to vote, sortition requires proper authorities to find participants. Instead, voting in democracies now should require electoral authorities to obtain a valid vote from everyone who is eligible. At that point, calling random juries and assemblies will be a breeze.

Are there measurable benefits in using a lottery to select leaders? A scientific experiment

Short answer: Yes, and no!

Longer answer:

Hubris is a tendency of leaders to hold an overly confident view of their own capabilities and to abuse power for their own selfish goals, sometimes with disastrous consequences for organizations. A major reason for hubris is the rigorous selection process leaders typically undergo. This study proposes a governance mechanism used successfully in history to tackle hubris: partly random selections, which combine competitive selections by competence with lotteries. A frequently voiced concern about the use of lotteries is that it takes no account of the competence of the leader chosen. We propose that partly random selections can mitigate the disadvantages of both competitive selections alone and lotteries alone and reduce hubris in leaders. We conduct a test of this governance mechanism by means of a computerized laboratory experiment. Our results show that partly random selections significantly reduce the hubris of group leaders. [my emphasis]

This is the Abstract from the Report. The full citation is: Joël Berger; Margit Osterloh; Katja Rost; Thomas Ehrmann (2020, May 13) ‘How to prevent leadership hubris? Comparing competitive selections, lotteries, and their combination’ The Leadership Quarterly, ISSN: 1048-9843 (paywall)

In order to test their theory, this group of Swiss and German scientists conducted an experiment, using a method instantly recognisable to experimental economists (and others, but they are the ones I’m familiar with). Their hypothesis was that a lottery could play a useful part in limiting hubris when selecting leaders.

We conducted a computerized laboratory experiment   ….  864 students of the University of Zurich and ETH Zurich, were randomly selected from a pool of students who had volunteered to participate in behavioral experiments for monetary compensation. Participants on average gained USD 30 for 45 min……The 864 participants were randomly selected into groups of six and randomly assigned to one of three treatment conditions.

Wow! As you can see, this sort of experiment is not cheap, so well done to the guys in Zurich to obtain the funds from the Swiss government to conduct an experiment on lotteries-for-jobs. Note, too, the use of a randomly selected sample and sub-samples. Ok, so it’s students, it generally is in these scientific tests, but for obvious practical reasons.

Briefly, the experiment proceeded thus: A leader for each group were produced by one of three methods. 1. Using a general knowledge test and appoint the top scorer; or 2. Same test, but select at random from the top three scorers; or 3. A simple lottery where every member of the group has an equal chance.

How ‘hubris’ of the selected leaders was measured was complicated, and if you want know, you’ll have to read the article, but it did involve the well-known ‘Prisoner’s Dilemma Game’. It was from this, and 11 pages of statistical analysis of regression models that the conclusion was reached.

Our study follows a pioneering approach to investigate an unusual selection method for appointing leaders in organizations, partly random selection. This selection method has been extensively used in history but has nearly been forgotten. Today, random decisions are considered by many people to be “irrational”. Our study shows that purposeful random selection, in particular combining competitive selections with a random component, is a rational and promising way of recruiting leaders that tackles hubris in overconfident leaders. Our proposal to “draw your CEO by lot” is provocative but may be promising.

Most of the members of this group engage in philosophical discussion, where the merits of a proposal are a matter of persuasive rhetoric. Elsewhere, exhortations to ‘follow the science’ abound, and mere rhetoric is treated with caution. Even calling in aid ‘common-sense’ can be suspect.

This is, I believe, the first time any hypothesis of us Kleroterians has been subject to what has been described as ‘The gold standard of science’. I have another example from Levitt of Freakonomics fame which almost constitutes Science, which I will post about later.

A lottery for top jobs is not such a crazy idea

So says Amanda Goodall in an article in the Financial Times 9 Sept 2020.  You can read the article in full here (dodging the FT paywall!).

Dr Goodall of the Cass Business School, London has produced many papers on management and HR. She has tried to promote the idea of Lotteries for Jobs with Margit Osterloh, a Swiss academic.

Amanda tells me “It is a hard one [the idea of lotteries for jobs] to get off the ground. It has been hard to publish our article.”

It is very rewarding to find others in the field working on this form of ‘Local Democracy’ as Elster calls it. There are further developments which I will post here shortly.