“Putting the practice of sortition on firmer foundations”

An article in Nature by Bailey Flanigan, Paul Gölz, Anupam Gupta, Brett Hennig and Ariel D. Procaccia proposes a sampling algorithm which produces samples with specified quotas for given subgroups of the population. Since the quotas do not match the proportions of the groups in the population, the probability of selection of each person in the population is not the same. However, the algorithm aims to make those probabilities as equal as possible.

The authors propose the use of their algorithm for selecting citizen assemblies from groups of volunteers. In existing practice, the group of volunteers for a citizen assembly is usually very unrepresentative of the population as a whole and the quotas are used to supposedly compensate for this unrepresentativity and make sure that the selected assembly is descriptive of the population as a whole. The authors claim that “[b]y contributing a fairer, more principled and deployable algorithm [than the previous algorithm used], our work puts the practice of sortition on firmer foundations. Moreover, our work establishes citizens’ assemblies as a domain in which insights from the field of fair division can lead to high-impact applications”.

In my view, while this work may be of theoretical interest in the field of sampling, and while the authors may have the most commendable intentions of promoting democratic decision making, the notion that this work in any way improves the political application of sortition is not only unjustified, but may actually be the opposite of reality.

First, it is obvious that unless absurdly arbitrary and drastic assumptions are made, quotas can in no way compensate for the unrepresentativity of a volunteer sampling group. For quotas to be able to compensate for the unrepresentativity of the volunteer sampling group, it must happen that within each quota group the probability of volunteering is uncorrelated with (informed and considered) opinions on the matters at hand. One would have to have a horribly mechanistic and reductionist notion of what determines individual opinions in order to make such an assumption. Thus, the entire endeavor of quota-adjusted sampling is no more than cosmetics over the reality of bias introduced by low volunteer rates in existing applications of sortition in politics.

Second, in existing practice, the entire sortition process – from initiation, through agenda setting, witness selection, and participant recruitment, and all the way to implementation of measures adopted by the allotted – is managed by professionals – elected, or appointed. The sampling algorithm is just another component of this specialist control, and its sophistication in fact goes against a crucial consideration, that of simplicity. Simplicity is a prerequisite for transparency which is an important component for trust. The fact that a team of computer scientists and statisticians assures the public that their sampling procedure is “fairer and more principled” is in no way a cure for the fact that the procedure is too complex for the vast majority of citizens to understand by themselves, and is hard to verify even for those who do understand its workings. Through increased complexity, algorithmic manipulation becomes another opportunity for elite control of the sortition process, reducing public trust – whether or not manipulation in fact occurs in any particular application.

Thus, in view of the first point, no algorithmic solution can fix the problem of bias in the volunteer pool, and, in view of the second point, employing algorithmic sophistication may actually reduce trust in the procedure. The way to address these fundamental problems is to simplify the procedure dramatically by abandoning quotas altogether and instead being very open and transparent about the rates of volunteering, acknowledging that low volunteering rates are a clear indicator that the sortition setup is dysfunctional.

My proposal is thus as follows: allotted bodies are to be constituted through simple random sampling and are to explicitly include “absent members”. Thus, if a body is designed to have 150 members (as was the case with the French Citizen Climate Convention) and if the volunteering (acceptance) rate is 1/3 (as was the case with the CCC, we are told), then the body will in fact have 50 “present members”, and it will be made very clear and very visible that there are 100 additional “absent members” – i.e., members who turned down an offered seat on the body. In addition to publishing the demographic, social, economic characteristics of the present members and the absent members, the reasons stated by the absent members for their absence would be published as well.

In situations of low acceptance rates, the fact that the body is not truly representative of the population would become an important part of the setup and the organizers would have to explain why it is that they were unable to obtain higher acceptance rates. The public would have to judge whether the body’s democratic authority is acceptable or is undermined by the non-representativity of its makeup. The organizers of allotted bodies would be motivated to increase acceptance rates by incentivizing the public to accept offered seats. This could involve paying participants properly for their time and effort and accommodating specific needs such as child-care or obtaining leave-of-absence from a job, but could also involve making sure that citizens feel that participation has a real impact because the allotted body has a meaningful agenda, real powers and significant public attention.

All of this, of course, increases the burden on the organizers. The organizers would no longer be able to set up allotted bodies easily and cheaply and claim that they provide legitimate democratic basis for decision making. Thus, it is quite unlikely that the direction proposed here would be adopted enthusiastically by sortition professionals and their clients. But such difficulties are not unexpected. Democracy in a mass society is not easy to achieve (or maintain). It requires careful design and determined attention to details during implementation. The public should make such demands of those who expect to rule it and especially of those who claim to be constructing a democratic government system.

4 Responses

  1. The authors propose the use of their algorithm for selecting citizen assemblies from groups of volunteers. In existing practice, the group of volunteers for a citizen assembly is usually very unrepresentative of the population as a whole.

    I’m glad to see that what was previously only a hunch is now confirmed by a paper in Nature (presumably on a quantifiable basis). It will be interesting to see if this puts a stop to the oft-repeated, but entirely mendacious, claim that small, randomly-selected groups of volunteers are representative of the whole population.

    Liked by 1 person

  2. From the original paper:

    the pool tends to overrepresent groups with members who are on average more likely to accept an invitation to participate, such as the group ‘college graduates’.

    Given the strong correlation between [lack of] a college degree and voting for Trump and Brexit, this would mean that volunteering would lead to a sample that is significantly skewed towards “progressive” beliefs and preferences. But it’s even worse, as

    average response rates are typically between 2 and 5%

    The obvious solution is that participation in citizen assemblies needs to be viewed as on a par with jury service — a (quasi mandatory) civic obligation rather than a right. I agree with Yoram that the complex algorithmic solution suggested by the authors of this paper would make the matter far worse.

    Liked by 1 person

  3. Under the rather absurd title “Can AI make democracy fairer?”, Harvard is promoting the Nature article discussed above. Their article starts as follows:

    Democracy in ancient Athens looked quite different from democracies today. Instead of elections, most offices — including those in the legislature, governing councils, and magistrates — were filled by citizen volunteers, selected by random lottery. These citizens’ assemblies drafted, debated, and passed laws; made major foreign policy decisions; and controlled military budgets.


  4. […] Harvard dealt with sortition and an algorithm for quota sampling from unrepresentative volunteers made it into […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: