Readers of Equality by lot might be interested in this use of sampling to improve not equality but quality.
The “Quality Control” Interview for Big Classes
I teach a lot of big classes – the undergraduate advanced data analysis class passed 100 students many years ago, and this fall is over 230 – which has some predictable consequences. I don’t get to talk much to many of the students. They’re mostly evaluated by how they do on weekly problem sets (a few of which, in some classes, I call “take-home exams”), and I don’t even grade most of their homework, my teaching assistants do. While I try to craft problem sets which make sure the students practice the skills and material I want them to learn, and lead them to understand the ideas I want them to grasp, just looking at their scores doesn’t give me a lot of information about how well the homework is actually working for those purposes. Even looking at a sample of what they turn in doesn’t get me very far. If I talk to students, though, I can get a much better sense of what they do and do not understand fairly quickly. But there really isn’t time to talk to 100 students, or 200.
About ten years ago, now, I decided to apply some of the tools of my discipline to get out of this dilemma, by means of random sampling. Every week, I would randomly select a fixed number of students for interviews. These interviews took no more than 30 minutes each, usually more like 20, and were one-on-one meetings, distinct from regular open office hours. They always opened by me asking them to explain what they did in such-and-such a problem on last week’s homework, and went on from there, either through the problem set, or on to other topics as those suggested themselves.
Continue reading
Filed under: Academia, Applications, education | Leave a comment »


Drawing Lots: From Egalitarianism to Democracy in Ancient Greece, a new book by Irad Malkin and Josine Blok, has just been published by Oxford University Press. The book is a major landmark in the study of sortition and its association with democracy. The book aims to show, via a review of the history of the application of allotment in the ancient Greek world, that Greek democracy grew out of an egalitarian mindset, a mindset that was expressed, as well as presumably reinforced, by the widespread application of allotment in different contexts over a centuries-long period.1
Student Assembly officials shouldn’t be elected, they should be randomly selected. This somewhat radical idea has roots in ancient Athens where, for centuries, public officials were chosen via sortition. Sortition is the selection of public officials by lottery rather than election. We know, it sounds like an insane idea, but bear with us. Our goal with this article is not to convince you that sortition is a perfect system that should be implemented everywhere, we haven’t even convinced ourselves of that, but rather that it is a system with enough merit to be worth trying, and that the College of William and Mary’s Student Assembly offers the perfect laboratory within which we can test out the concept.
Cristina Lafont, Professor of Philosophy at Northwestern University,