Equality by lot

Readers of Equality by lot might be interested in this use of sampling to improve not equality but quality.

The “Quality Control” Interview for Big Classes

I teach a lot of big classes – the undergraduate advanced data analysis class passed 100 students many years ago, and this fall is over 230 – which has some predictable consequences. I don’t get to talk much to many of the students. They’re mostly evaluated by how they do on weekly problem sets (a few of which, in some classes, I call “take-home exams”), and I don’t even grade most of their homework, my teaching assistants do. While I try to craft problem sets which make sure the students practice the skills and material I want them to learn, and lead them to understand the ideas I want them to grasp, just looking at their scores doesn’t give me a lot of information about how well the homework is actually working for those purposes. Even looking at a sample of what they turn in doesn’t get me very far. If I talk to students, though, I can get a much better sense of what they do and do not understand fairly quickly. But there really isn’t time to talk to 100 students, or 200.

About ten years ago, now, I decided to apply some of the tools of my discipline to get out of this dilemma, by means of random sampling. Every week, I would randomly select a fixed number of students for interviews. These interviews took no more than 30 minutes each, usually more like 20, and were one-on-one meetings, distinct from regular open office hours. They always opened by me asking them to explain what they did in such-and-such a problem on last week’s homework, and went on from there, either through the problem set, or on to other topics as those suggested themselves.
Continue reading →

Filed under: Academia, Applications, education | Leave a comment »