Research: Sharing the funds by lottery

Posted on May 28, 2011 by Conall Boyle

Hans de Jonge, a university Education Policy Advisor in the Netherlands asks for our help:

“I believe there is some similarity between the arguments used to support lotteries in the allocation of scarce places in medical school and the case for using lotteries in the distribution of research funds. Do you know of any papers in support of this, or instances where it is used?”

“I am responsible for matters concerning selection, numerus clausus and student admission and access regulation. I found it very interesting to see your assessment of the Dutch model of weighted lotteries for our medical schools!

Your book mainly deals with using lotteries in an educational context. It might also inspire the reader to think of other possible situations in which lotteries can be used to support decision making processes. One obvious possible application to me seems the distribution of research funding.

One problem is the impossibility to select “the best from the very best” research proposals due to the lack of valid comparators. This is an argument that continues to be used in support for lotteries in medical schools in the Netherlands today.

Another striking feature relates to the costs. If the costs for selecting “the best from the very best” are no longer proportionate to the efforts put into it, and if these costs are a drain on other important primary processes (e.d. education, research), maybe it is better to think of a cheaper system such as a lottery.

This argument is significant when thinking of the highly bureaucratic and expensive systems used for distributing research funds. There are great costs incurred in maintaining the ever-increasing number of national and international research councils. Their main task it is to organize peer review, which this by no means is always able to distinguish between the best and the very best research proposals.

I was delighted to read in your earlier paper for the Journal of the Royal Statistical Society that professor Goodhardt has made a similar suggestion about the use of lotteries in allocating research funding. [this will be posted as a comment]

Until now however I’ve been unable to find any literature in support of the idea of research funding distributed by lottery.

Do you know of any papers?

Or do you know of funding systems in which forms of lotteries are used?

Thanks in advance for any suggestion you have,

I would be delighted to hear from you,

Best regards,

Drs. H.L. de Jonge • Beleidsadviseur onderwijs • T (070) 302 14 33 • dejonge@vsnu.nl

Vereniging van Universiteiten (Lange Houtstraat 2 • Postbus 13739 • 2501 ES Den Haag • T (070) 302 14 00 • M 06 255 38 047 •F (070) 302 14 95 • http://www.vsnu.nl • werkdagen Ma t/m vr

Filed under: Distribution by lot, Proposals |

3 Responses

conallboyle, on May 28, 2011 at 1:13 pm said:

Here’e what Gerald Goodhardt (London) said:

“My second example concerns the allocation of grants by bodies such as research councils. As I understand it grant applications are first assessed by the system of peer review.

However, this results in far more applications being judged worthy of support than there is money to fund them. The Economic and Social Research Council has reported that, in 1995, 522 applications were given the highest grade but only 225 could be funded.
Therefore a second round of sifting is undertaken by, I believe, the various boards and committees. The criteria used in this round are not generally known, but it is widely believed that a major influence is the past record of applicants in successfully completing past projects.

This is a reasonable risk reducing strategy, but it tends to lead to the creation of a limited circle of researchers, and to discourage new applicants from applying.

My suggestion is that the majority of the money, say 80%, should be allocated as at present, but the remainder should be allocated by means of a lottery among the remaining top-graded projects. No-one can be aggrieved by this system since it cannot be known who would have benefited in the absence of the lottery. The winners in the lottery need not be identified either, unless it was felt desirable to publicize the method.”

In a Comment on Boyle Conall (1998) Organizations selecting people: how the process could be made fairer by the appropriate use of lotteries The Statistician 47 Part 2, pp 291-321 London: Royal Statistical Society

LikeLike
conallboyle, on May 28, 2011 at 1:18 pm said:

and this was my reply to Hans de Jonge:

I occasionally see proposals for the idea of lotteries for distribution of research funding, but usually as a sort of joke.
I haven’t got the references — I only keep records of actual use of distributive lotteries. (On my website http://www.conallboyle.com ). So far I have never encountered research funding with a lottery element.

Of course some form of random distribution of research funding could be easily arranged and the benefits could be significant. Imagine that the application-for-funding process remains the same. The Committee could then rank the proposals for big, medium and small applications in order of merit.

Next step could be some form of weighted lottery, according to ‘merit’. (Note that here, no project has been eliminated — there remains a small chance of a ‘poor’ project winning funding.)

So far the benefits are that all universities have some chance of funding. This breaks the (almost certain) grip of the elite who otherwise would hog all the funding. Allowing some funding to leak into unconventional or speculative research seems to be exactly what should be happening.

Once this process of lottery-distribution is established, then some of the elaborate and time-wasting rituals could be slimmed down. The huge amount of paper-work required, the elaborate documentation consumes huge amounts of time. (Economists call this ‘rent-seeking’, which by definition is wasteful)

So a better spread of research funds, more likely to reach more interesting projects; and a reduction in time-wasting form filling: What’s not to like about that?

At this point the vested interests, the members of existing selection panels, but above all the elite institutions who know the system, and capture the bulk of the funding will see their privileges threatened.

The history of the introduction of the weighted lottery for university places in the Netherlands shows that it requires a curious and unusual balance of forces to produce a rational compromise.

Perhaps the best idea is to start small — reserve a small portion, say 5% of total funds to be awarded entirely randomly, funding as many as possible until the money runs out.

LikeLike
Research: Sharing the funds by lottery « Equality by lot | Today Headlines, on May 28, 2011 at 4:28 pm said:

[…] is the original: Research: Sharing the funds by lottery « Equality by lot Share […]

LikeLike