Found It!

I am obliged to confess that I should sooner live in a society governed by the first two thousand names in the Boston telephone directory than in a society governed by the two thousand faculty members of Harvard University.

-William F. Buckley, Jr., Rumbles Left and Right: A Book about Troublesome People and Ideas (New York: G.P. Putnam’s Sons, 1963), p. 134.

I know this quote has been mentioned here before, but this is the first time I’ve ever had in my hand an actual primary source by Buckley for it. Thanks to Ralph Keyes’ The Quote Verifier (New York: St. Martin’s Griffin, 2006) for directing me to it.