4 thoughts on “The Wisdom of Crowds

  1. Interesting site!

    But as a machine learner, I should point out that the sample size seems way too small for the rankings to be stable, ๐Ÿ™‚ particularly at the top and bottom. Of course unless they are doing some fancy active sampling, which I don’t think they are ๐Ÿ™‚

    Currently, they have ~4800 votes for 68 schools; assuming uniform sampling, a certain pair of schools has appeared in a comparison about 2 times on an average; so the relative positions of say MIT vs. UC Berkeley, is really non-robust, and depend on just a few votes. Of course the coarse-grained positions are still okay…

  2. All these rankings are useless. It is obvious, for instance, that in TCS, MIT is no better, if not worse, than any of the top 30 places.

  3. Actually, more serious than Kamalika’s objection of small sample sizes is the fact that the site is pretty indiscriminate about how the data are collected. As far as I can tell, it will allow you to submit inconsistent votes (“A better than B” and “B better than A”, or “A better than B better than C better than A”, or “A better than B” and “I don’t know anything about A”). It also doesn’t seem to mind if you do something like always vote for the school on the left. Finally, as far as I can see, there is no limit on the number of votes cast, nor is there even a rudimentary check to see if the voter is human. So if one had the inclination and time, one could get these ratings to be whatever one wanted (although of course there is the separate question of why one would bother…)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s