An anonymous commenter asked an insightful question, worthy of a real answer: "Hi Prof, Why are you so obsessed with ranking things?"*
Honestly, I don't think I am. I have 3 children, and I have thus far avoided assigning them a preference ordering.** If you asked me for my favorite TV shows (or movies, or songs, etc.) I could think of some off the top of my head, but I haven't ever thought hard about coming up with a list of favorites.*** Same with restaurants, food, vacation destinations, whatever. I don't spend my time giving rankings for Netflix or things like that. I could probably come up with rankings with some thought, but it's not like I go around ranking things constantly.
That is, I don't do that in my personal life. In my professional life, come to think about it, I spend an awful lot of my time ranking things. I serve on multiple program committees each year where I'm asked to rank papers. (And I send my papers to conferences, where they are in turn ranked, and my submission is, implicitly a ranking of sorts on the conference.) I serve on NSF panels to rank grants. I write letters of recommendation which, implicitly or explicitly, provide a ranking of students (and, occasionally, faculty). I interview and evaluate faculty candidates. I grade and assign grades in my classes, and similarly grade senior theses. I serve on a Harvard committee that decides undergraduate thesis prizes. And I'm sure if I thought it about some more, I could come up with even more examples.
My blog is meant to be a professional blog, about my professional life. If it seems that I'm obsessed about ranking, that is a reflection of my professional life. I am asked to rank a lot as part of my job.
So I think I can turn the question back -- why are all of you so obsessed with ranking, that I end up having to spend so much time doing it?
* This comment came up in my last post about the possibility of FOCS/STOC asymmetry, where ranking was at most a tangential concern. But my previous post was on ranking networking conferences, so I can understand where the comment comes from.
** That's meant to be humorous.
*** Well, that's perhaps not quite true. Any undergraduate who has taken my algorithms class can correctly tell you that my favorite TV show of all time is Buffy the Vampire Slayer, so I'd best admit to it before it comes up in the comments.
20 comments:
Hi Michael,
I disagree with you to a certain extent. Ranking is important when a special task has to be done (e.g. for a recruitment, candidates have to be ranked with the criterion of "being the best for this position"). Nevertheless, I do not see the point to have "universal" rankings of everything. To continue with my example, I do not the point to say Mr X is better that Mr Y, in a general way. "Who is the best computer scientist between Mr X or Mr Y?" is not a well-posed question.
So, I think that in the same way, looking for "the best conference", "the best paper", etc... is not a good idea! For instance, before I submit a paper, my only interest is to know their scopes and where my paper better fill in.
Another example that I personally dislike is Universities ranking. What is the point of saying "Harvard is a better University than Stanford"? The only interesting ranking is for a particular student, "what is the best University FOR ME to study THAT SUBJECT?", taking in account lots of things such as the distance from their parents' home (just as an example). Besides, for a ranking such as Shangaï's ranking, its authors say that it is a ranking to know "where the chinese student should go if they want to study in the US, the UK, Japan or China", and they don't say that their ranking is valid for everywhere in the world (yet we can ask them why they also rank Universities from all over the world then...).
To conclude, I think that politicians want us to rank everything everytime (because it's simpler to understand such rankings than the diversity of Universities) and we have to be careful not to do so because it's a damageable simplification. We should try to avoid all "useless" rankings, in the sense that they have no direct applications (such as ranking of candidates for a position).
B --
I actually quite agree with you, in that I think many times rankings are inappropriate. Your university example is a good one -- the best university may depend on the student asking. Similarly, when universities accept students, I disagree with the idea that a ranking based on some arbitrary metric (like test scores) is a good way to go.
However, even in these settings, we are often forced to rank, and rankings can provide useful preliminary information. Saying "Harvard is better than Stanford" may not convey much true meaning, but having a list of the top 10 ranked graduate schools can be helpful for students looking to apply. Similarly, when a school has 10 times as many applicants as slots, test scores can provide a useful preliminary filter but all but a small number of exceptional cases.
Again, I was not trying to justify ranking in any specific circumstance in this post; I was simply pointing out that ranking seems to be a major component of my job.
A list of the top 10 universities is akin to People Magazine's 10 sexiest men in America in that:
1- Most people will agree, but many will not.
2- The fact that you are on the list, by itself, will make you irresistibly sexy to large swaths of the public.
3- The average person does not stand a chance.
Maybe there's something about Harvard that attracts people who are obsessed with rankings...
Hardy said in his "Apology" that
"Exposition, criticism, appreciation, is work for second-rate minds." Time has now come to add "ranking" to that list.
I would say, somewhat simplistically, that the obsession for ranking is a cultural matter. When you live and work in the US, as a capitalistic society, it is inevitable to become "obsessed" with rankings, because this is the way the society behaves: it's an ideal or a value in itself to "be the first", to "win". And since the gaps between rich and poor are so big, you better be the winner. This concept of a looser and a winner is not necessarily true, or at least doesn't hold the same strong meaning, in other societies.
The full quote is:
There is no scorn more profound, or on the whole more justifiable, than that of the men who make for the men who explain. Exposition, criticism, appreciation, is work for second-rate minds. --Hardy
This quote is particularly striking given that Hardy wrote over a dozen books himself.
Why do you pay any attention to what Hardy writes. He was a great mathematician, but his non-mathematical statements are quite often thoughtless and superficial, and so should be ignored.
Why do you pay any attention to what Hardy writes. He was a great mathematician, but his non-mathematical statements are quite often thoughtless and superficial, and so should be ignored.
Like his other Cambridge contemporaries -- such as Russell, Littlewood and a host of others etc. Hardy was not just a mathematician but a deep thinker, a rabid dissenter, socially and politically aware -- really a towering intellectual of his times in a sense that most modern day researchers (especially in TCS) are not. To say that his writings were thoughtless and superficial (even if one does not agree with the contents) is just nonsense -- perhaps indicative of the intellectual depths to which we have sunk collectively.
Anon 7 -- thanks for the full quote.
I don't think ranking "is work for second-rate minds", it's more like what many of us need and use because we have to make many decisions within very limited time and with limited resources.
If I want to buy a TV I'll go to amazon to read the comments and and hence I'm using indirectly some rankings. I don't want to become an expert in TV technologies to make the decision: I want to rely on the expertize (?) of others.
Parents and school kids want to see how a given school is doing in the rankings, because they want to make their choices for undegrad schools based on most reliable sources and this is the first general information they can get access to.
Deans want to assess (at least approximately) the quality of their faculty without the need of being expert in a given field, and citation indices or impact factors are of some help for them.
Most of us are hopefully smart enough to know that all these ranking are subjective, based on some partial information, etc. But still, they do provide some valuable information; even if it is incomplete.
A
1. The quote of Hardy you gave is clearly superficial. It has no argument in it. Just a bold and provocative statement.
2. To quote someone doesn't constitute any meaningful argument. It is immaterial who you are citing when the subject of the quote is not an empirical fact or an outcome of some systematic thinking. This is not the case in the quote above, which is on matters of taste.
3. I am not aware that Hardy made any deep philosophy. His philosophical achievements are scarce if any, and could hardly be compared to an influential philosopher as Russel.
Please correct me if I'm wrong, by listing his main contributions to modern philosophy.
The quote of Hardy you gave is clearly superficial.
That was my point (this is anon 7). The quote was so poorly thought out that Hardy himself contradicted it with each of the twelve or so books he published.
I'm under the impression that academics are "obsessed" with rankings for two reasons: first, because resources are limited; and second, because academia has chosen to allocate limited resources based on merit (as opposed to, say, nepotism).
Ratings are unavoidable in a meritocracy. Having said that, I agree that ratings can be overdone.
That was my point (this is anon 7).
Yes. My comment was to Anon 9.
(It would have been easier if we picked names...)
That was my point (this is anon 7). The quote was so poorly thought out that Hardy himself contradicted it with each of the twelve or so books he published.
Not so. If you read it in context, it is more of a lament that he is reduced to doing second-rate work i.e. write about mathematics rather than doing it -- because he considered himself past his prime. Mathematics as he says in the same book is a "young man's game" --he gives the examples of Galois, Riemann, Ramanujan
all of whom did their work (and died) quite early in their lives.
Also, Hardy does not say that "exposition, criticism and appreciation" are not important. All he says is that a first-rate mathematician in his prime should not be doing any of this (at the exclusion of mathematics). In *my* opinion the same is true for this industry around *rankings* in academia. Thankfully though, the first rate people in academia do tend to stay away from these exercises.
Mathematics as he says in the same book is a "young man's game"
Which has been thoroughly disproven as it can be fully explained by life expectancy considerations.
All he says is that a first-rate mathematician in his prime should not be doing any of this (at the exclusion of mathematics).
Luckily most first rate mathematicians ignored him on this one. The list includes Smale, Gowers, Tao, the entire Bourbaki group, etc.
Also, Hardy does not say that "exposition, criticism and appreciation" are not important. All he says is that a first-rate mathematician in his prime should not be doing any of this (at the exclusion of mathematics). In *my* opinion the same is true for this industry around *rankings* in academia. Thankfully though, the first rate people in academia do tend to stay away from these exercises.
Eating, sleeping and writing comments on blogs are probably third-rate, or even fourth-rate activities. Any idiot can do those. Therefore, a first-rate or even a second-rate mathematician, should not do any of those.
Sarcasm aside, what makes someone a good mathematician, or a good computer scientist, is what she does, not what she avoids doing.
In any case, "not be doing any of this (at the exclusion of mathematics)", and "staying away from these exercises" are two very different things.
Academics tend to finish at the top of their classes all their life. Why would that change?
Ranking == modern caste system?
Recently, I came across an interesting TED talk: http://www.ted.com/talks/alain_de_botton_a_kinder_gentler_philosophy_of_success.html
which made the point that people increasingly evaluate others on the basis of some measure of 'regret' - in a free country, everyone has the potential and freedom to be a billionaire, so if one isn't yet a billionaire then clearly one is failing.
Stated monetarily, this kind of counter-factual reasoning sounds absurd. However, most academics subscribe to some version of this when evaluating others.
So, the obsession with rankings seems like a natural byproduct of this. If all of us are eligible for major prizes then those of us not winning them are 'losers'. This is too harsh, so maybe we can find some metrics by which we are not such 'losers', afterall.
The really smart academics (and people, at large) catch on to this and realize that the main reason they entered this game was something else and it may be better to focus on that...
Post a Comment