I've long felt that its a fallacy that there exists a fine-grained Platonic ideal of "goodness" for researchers (so too for papers), but its an even bigger fallacy is to expect that decision makers would abide by such a scale even if it existed. In my experience, job offers are job offers, just as paper acceptances are paper acceptances. Trying to analyze such results at a finer or deeper scale is unlikely to reveal many useful truths.The whole comment, well worth reading, can be found somewhere in here.
There seems to be in the previous comments (mostly from anonymous commenters) the idea that getting a job is like those contests many of us did back in high school -- you get more points than the next person, you get the prize. This idea, in my mind, requires some underlying assumptions. First, that merit can be precisely measured -- if you get a high enough score, you get the corresponding job, and anything else is a failure of the system. Second, merit [for a position at a top research university] corresponds explicitly to quality of research, and again, using other considerations is a failure of the system. (I should point out these ideas are in no way novel; indeed, this argument seems to arise constantly in debates on undergraduate admissions, regarding admission of underrepresented minorities/legacies/athletes and so on.)
I think both assumptions are invalid in the setting of faculty hires. First, even if you think research quality is the sole criterion on which to base a hire, how do you measure it? Number of papers? Number of citations? Practical impact/number of actual users? Convene a panel of experts to assign a score? There can be, and will be, disagreements; in some cases, only the test of time will tell. Of course it's often easy to separate "the top" as a rough equivalence class, but going beyond that to a rank ordering is often difficult, especially when comparing people in even slightly different research areas.
Second, I don't think research output alone is the sole measure for a faculty position. Obviously, there's teaching, advising, and administration to consider, but there are other less tangible issues as well. Joining a faculty is like joining a team, and the question is what person can best help the team -- the quality of a team is not merely the sum of the quality of the individual members. Will the potential hire collaborate with others, fill in an area where the department needs someone, or offer useful leadership? Can they fit into, and enhance, the department culture? And yes, the question of is this someone everyone can get along with for a couple of decades also comes to mind. Certainly research quality is a primary consideration -- really the primary consideration -- but most or all of the people brought in for interviews have passed a very high bar for research already, and the other issues can come into sharp focus in the late hiring stages. People might skip such considerations for a suitably good researcher -- I imagine many departments, for instance, would take a Turing award winner, even if the person had a destructive personality, assuming the benefits would outweigh the costs. (I don't actually know of a case like that, but the issue has come up, as a purely theoretical issue, in discussions on hiring in the past.)
This may not be the way some people wish things would work, but it's counterproductive to not recognize that this is the way it generally works -- as Stefan suggests. Further, I strongly suspect that the idea that a pure "merit-based" system, whatever that means in this context, is the universally right approach to faculty hiring is based on assumptions that are faulty in both theory and practice.
[Interestingly enough, I recall a similar topic comes up in the Justice class I posted about before; I'll have to review those lectures!]
13 comments:
Whoever thinks hiring decisions are solely "merit-based" has obviously never served in a hiring committee.
The process of choosing the new Miss Universe is way more fact-based than choosing a new faculty member.
Granted, I am biased as I totally tanked on the swimsuit competition at Stanford.
Well, the simple answer to everyone's question is that hiring committee members are human beings. They can never be impartial, they are always biased.
So Prof. Mitch or Mihai should stop writing these posts and just accept the fact that people are inherently biased.
And, I think that it is not wrong to be biased. It is only in US where equality and equal opportunity is emphasized but in reality such utopian concepts can never exist
There are many very good researchers. (More than maybe is realized, because the best research isn't always, or ever?, the most hyped.) If you want a job, you need to make the case that you add more to the department. If all you contribute is research and nothing else, then why should MIT care whether you do it there or at AT&T?
I think some people may be missing the point. Of course, human beings are biased, but I think focusing on this can lead to a mistaken impression that, if only people were unbiased, then there would be a universal clear and true decision criteria for selecting between candidates. I think this is the important fallacy to see past. Hiring is multi-dimensional and there are many different criteria that get evaluated -- some implicit, some explicit, some easily quantifiable, others qualitative almost to the point of being ineffable. People wave their hands and say "research quality" or "merit", but indeed this is not one single thing by a long shot, nor is it by far the only aspect that gets evaluated. These are not issues of bias (at least in the sense the word is typically meant). On top of that different institutions have different needs and cultures and may weight these aspects differently. Then we _also_ add bias which I can assure you exists, but is both more subtle and more complicated than I suspect most people appreciate unless you spend a lot of time on the phone chatting up a broad set of recruiting chairs.
I'll point out that I witness the same fallacy on the other side as well. That candidates both will and should select jobs based on some universal ranking between positions, as though selecting a job is a uni-dimensional optimization function that has already been largely solved for everyone by US News and World Report. This too I believe is based on an overly simplified model of "what matters" both in research and in life. To wit, MIT is a great place, but that doesn't make it the ideal place for everyone or in every circumstance.
In general, I think generalizations about how decisions are made -- both on the part of candidates and employers -- tend to be poorly founded and, while they clearly serve as bit of shared short-hand in our academic culture, they don't typically reveal many useful truths.
Stefan -- Thanks for the follow up. I was myself going to say that the first two anons show that some people are simply going to refuse to get the point. Perhaps your explanation will at least help them see that just because what they personally interpret as merit is not how job decisions get made, that is not the same as the process being capricious, misguided, or even unfair.
Nice post professor. But, I still think that we are spending too much time on this. I completely agree with what you have said, and in a way you strengthen my point.
When I say biased, I mean other factors like a person is a minority or not, women or not etc.
I am sure you cannot deny the fact that given two potential faculty members, the committee will give more weight to the female candidate. This is what I meant by bias.
I am sure you cannot deny the fact that given two potential faculty members, the committee will give more weight to the female candidate. This is what I meant by bias.
Outwardly they might, but internally many generally often do the reverse.
You can find a large number of studies in which the same identical resume is sent with a man's name and with a woman's name. Even today with EE and AA and all that, the man's resume gets more offers than the woman's. A similar study was performed with "white names" and "black names" with even more dismaying results.
I agree, there are biases in the system. Sometimes they work against you, sometimes they work in your favor; and unless you are a member of a historically disadvantaged group the impact on your career ought to be minor.
I imagine many departments, for instance, would take a Turing award winner, even if the person had a destructive personality, assuming the benefits would outweigh the costs.
It is essentially impossible to get a Turing award if you are disliked by a majority of your community. Some people need to nominate you, push your case, and vote for you. The vote is going to come down to some pretty incomparable folks (say, somebody from theory and somebody from operating systems). If people don't like, they may (perhaps unconsciously) find the case for the other candidates more compelling.
All in all, it's always a popularity contest. It just happens that research quality often makes you more popular.
It is essentially impossible to get a Turing award if you are disliked by a majority of your community.
For sure, however do not underestimate the number of people who project a cordial and friendly image to the community at large while being abusive and destructive within their departments.
Just a few years ago the press had a hard time finding a colleague who would comment on the career accomplishments of that year's Turing award winner since he had created so many enemies locally over the years.
Curiously, in discussions like this people tend to focus on the rare counterfactuals rather than the mundane parts that actually dominate the hiring process. To put it another way, the empirical truth is that the number of women and minorities (to say nothing of Turing Award winners) who land interviews in the academic job market each year (esp for R1 schools) is small compared to non-minority males and thus, even if there was a supremely strong bias in their favor this would not significantly impact the outcomes for most other candidates (and even in the cases where such a “match up” occurs, I think both the direction and magnitude of any such bias are far from clear cut).
To put it another way, these are not likely to be the reasons you don’t get a particular job. The important factors, research aside, are far more mundane but largely under-appreciated. First, how do you land an interview? Here’s the priority list: 1) someone in the organization already knows you and suggests your name (this is frequently the majority), 2) you have strong letters from well respected researchers in the field (note: “strong” is a complex term here… relatively few faculty are really good at writing letters – the kind that place the candidate in the right context for readers to appreciate them – and there are quite a few faculty, even quite famous ones, whose letters are uniformly discounted because they routinely oversell), 3) the quality of your research as revealed by publications. Note that publications etc ranks last. Now generally (but not always) if you did well in categories #1 and #2 then #3 works out as well, however the converse is not at all true. Recruiting committees by and large cannot be expected to read all the papers of their applicants and thus rely on attestations from their own members, from letter writers or of third-parties (e.g., I field calls each year from other schools asking my opinion about various candidates they are considering in systems/networking/security). Most of this you can’t control… although good social and networking skills plays a significant role in #1.
Now lets say you do get an interview. Guess what? Most of the factors that got you there no longer matter. New evaluation criteria apply. The easiest way to fail here is during the visit. The talk was too hard to understand, didn’t “click” with some researchers in one-on-one meetings, weren't energetic/enthusiastic enough, no sense of vision, seemed confused when handling questions, too standoffish, too arrogant, subject matter seemed too theoretical, subject matter seemed to applied, etc… There are now a whole bunch of people who are evaluating you both as a researcher (because they believed the letters enough to invite you, but were skeptical enough that they wanted to kick the tires themselves) and as a colleague (would I want to work with this person? be in meetings with this person, etc) and depending on what they value, they come to a different conclusion. This aspect you have some control over because much of it is gated on your own behavior. Perhaps this bothers people, but I think it’s a ridiculous position, bordering on solipsism, that one’s research should be enough. A tree falling in the forest. Academic research is a social discipline and much of our responsibility as researchers is to communicate our results clearly and advocate for them. Why they are interesting, valuable, exciting, etc. The best way to accomplish is generally to cultivate the interest of others. This _starts_ with doing the research, but that’s not where the responsibility ends.
[continued from above, I'm not made for the blog format]
Now after all these come the external issues. How many positions are available? What kind of positions are they (e.g., do they have any restrictions associate with them)? What kind of search is going on? Is it a broad search across all areas or a focused search? How do the different constituencies feel about hiring in different areas? (as an aside: when you interview at a university you are generally trying to convince the people _not_ in your area. If the people in your area don’t want you, then you’re dead anyway, but they can’t get you a job typically because they are a single constituency in a quasi-democratic process. You need to convince the folks in the other areas that it won’t be such a big sacrifice to take you instead of the superstar in their area they’d really like to hire). How are the other candidates regarded (both in your area and others)? Who came to the hiring meeting and who happened to be sick that day? What are the interests up the “food chain” (e.g., is there an interest in “synergy” with some other division/department? How do they feel about junior vs senior hires? What are instructional needs? What is the track record/yield? What is the budget for next year looking like?) This stuff has virtually nothing to do with you, but will almost always come into play.
Hiring is a messy chaotic process. Its not simple, there aren’t simple rules or biases that govern it. Moreover, it can be quite different at different institutions. This isn’t to say you can’t reason about it, but much of that reasoning doesn’t generalize. The parts that do generalize are pretty straightforward: do good work, try to appreciate the field broadly, work hard on your communication skills, both presentation and networking/social, and try to be someone who is enjoyable to work with (or at least fake it). You can succeed without having all of these, but its much harder to do so.
"do good work, try to appreciate the field broadly, work hard on your communication skills, both presentation and networking/social, and try to be someone who is enjoyable to work with (or at least fake it)."
--- So true. Yet so difficult to follow :)
There is also the issue that faculty are expected to teach and train students. If someone is unpleasant or unable to handle interactions with other people normally, chances are that they would make for bad advisers. Even faculty who are otherwise socially skilled can turn out to be obnoxious jerks when it is a matter of mentoring students.
It is not reasonable or fair to hire someone who is either unable or unwilling to show the minimum skills required to teach and mentor graduate students.
Post a Comment