Wednesday, January 14, 2009

Graduate Student Folders

When I'm not looking over STOC papers the next few weeks, I'll also be looking over graduate student admissions folders. (At Harvard, we're small enough that the theory folks just split up all the folders and then meet to discuss them.) Even though I've been doing this for a while, I'd love to hear from all of you -- what should I -- or we as a community -- be looking for in selecting graduate students?

I think that on the theory side we tend to look for pure brainpower over other skills. Arguably moreso in theory than in other subjects, raw intelligence matters most. (I'm not saying I agree with that argument; I'm just saying it's an argument.) I think we also know that grades don't necessarily provide the right information we need to judge this, however, and so we look for evidence that one has the ability to do research in the form of actual research projects accomplished or letters from faculty members we know (and believe) who suggest that there's intellectual power and creativity there.

I do think most faculty do keep in mind the other skills that can make for a successful student. Communication skills, the ability to work well with others, a sense of what they want to accomplish, the mindset to deal with the one-step-forward-two-steps-back nature of research -- all of these also come into play in decision-making. These can be much harder to judge, however, unless they're clearly marked by their absence in some way.

I wonder if the competitive landscape for graduate slots at the top schools causes us to miss out on some promising people. Over at the FemaleScienceProfessor blog, she discusses why she'd rather take the motivated B student over a lot of A students for undergraduate research projects. I'm curious, having no evidence one way or another, do people think we're doing a good job getting B undergraduates who could become great researchers into the pipeline early on enough so they can construct a good application for graduate school? Do we accept enough of these students when they apply?


Anonymous said...

I wonder if you could give some specifics about how you weigh the various factors.

i. Grades.

How do you compare GPAs across countries? Is an 8/10 from one of the iits the same as a 4/5 from a russian university and a 3.2 from an american one?

Is there a threshold beyond which a higher GPA does not improve your standing much?

How do you currently deal with
the issue you discussed at the end of your post( B students)? To be more concrete, suppose you had an application from a student who seemed quite bright (say judging by undergraduate research and recommendations) but had a considerably lower gpa than usual?

ii. What is the typical amount of undergraduate research done by someone admitted?

iii. Background.

Would you ever accept someone with a math degree with no background in cs? What about physics? Would these be less advantageous or not versus an undergraduate cs degree?

I know this is a lot of questions, but it would be great if you could answer some of them...

Anonymous said...

These days among grad applicants there are fewer and fewer students who haven't had the opportunity for undergraduate research. You really don't have to be one of the very top students to get involved. However, the quality of this experience varies dramatically and this variation often has little to do with the student and much more to do with the properties of the particular faculty they've happened to interact with.

One worry about all this undergraduate research is that students will tend to keep doing precisely what they've been successful at which will make them much narrower graduate students; they have specialized based on much less information than they would have had access to in grad school.

Anonymous said...

It seems to me that there is a certain threshold up to which more brainpower directly correlates to higher success but beyond that it does not. From my experience in grad school and also now as a professor and supervisor, some of the best brainiacs fail and some ok but not so great minds go on to great accomplishments. I'd place that threshold somewhere in the upper 1-3% of the general CS population and upper 10-20% of students in an elite school.

FWIW: I was very much a type A student (straight A's, IQ in top 0.1%, started doing research in 1st year undergrad, you name it), so this defense of Type B students is not a self-serving remark.

Michael Mitzenmacher said...

Paul --

I'll try to pay attention, but while I certainly see more undergraduates who have tried research these days, it's certainly far from universal that applicants have research experience. For various reasons -- the school they went to, their extracurriculars, their need to work to support themselves -- many students don't (or even can't) get involved in research as undergraduates, but are still interested in graduate school.

This is part of what makes it hard to judge potential students -- never mind, as you discuss, the large variance in research experiences that is outside the student's control.

Anonymous said...

I would go for a very motivated and focused student who has done a project by him/herself. That student can be an A or B student. I'll never consider grade A students that seem good at passing exams only.

Anonymous said...

I think the short answer is that you never know. I was a straight A student, but did not do any research as an undergraduate. And, indeed, it took me a while to be successful in research. However, I was lucky MIT still accepted me and gave me an opportunity. On the other hand, I saw students who: (a) were straight A, did not do much research, and painfully failed in grad school; (b) were B students, did some (not too deep) research, were accepted only because of that, and then failed to do more serious research, because of lack of knowledge/talent; (c) were B students who succeeded greatly in research by doing it early.

So one must always make a guess. Personally, I would almost always take an A student over a B student, unless the research done by B is already serious enough to judge (i.e., appeared at a major conference, and not just some incremental junk). But this is because I'm in theory and knowledge is important. In systems, I would imagine B with research is a much safer bet, because you are almost guaranteed success given enough patience to code and test stuff. Having said this, you never know, and need to gamble...

Anonymous said...

It seems to me that part of the problem here is the variety of undergrad courses and what grades mean. For example, a good undergrad theory course shouldn't just be giving fill-in-the-blank proof exercises, so an A should actually tell something about how well the student does coming up with "new" ideas (new to them at least). So perhaps the grade is less telling than the particular undergrad institution they're coming from.

Anonymous said...

Quoting Anon (January 15, 2009 6:09 PM):

"In systems, I would imagine B with research is a much safer bet, because you are almost guaranteed success given enough patience to code and test stuff."

Really? Wow. That simple eh? What about:

"In theory, I would imagine B with research is a much safer bet, because you are almost guaranteed success given enough patience to work out the math."

Michael Mitzenmacher said...

Anon 8 (and 6):

You appear to be using slightly different versions of "successful".

Generally, I believe a networking (or more generally in systems) a graduate student with reasonable technical skills, patience, and an advisor willing to find a good project, will be successful in completing their degree. There can be challenges along the way, but a minimal thesis can usually be managed, and in this sense Anon 6 is right. This doesn't mean that it's easy to become successful in the sense of, for example, turning into a good faculty candidate. For that, more significant skills -- in particular, the ability to choose good problems and come up with creative ways to solve them -- is clearly required.

In contrast, anonymous 8, for theory, patience and basic skills are just simply not generally sufficient to get through with a PhD. Your phrasing of "patience to work out the math" demonstrates a lack of understanding of what goes into a theory PhD, although I suspect you do understand and are just choosing a provocative phrase.

This issue comes up in my undergraduate theory classes all the time. Students who are used to programming complain that they can spend countless hours on a theory problem and not get anywhere. (The horror, the horror.) In systems work that is primarily programming-based, there's more nearly a linear relationship between time input and work output. Of course, there's the issue of figuring out WHAT should be programmed, but again, in systems work, if your definition of success is JUST to get out with a completed thesis, in my experience an advisor can more readily give you something to do.

So Anon 8, you're right to take Anon 6 to task, but your analogous statement for theory is, I think, way off the mark.

Anonymous said...

In response to Anon 1, I can only say what we do at my school. (Top 20 school; I have been on the grad admissions committee multiple times.)

> What is the typical amount of
> undergraduate research done by
> someone admitted?

As someone else pointed out, a majority of accepted applicants have some prior research experience. Note that this does not mean a published paper, or even significant progress. What it means is that you were interested enough in some topic to seek out a professor and become more knowledgeable in an area. This demonstrates enthusiasm and initiative; indicates that you will have a better chance of knowing what you want to work on when you get to graduate school; and will give you a meaningful recommendation letter that says more than "did well in my class".

> Would you ever accept someone
> with a math degree with no
> background in cs? What about
> physics? Would these be less
> advantageous or not versus an
> undergraduate cs degree?

Absolutely. Some of the best candidates come from math, and I have seen some good ones come from physics. But you have to make the case (1) why you are switching and (2) that you will be able to complete a CS course-load (which includes programming courses).

GASARCH said...

1) I was ACCEPTED to Harvard Comp Sci
in 1980 and went (it was probably called
Applied Math. I'll need to get
that right if I ever run for public
office). AT THE TIME I looked
like I was very good at math,
had NO programming, and NO

2) I'm ON the UMCP grad admissions
committee. For a theory grad student
I like to see math (doesn't need
to be a math major, but needs
to have taken some extra courses in it). I also like to see a
PROJECT-- some sort of research.
Where I differ from my colleagues
is that its OKAY if it didn't get
published--- I just want to know
that he or she did SOMETHING.
If its enclosed I'll read it.

3) As for brainpower vs hardwork:
Its never that simple a discussion.
I WOULD NOT want someone who
is brilliant but lazy or
someone who is hardworking but
stupid. Judging what the right
combination is and if the applicant
has it can be difficult.

bill gasarch

P.S. You can ask Harry Lewis why
Harvard accepted me and if it
was a mistake.

Anonymous said...

I think Michael's emphasis on success as completing a thesis is important. The last thing we (faculty) want is someone who will not complete a thesis.

In that sense "brainppower" is important. If someone is *extremely* talented in math, then they're likely to be able to complete a theory thesis even if not a great one. Also, math skills are relatively easier to measure - they are visible at an early age. In contrast motivation may only appear when the student found the right topic.

The short answer is that at the end it's very hard to predict the future, and indeed there are many current theory superstars who were not admitted to their top choice. But when you have to make a decision, you go with whatever information you have.

I'd guess that a place like Harvard can't afford to take a chance on a B student, even though such a bet will pay off handsomely once in a while. Still as i said, from past examples it seems these student will still go on to be successful at his/her second or third best choice, and the loss is only Harvard's.

Grades in certain courses, achievements such as math olympiads, all-India exam, etc.. are good indicators of math ability. In addition we take a very deep look at the recommendation letters. If a certain person has built trust over the years, by recommending the best people enthusiastically and giving clear indication of who is only second best, then we give great weight to their opinion.

Anonymous said...

I think the admissions process should be more open minded. I've noticed that depts typically go over the top student from the top school who worked with the most famous person at that school. Obviously, these people are likely to be successful. However, there are a lot of students with "unconventional" backgrounds, or that (for geographic / financial reasons) came from schools where undergraduate research is fairly bad, and so they choose to work in industry instead during the summers. I think ignoring those candidates in favour of the more conventional ones is a mistake. It also makes the research at the institution somewhat more 'conformist' - since students with unusual backgrounds might have usual ideas to contribute.