Wednesday, August 08, 2007

Death of a Research Laboratory

Most of you probably don't know about Mitsubishi Electric Research Laboratories (MERL), a small lab nestled in Kendall Square (very close to MIT, and a pleasant walk from Harvard). While they did basic research there, it was definitely more application/development oriented, and it specialized more in areas like UI, graphics, speech, etc. Information theorists would know it as the home of Jonathan Yedidia, a big name in coding and belief propagation, but CS theorists might not ever have heard of it.

I've done some consulting work there over the years. I knew the lab director, Joe Marks, from my time as a Harvard undergrad, and he hooked me up. It was a very nice place -- very theory-friendly in its application-oriented way. Joe clearly had the mindset that the goal of the lab was to generate new ideas, and that would drive new products.

So I was disappointed to hear several months ago that Joe had been removed as lab leader. (Don't feel too bad for Joe -- he's now at Disney, and is chairing SIGGRAPH this week. A talent like him will continue to be successful...) And even more disappointed (but not surprised) to read in Xconomy that MERL was being "re-organized", essentially phasing out the basic research component, and many people were leaving.

I've lived vicariously through this before -- I left Digital Systems Research Center before it disappeared (after Digital was bought by Compaq was bought by HP), but knew several people who went through the process. It's very disturbing to see again how hard it is for companies to make research labs work.

What makes research labs successful? Can they really last long-term in any non-monopoly environment? Let me make a bold prediction. I personally am not planning on retiring until at least age 65. Pick any research division that's around today. (Microsoft, Yahoo, Google, AT&T, Bell Labs, IBM, any of them...) I'd bet on pretty much any of them that their research wing will be gone before I hit 60. (I'm still under 40. OK, maybe I should just bet they'll be dramatically reduced or transformed into Advanced Development for a number of years somewhere along the way...) This is something people who go to research labs should know going in. Odds are likely you'll have to change jobs somewhere along the way -- not because of your own talents, but because of company-level problems. This might make such jobs seem risky, but let's not exaggerate -- when the company has problems, the talent moves to a new company.

I'd love to hear thoughts on what makes good research labs, why I'm wrong about research labs dying out, how people who have experienced such moves feel about them, or anything else on the topic.

Update : Jonathan Yedidia says in the comments that the reports of MERL's death have been highly exaggerated.

18 comments:

Anonymous said...

How old are you now ?

Michael Mitzenmacher said...

Still sub-40, though getting closer...

Anonymous said...

IBM --- the company is overly beaurocratic, inefficient, and catering mostly to bigger, stupider organizations (eg government). So the labs are safe in this environment.

AT&T --- the company has just come out of bad times, and the lab has lost a lot of people in the process. The business cycles are in the 10 year range, so they should be safe and growing for a while.

Bell Labs --- with Ken Clarkson and Bruce Shepherd as the most recent victims, the lab is all but officially dead.

Yahoo, Google --- labs are rather development oriented now... It's not clear they will take off in the direction of basic research, so it's not clear there is even a future to discuss.

Microsoft --- going extremely strong now, but MS may go down like all big monopolies. Still, they probably have a 20-year future, since they're too big to disappear all of a sudden. Plus Microsoft is among the big+stupid organizations, almost like IBM.

Anonymous said...

As an undergraduate in Computer Science who is planning on going to graduate school (and devoting his life to research), this is unsettling. Does this mean that the only place to conduct research when I graduate will be in academia? After reading blog article after blog article about how the number of potential researchers in academia far surpasses the number of available spots, what should I someone in my situation do? Head for the hills?

Anonymous said...

Get a job in IT and do research on the side ;)

Anonymous said...

My advice would be to head for Wall Street.

Anonymous said...

Recently saw that Jonathan Yedidia has a new blog:

nerdwisdom.com


-r

Anonymous said...

Hi Michael,
It's nice to see you've got a new blog too. To the point of your post, the reports of MERL's death are greatly exaggerated.
We're hiring!

Jonathan Yedidia

Michael Mitzenmacher said...

Jonathan -- Hi! Long time no see! I didn't know you started a blog -- but now I do, and of course it's going on my reading list!

I'm glad to hear that you think MERL will survive this re-org, and hopefully it will continue to be a home of great research.

Michael Mitzenmacher said...

To the various negative anonymous posters -- hopefully some more positive research lab people will post tomorrow. I'll just say that I greatly enjoyed my time working at a research lab, and I still enjoying visiting labs whenever possible. For those who really want to do research, IT and Wall Street just won't cut it.

The downside this post (over)emphasizes is that working in a research lab is a corporate job, and it's not really possible not to be affected by changes that occur in the corporation. But to the anonymous undergrad, there will always be research jobs; it's just that at times you may have to switch companies if you work in corporate research. When my old lab folded, people went to Microsoft, Google, HP, or elsewhere, but as far as I know people who wanted research-type jobs got research-type jobs.

Suresh Venkatasubramanian said...

As an ex-research lab person myself (AT&T; 7 years, through the turmoil), I think Michael is overstating the case for the demise of research labs.

Let's ask two questions:

1. Is there a place for research labs in the grand scheme of things ?

The answer to this is a simple yes. If anything, the kinds of large data and network problems we deal with today are even more taxing than they used to be. I think it's not an exaggeration to say that we're only scratching the surface of how to manage large data sets and extract useful information from them.

Universities are not geared to this kind of work; it is difficult for professors to get access to the kinds of large data sets that lead to interesting problems, and the nature of academic research is that you want to think longer term, and further out.

I view research labs in general as crucibles for nontrivial interesting problems, especially in data analysis and networking. You need researchers who can sit on large data and pose the really interesting questions: questions that go to the heart of key practical questions, while being intellectually deep in their own right.

Once the questions are posed, then it's anybody's game of course, and universities are well suited to taking these problems, chewing them up, and spitting out solutions. And sending grad students to labs helps expose them to these raw unformed problems as well.

So this is in an ideal world, so to speak. The other piece of this puzzle is:

2. Are current research labs likely to evolve and grow to foster this kind of research ?

For many years, AT&T Bell Labs was the model of a good research laboratory, its success driven in no small party by its monopoly status in the telephone industry. Money was easy, problems were aplenty, and the best of the best wanted to work there.

But with the breakup of AT&T, the gradual decline of Bell Labs, and the tight corporate focus of IBM (to name the big labs), focus shifted to places like Microsoft (and MSR is much more in the mold of classic research labs), and now Google and Yahoo. It's hard to look at the research lab landscape today and imagine that there was a time when research was not tightly linked to larger corporate goals, and people were free to explore problems as they saw fit.

But even in this climate, corporate research is nowhere near dead. Google, much disparaged for its "one day a week" policy on private research, still managed to churn out profound and innovative new ideas. MapReduce is one such example. Yahoo is much closer to a mainstream research model, and all kinds of interesting work on auctions, pricing, web analysis, and machine learning is coming out of there. MSR again is an interesting place to be, with many many stellar researchers, starting with Laci Lovasz and Uriel Feige (just to talk of pure theoreticians) taking up residence there.

And even AT&T, my first job, is coming back from the dead: I've just come back from a visit, and although the problems people are looking at might have changed, there are many many interesting problems to be examined there, and the labs is trying to grow and expand its research presence, which I have no doubt it should be able to do.

Why is this ? It goes back to my first point. I believe there is a structural role for the kind of research done in labs, and more importantly there are many researchers who genuinely prefer that environment, because of the potential for direct impact, the access to real (not "real") data, and the lack of other distractions that take away from research.

So I'd be willing to take Michael on with his wager, although given that labs evolve and change, we'd have some serious work to do to nail down an actionable bet :). Maybe we need intrade.com for this.

Anonymous said...

nice post (one of several here) - i dont know about research labs but i will say the complexity blog is headed towards its demise for sure.

Anonymous said...

Demise is too strong a
word. CYCLICAL- research
labs will come and go
as companies learn that
basic research pays off,
then in hard times forget
this, and then re-learn
it again.

As for Sam Jacksons's
comment on Complexity Blogs
demising, that depends
on what you mean by demising. If an entry
is written and there are
no comments, did it make
a sound?

bill gasarch

Michael Mitzenmacher said...

Suresh --

I don't think we're far apart at all. In my post, I suggested that current labs were likely to disappear, or go through some sort of major disruptive change, over the next 20 years. You point out in your comment how this has happened -- AT&T, Bell Labs, IBM, and Digital/Compaq Labs have all either disappeared or have gone through massive disruptions (leading to layoffs or large-scale exoduses) in the past decade. Sure, new labs (with arguably different priorities) have taken their place. I didn't suggest that corporate-sponsored research would disappear -- I'm sure other research labs will arise to take the place of current labs -- but this process is still very disruptive for individuals working in such labs.

I think Bill nailed it. Research labs are often cyclical, as companies learn that basic research pays off, then in hard times forget
this, and then re-learn it again. Sometimes in that cycle labs even disappear. That's unpleasant. I guess I'm asking if that has to be the nature of the beast?

Anonymous said...

Did IBM really go through "massive disruptions (leading to layoffs or large-scale exoduses) in the past decade" ? I am asking seriously, I just dont recall this happening. Or perhaps this is just a question of what a "large-scale exodus" is.

Cheers,
Piotr

Michael Mitzenmacher said...

Piotr --

I certainly remember a lot of talk about IBM's commitment to basic research flagging the last time there was an IBM bust. And maybe it's just natural attrition, or the lure of other opportunities, but I think the majority of people who were at IBM Almaden when I was a grad student have all moved on.

Anonymous said...

Michael,

The almaden example you gave is very different. People indeed left but I suspect this is mostly due to aggressive hiring by Google and Yahoo (sometimes for a more money/less research deal).

Having said that, I think IBM had a hard time in the early 90's, but I am not sure whether it lead to an "exodus".

Alvin Anony said...


I didn't suggest that corporate-sponsored research would disappear -- I'm sure other research labs will arise to take the place of current labs -- but this process is still very disruptive for individuals working in such labs.

Yes, it's disruptive, but is disruption necessarily a bad thing? I've changed from an academic position to a government lab to industrial labs (and then another industrial research lab). Each time I changed I lost some things along the way but gained others. Sometimes disruption can be a healthy thing because it can cause you think about what problems will have bigger impact.

The sad part is when people are caught unaware and unprepared for a tidal shift in employment conditions. This sometimes happens from economic conditions, but also from political conditions.