Several days ago Lance wrote about "Is Complexity Math or Science?", and in that context wrote "We don't do experiments...", which has caused me to overreact.
I'm not sure who he meant by "We" -- I guess complexity theorists -- but I found the statement very strange at the time. I do experiments all the time. (Note: Here I am considering computer simulations of various sorts as "experiments". Some people may quibble at this, but since it seems ALL THE REST OF SCIENCE is moving toward increasing use of computers, including and perhaps especially computer simulation, I think it would be odd not to call a computer scientist running a simulation an experiment.)
For example, a couple of days ago, I thought I had proved something about random sequences for a project I'm working on. I asked the graduate student on the project to code it up to check my work -- I often sanity check proofs with simulation code when I can -- and he shipped me some results that seemed surprising. They were consistent with my proof, but effectively showed that things behaved even better than I had proven (or expected).
So on the car ride home, I thought about it, and came up with what I think is a nice proof that explains what the student found in the simulation experiments. This improved proof will end up in the eventual paper, I'm sure.
While I wouldn't call myself a complexity theorist, it seems to me the results I'm working on here are in the class of complexity results -- I'm trying to show that certain permutations have certain properties with high probability, and I'm showing it by developing an algorithm that allows me to prove what I want. Perhaps that's sufficiently far removed from "complexity theory" that some people think it doesn't count, but then it seems you'd have to throw the whole probabilistic method out of complexity theory, which seems strange to me.
My point is that the actual use of computers -- for example to simulate processes in order to sanity check proofs or develop insights and conjectures -- is part of how I do my theoretical research. While certainly that approach might not be for everyone, I have deep concerns when a major-blogging-theorist says something like "We don't do experiments..." I worry that increasingly that computer science graduate students in theory are loathe to actually use computers, and indeed that this is part of a broader problem that theory students specialize narrowly so early they don't get exposure to and an understanding of the rest of computer science.
So I'd just like to say, I personally find experiments very helpful.