Tuesday, August 25, 2015

CACM Viewpoints on Theory and Experiments

There's a fun pair of viewpoints in the September CACM by Jeffrey Ullman and myself on experiments in computer science research, with him addressing systems conferences(/people) being far too focused on experiments as the research validation methodology, and me addressing theory conferences(/people) being almost strangely averse to experimental results.  (This link may bring you a digital version of his viewpoint, and this link to mine.)  I hope they might be interesting reading or food for thought.  As someone who works in both camps, I find this separation -- which we both seem to think is growing -- worrisome for the future of the CS research community.   

We actually wrote these up about a year ago (or maybe longer).  Jeff wrote something on the topic on Google+, and I responded.  I think he got drafted into writing something for CACM, and then I got drafted in later.  There was a pretty thorough reviewing process, with a couple of back and forth rounds;  then there was a non-trivial wait for publication.  This seems OK to me -- I'm glad CACM has a non-trivial queue of items for publication.  Overall it was a thorough and reasonably pleasant publication experience, and it's appealing that CACM offers a platform for these types of editorial comments.


vz nuri said...

an old debate but one which it seems has not been definitively surveyed or studied anywhere. have attempted to compile a lot of info/ viewpoints myself on my blog. easily deserves an entire book. there are at least two schools of thought. some of this is due to CS being a young science and its fundamental boundaries have still not solidified. my feeling is that this is a very important issue & personally feel that the biggest problems in CS will only yield with a masterful "yin yang" balance of theory with experiment. anyone can see, they crosspollinate. neither should be devalued at the expense of the other. latest collection of links here & intend to write again. thx for a nice summary/ counterpoint.

vz nuri said...

oops sorry wrong link! see here, triggered by the konev/ lisitsa applied results on erdos discrepancy problem.