Sunday, February 08, 2009

STOC PC Meeting : Part V (Software)

I used Shai Halevi's conference software for STOC. I must admit, I was swayed by Shai's willingness to host the software and help out in a pinch -- I didn't want to deal with Harvard's IT staff being off-hours should a disaster happen. And while no disasters happened, Shai was a great help whenever anything did come up. I (and the rest of the community) owe him thanks for volunteering his time specifically for STOC, as well as more generally for creating the software. I really can't imagine running a PC meeting without such software.

Shai's software seems very similar in spirit to HotCRP, which I've been using recently as a PC member for some other conferences. I like them both. I'd have to say HotCRP seems a bit more polished, like it has gone through some more iterations, and has some more features than Shai's software. On the other hand, if you like control, you can apparently readily modify Shai's code (and the resulting conference database) directly to get what you want. I prefer them both to my experiences with EasyChair. If you're going to chair a conference, you should definitely check out Shai's page.

It might be worthwhile for SIGACT (or some other appropriate group) to look into having a common hosting platform and support for this sort of stuff. (Someone else I knew was using HotCRP for her conference, and when I asked how she set it up, she said that USENIX, the sponsor organization, hosted the conference software for them...) A standard default configuration maintained centrally would avoid having to "re-invent the wheel" each time someone takes over the conference. Or perhaps in the end we'll just leave it to EasyChair, which offers a perhaps less appealing standard default, but at a good price (free, currently).

7 comments:

Anonymous said...

At least once in the middle of the meeting EasyChair went down. So if we had used that, we might have had a problem, though that's only an issue because of the short duration of a physical meeting. And one outage doesn't really say much about the relative reliability of different systems.

Jonathan Katz said...

Having just been a PC member for a conference that used HotCRP, I have to say that HotCRP is awful. Too much clicking was required to view reviews; there was no easy way to get a quick, global view of papers and their scores; and it was hard to identify the latest updates of reviews.

Michael Mitzenmacher said...

JK --

Always nice to have a different opinion. :) I should be clear that while I haven't found HotCRP so awful as a PC member, I have ONLY used Shai's software as chair, and have ONLY used HotCRP as a PC member, so I'm probably the wrong person to make an apples to apples comparison.

But I'll take your comment as a strong vote for Shai's software!

Yuriy said...

I have heard some pretty amazing things about CyberChairPro ( http://www.borbala.com ). It costs money, but for large professional conferences for which you are willing to pay a little to have live support and more functional and well-working features than the free options, CyberChairPro is a must. The top-tier software engineering conferences (e.g., ICSE) refuse to use any other software.

Anonymous said...

Michael, thanks for using my software.

I wholeheartedly support your suggestion in the last paragraph that organizations such as SIGACT will maintain a server that provides this service to their members. (I am currently doing the same for the IACR, which is how I was able to host the STOC submissions and review.)

One aspect that is worth pondering is the long-term maintenance of the software. Both the needs of the community and the available technologies keep changing, and the odds that a one-man project like my software will be able to keep up with them for many years are quite small.

Maybe there are enough people out there with too much time on their hands that will keep providing us with good volunteer-based systems for our conferences. But if not, then at some point we would have to start using more "established" systems. (And if that happens, then I truly hope that it will be something better than easychair.)

Michael Mitzenmacher said...

Shai,

Again, at the risk of being repetitive, thank you for your incredible support.

We agree that, at some point, institutional maintenence rather than individual makes the most sense -- probably at a point where changes to the system are sufficiently rare. I admit, I don't know how one would move, for example, your code (if you volunteered it) into such a framework. Perhaps others have an idea, or it's something SIGACT as a body could look into.

JS said...

What about EDAS? At least from an outside perspective, it looks and feels more professional and polished than EasyChair.