New Year's Affirmations

As the New Year beckons, I figured it was time for a post on affirmations, which I remember being interested in when reading about it one of Scott Adams' (of Dilbert) books (though I don't think he called them that). If you haven't heard of affirmations, it's basically the power of positive thinking. There are various forms; one is, take a list of goals you want to accomplish, write them down or repeat them to yourself every day, and you'll find they start happening.

Now, while I don't actually believe that positive thinking alone will allow me to prove P = NP (or the other way) in the coming year -- or, for that matter, win me a lottery! -- I do believe that the act of thinking clearly about the goals you want to achieve, and keeping them firmly in your mind, increases the probability that you will actually accomplish these goals. I personally find that when I set myself goals over multiple time scales -- a task list for the day, for the month, and for the year -- I'm surprisingly much better about getting things done. When I get distracted from setting goals, less happens. The conscious effort of writing tasks down and reminding myself of them makes them easier to accomplish.

So I encourage all my readers to take some time around the New Year and set some tangible, if difficult, work-related goals for the coming year. Maybe it's time to learn a new area or work with that person you've always wanted to work with. Or there's that result you know is just out of reach -- but you should keep reaching for it. Post them somewhere, remind yourself of them, and work to make them come true. I'd bet more do than you'd first think.

Whatever your goals are, best of luck with them, and for the New Year.

INFOCOM Miniconference

A number of commenters on my last post have mentioned the INFOCOM Miniconference. I hadn't actually known about the miniconference -- although I found out more details about it soon after the comments, as my second INFOCOM submission, rejected from the conference, was accepted to the miniconference. (I did have a paper in INFOCOM 2008, but my student Adam Kirsch went to the conference to deliver the paper, and I can't recall ever hearing anything about the miniconference format. Someone should have blogged about it before. :) )

100 additional papers -- a bit over 7% of the original submissions -- were apparently accepted to the miniconference, covering most of that "top 20-30%" range. The main difference appears to be the labelling (INFOCOM miniconference, not INFOCOM) and that the paper will be limited to 5 pages in the proceedings.

While I'm happy the paper got in, I must admit, I don't understand the reasoning behind the INFOCOM Miniconference, and I hope some readers in the know will explain and elaborate. If the purpose is to have a 2-tier conference, it seems an odd structure -- why not just accept 25-30% of the papers? (A few hundred pages in the proceedings wouldn't seem to matter much since it's on a CD?)

I wonder if theory conferences like STOC, FOCS, or SODA should adopt some sort of 2-tier structure in order to accept more papers. Certainly most people who have papers rejected from these conferences (myself included) believe they should have gotten in, and some fraction of them are probably right. On the other hand, such a structure would seem to lessen the prestige associated with these conferences. Any opinions?

INFOCOM paper, network coding

A paper I co-authored on network coding -- Network Coding Meets TCP (arxiv version) -- was accepted to INFOCOM. Full credit for the success goes to the graduate student Jay Kumar Sundararajan who led the project (and is graduating and looking for jobs this year...) Our goal (as the title suggests) is to make a TCP-compatible network coding congestion control scheme, and our approach uses an interesting variation on acknowledgments Jay Kumar had utilized previously; instead of acknowledging packets, you acknowledge "degrees of freedom" (or, encoded packets that will eventually decode to message packets).

The INFOCOM mail said 282 papers were accepted from 1435 submissions (post-withdrawals). A quick check shows that INFOCOM has been below a 20% acceptance rate regularly in recent years, and even assuming a completely unverified estimate that 10-25% of the submissions are things that really shouldn't have been submitted in the first place, in my opinion that's still a pretty low acceptance rate for what's supposed to be the big, open-tent networking conference of the year. (In the 1990s, the acceptance rate was more commonly around 30%.) I'm sure there were a lot of good papers that got rejected this time around.

Most networking conferences have acceptance rates around 20%. Is this a good thing? Conference competitiveness has been blogged about before, but there doesn't seem to be much of a high-level discussion about the issue -- I recently saw Ken Birman and Fred Schneier wrote an article about it for Communications of the ACM. Any ideas out there?

What Else Should Grad Students Be Learning?

Apropos of application season...

Graduate school is a long stretch of time -- 5 years (or more) for most people. There are few clear goals during that time, although the obvious one is to learn how to do good research, with the hope of getting a tenure-track faculty job. With this somewhat singular -- and difficult -- goal, it's easy to fall into the extreme of focusing only on your research to the exclusion of most everything else, or to waste a lot of time not really working. Both of these things may be OK for various individuals. (As some will undoubtedly respond, if you're doing great research, it can excuse a lack of many other skills. And many people don't mind spending an extra year at graduate school with a more relaxed lifestyle than life after graduate school.) But with the new year approaching, I thought it worthwhile to suggest some of the additional skills one should try to develop in graduate school over those stretches where you need to break from research -- skills which, unfortunately and understandably, are often given short shrift by the university. (Please add to the list in comments.)

1) Time management: How much are you working each day? (And how much time do you waste reading -- or worse yet, writing -- blogs?) Even if you don't set yourself to a regular 9-5 or 10-6 schedule, it's a good time to learn to manage your working and non-working patterns. My suspicion is that people who manage a regular work schedule graduate on average a semester or year earlier.
2) Writing/speaking: If ideas are our business, idea presentation is a big contributor to the bottom line. And if you want a faculty position, the ability to give a good talk to a general audience goes a long way. If your institution doesn't have a program for improving writing and speaking, start your own (like a student seminar series, no faculty invited).
3) Leadership: Find a way to lead a research project -- maybe advising/mentoring some undergraduates. Or organize a club or student group to make your department a better place to be. Eventually, the ability to organize people to follow your goals will make you more productive.
4) Entrepreneurship: Have you looked at the economy? And professor's salaries? Graduate school is where you're supposed to learn to be creative, and to develop specialized skills. It's quite reasonable to spend some of those creative efforts or utilize those specialized skills on money-making endeavors. While it's not for everyone, for some the tangible reward of money helps unleash creativity; for others, you may learn the satisfying lesson that your intellectual achievements bring you higher rewards than a big paycheck could (a lesson worth learning early on).
5) The skill to learn additional skills: If you're a theorist, learn to program a little. If you're a systems person, learn some probability or other theory. Maybe set aside a few days to learn time-saving Latex tricks, or some other piece of useful software. There are plenty of skills that will make you a better researcher/teacher/writer in the future.

NSDI Program Committee , Part II

Some lessons from a 1-day PC meeting (nothing really new, but I thought I'd write it down):

1) Face-to-face PC meetings involve far too much sitting. Especially if you fly in and out on the same day. (I know there's a time tradeoff in scheduling an "exercise break", but seriously...)
2) Conferences with 20% acceptance rates are, by their nature, a bit depressing -- it's hard to reject so many papers, some of which simply MUST be pretty good.
3) It's easier to argue how a paper is flawed than to argue about how it's making an important contribution.
4) Taking reviewer expertise into account is important, and one outlier can cause problems; sometimes papers live on longer than they should if one reviewer gives too high a score, and sometimes papers are put way lower in the ordered list than they should be because one reviewer gives too low a score. (Of course, one expert reviewer can also bring an otherwise ignored paper back to life.)
5) There are more interesting papers than paper slots.
6) There's generally plenty of down time, when papers you didn't read are being discussed; bring something to work on quietly (but pay attention to what's going on).
7) You really do learn a lot reading 20-30 papers for a conference PC.
8) As you go down the paper list by score rank, eventually (and sooner than you think) you hit a paper that you start to question -- are these scores too high? And as you go up the list from the bottom, you'll hit a paper where you question -- are these scores too low? The rules of randomness tell us that some papers will get comparatively mis-scored in the first round of reviews, so it is good to stop and talk about the papers (and not just take the first X).
9) Hot trends come and go.
10) A steady supply of drinks (mostly caffeinated) and a good lunch can help the PC move happily along.

Thanks to the PC chairs and other members -- I had a good time. But now I'm very tired...

NSDI Program Committee , Part I

I'm spending tomorrow at the NSDI (Networked Systems Design and Implementation) Program Committee meeting. It's been a few years since I've been on the PC for a networking conference, but I've found it so far to be a lot of fun.

First, it's very "civilized" -- I had only 20 papers to read the first round, and then had 5 more for the second round. (The first round was designed to get each paper 3 reviews; the second round was for paper missing reviews, or where the scores suggested another review would be helpful.) That's not too much, compared to most theory conferences.

Second, there are a good number of "algorithmically" oriented papers for me to read. Overall, networking has become a lot more theoretical, which is good. It still seems to me, though, for a network-oriented conference, you have to be careful not to go overboard with the theory. They want results -- backed by theory, preferably -- but at the end of the day, it's the results that matter. As usual when serving on a PC, seeing how it works gives insight into how to frame my own papers.

Third, one thing that's impressed me is how long and detailed the reviews are for this conference. I tend to write shorter reviews, covering what I think the high order points are. (And one thing that has been interesting -- there's generally a lot of agreement on these high order points.) But most reviewers go into a lot more detail -- the average review is at least a good page plus of text. Very different than what I usually find in theory conferences -- although I know there's a push to improve that.

I'm not sure why the culture of networking conferences has led to more detailed reviews. Fewer papers per PC member probably helps; maybe because few papers go on to journal papers (but that's equally true in theory, I think). But it's a marked and interesting change.

Anyhow, now I'm looking forward to SIGCOMM... except that I'll need to be ready to write longer reviews.

Summer internships?

The bad economy already has people thinking about jobs -- it is, I am sure, going to be a challenging year (years?) for people graduating.

I was wondering if there would also be an effect on summer internship programs. Summer interns are often a different budget line item, but it's hard to believe that the Microsoft/Google/Yahoo/everywhere else programs, for both undergraduates and graduates, won't be curtailed in this environment.

I haven't heard anything about summer internships yet, and though it's a bit early, late December/early January is usually when I start get reminders from people to have good students apply for the summer. Can anyone comment (anonymously if needed) if they have any actual information?

Online Advertising and How People Cheat

Ben Edelman (who actually does some theory when he's not doing law and business) gave a great talk today at the Harvard Center for Research on Computation and Society lunch lecture series. It was about Web advertising scams -- how people cheat pay per impression, pay per click, and even pay per conversion schemes, for big bucks. The talk was based on this book chapter (here available as a working paper). (If he gives me a link to the slides, I'll update the post with it.)

One of the scams he gave for pay per conversion schemes is related to this blog. I often encourage people to buy my book on randomized algorithms -- to remind you all, the Amazon link is here. Now, when I put on that Amazon link, I've embedded a link that includes my Amazon Associates information, so if you buy the book, I get a small cut. Very small.** In fact, if you buy anything else after clicking that link, I get a cut -- I actually think this link alone puts a cookie on your system, so if you happen to buy on anything on Amazon for the next week or so, I get a cut. I think I got about $100 in store credit from Amazon last year, from people buying the book off my home page or buying things after hitting an Amazon link on the blog.

So again, I'll encourage you to click that link. (Heck, really, while you're there, just buy the book!)

For me naturally the gain of doing this is small, but apparently, scammers have found a way to make big money off this sort of thing. They have sneaky ways of getting these cookies onto your system -- not just for Amazon, but for other vendors also, multiple vendors at a time -- so if you click one of their links, and then buy things on the Web, they get a cut for "recommending" the purchase to you, even though they've really done nothing of the sort. When you think about it, a 5% cut from the purchases of a million Amazon customers can really add up...

If Ben Edelman happens to be coming to present at a venue near you, I highly recommend you go. And that's not false advertising.

**Actually, not so small. I get almost as much from Amazon as I do from the publisher if you buy the book through the link. Of course, I don't get much from the publisher either.

The Funding Cycle

Most of the last week was spent working on an NSF grant proposal. (By the way, does anyone know why they kept the pdf and txt format of the call, but removed the html? That was annoying as I was trying to figure out exactly how many colons my title needed to meet government standards.) I don't have any really entertaining stories about it. That's just how I was spending too much of my time. November 28 is a funny day to have deadline, though. For us, that translated into a submission deadline of yesterday. (Is your Sponsored Research Office really open on the 28th?) Next deadline (for small grants) is December 17, and I plan to get a proposal in for that as well.

I try not to continuously send in NSF proposals, but it's seemed like that's the way it's been going recently. I'd prefer to metaphorically let the tank get close to empty as far as funds go, then apply and refill the tank, so I could spend less effort on the grant process. But there's ever-growing competition, and smaller amounts available. And given the long lag between calls, and the high risk factor each year, I feel the need to be more proactive than even a few years back. Especially with the economy tanking, so I'm not clear if corporate money will be available in the coming year.

Anyhow, since a proposal did go in yesterday, I can relax over the holiday weekend. I hope all of you will be as well.

Complexity Theory Humor

Ah, the joys of outsourcing at http://www.getacoder.com/projects/bug_finder_92913.html. Read down to the bottom to find responses by Godel and Cantor...

Update -- someone at the site noticed and took down the original page (although it's still in the Google cache here.)

Center for Computational Intractiblity

This popped into my mailbox, and I can't see any reason not to pass on the announcement here. Especially notice the bit at the end about postdocs (get your paperwork in NOW!!!).

------------

Dear Colleague:

I am writing to tell you about a new Center for Computational Intractability that is funded by the National Science Foundation and is a collaborative effort between Princeton University, Institute for Advanced Study, Rutgers University, and New York University. The PIs are Allender, Arora, Barak, Charikar, Chazelle, Impagliazzo, Khot, Naor, Saks, Szegedy,Wigderson, and Tarjan.

The Center's mission views "intractability" fairly broadly, and covers both algorithms and complexity theory as well as other topics in theoretical CS. I would like to draw your attention specifically to the following: (a) Visiting possibilities for short-to-medium term (up to 1 year). We can pay travel costs and salary. (b) Several workshops a year, with possibilities to visit the center at the same time. (We can pay expenses.) (c) Postdoctoral positions both at the center and at the affiliated institutions. (d) Streaming videos of all center activities and seminars and other web-based resources.

I invite you to visit our website http://intractability.princeton.edu/ and to especially click on "Opportunities" from the sidebar.

I was wondering if you could forward this email to others in your department (including students and postdocs) who may be interested in the center's activities. Please note that the deadline for postdoc applications is December 15, and the deadline for postdoc applications to IAS is December 1.

With best regards,

Sanjeev Arora
Director

STOC notes

It looks like we ended up with about 329 submissions. (Hopefully that won't change much at this point.) That's between 40-50 papers per PC member at 3 reviews per paper.

Thanks to everyone who withdrew the papers they weren't submitting. Otherwise, I had to go through and withdraw them manually myself.

Thanks to Shai Halevi for continuing to help me with the reviewing system.

Yes, I did get mail from about 10 people who hadn't known they'd need to file an abstract the week before, and I accommodated them. I think this should become a standard and everyone should get used to it. Again, keep in mind there's 40-50 papers per PC member; anything that makes their work easier is a good thing. (And Shai's interface for choosing preferences lets you see a paper's abstract pop up with a mouse rollover, so it's really nice to have abstracts early!)

Assuming things continue to go well, expect e-mail from PC members asking for you to handle a subreview before you head off for Thanksgiving....

Technical Depth vs. Novelty vs. Potential Impact

Let's say you're the PC chair for a major (theory) conference, about to give instructions to the committee. The standard way to judge a paper in theory is primarily based on its technical depth, but there's certainly a push in the community (and, arguably, from our funders the NSF) to consider other aspects, including novelty (could this start a new research direction, or give us a new way to look at things) and potential impact (might people actually use these ideas)? How, exactly, should you instruct the PC to weight these various factors?

Conceivably, we could set up the reviews to have a score for each factor. For example, I'm on the PC for NSDI, a systems conference, and we have to give scores for Overall Merit, Technical Merit, Longevity (= how important will this work be over time), Novelty, and Writing (as if that score matters :) ). Personally, I don't like this, and I'm not intending to do it for STOC. It's more pain for me as a reviewer without I think giving meaningful information to the authors (instead of spending time trying to decide if a paper is a 2 or 3 in terms of novelty, let's give another comment in the review text!), and when it comes time to make the decisions, I'm not really sure what I'm supposed to be (Pareto-)optimizing in this multidimensional space.

I'm a big believer, for conferences, in the "simple" method, as I've said before -- papers just get a score from 1-5, under the following scheme:

1: Bottom 1/2 of submissions.

2: Top 1/2 but not top 1/3 of submissions.

3: Top 1/3 but not top 1/5 of submissions.

4: Top 1/5 but not top 1/10 of submissions.

5: Top 1/10 of submissions.

but that doesn't mean that reviewers shouldn't be using factors such as Longevity and Novelty, and even Writing, in deciding their overall score. So, as you're all finishing your submissions, now is your chance to make a suggestion -- how do you think the PC should weight these various factors?

"I write today about the global economic crisis and its implications for us at Harvard."

In what I'm taking as a sign of continuing Impending Doom on the economic front, I received mail from the 3 levels in the hierarchy above me regarding the financial state of Harvard in these tough times. (First from Harvard President Drew Faust which is where the title quote comes from, then the Dean of the Faculty of Arts of Sciences, and then the Dean for the School of Engineering and Applied Sciences.)

Little was given in way of specifics, but the general theme was clear. About 1/2 of Harvard's budget comes from the endowment payout each year. (The curse of a large endowment -- this number is probably higher than most institutions.) While nobody is giving a number for Harvard, the widely quoted statement from Moody's financial research is that endowments will lose about 30% this year. Given that Harvard has been outperforming the market and most other endowments over an extended time period, you can take your guess as to whether our losses will be above or below average. No matter how you do the math, it's not good.

So that means there will be some belt-tightening, and some delays in various plans. Again, very little in the way of specifics, but I'm sure (and Faust's letter suggested) that Harvard's new progressive financial aid program would not be touched. I imagine most everyone is getting similar messages at other institutions, but feel free to share your stories in the comments.

STOC deadline, reminder again

Remind your friends (and, if you're feeling generous, your enemies) that indeed the short abstract for your future STOC submission is due today (full conference submission deadline - Nov 17). Thanks to Shai Halevi's help, the server is even synched with the deadlines on the conference Web page.

I've seen over 150 submissions so far, and it's not even close to midnight...

Reminder, Abstract deadline for STOC

Remember that November 10 is the deadline for the "short abstract" for STOC.

More information at the web site, or head to the submissions server.

IPAM Networks of Networks (Day 2)

Day 2 of IPAM also had a bunch of interesting talks. My favorite for the day was by Ramesh Johari, who was considering the following problem: suppose that you only allow bilateral transactions -- corresponding essentially to a barter system, or in his view roughly how BitTorrent currently works. (People upload to you if you download to them -- not necessarily the same amount, but there nominally should be some equivalence in terms of utility.) How much do you lose over a system where you allow multilateral transactions -- which corresponds to the system we all know and love, where there's money behind exchanges, so you can always get money and use it to buy something else later. He set up an interesting model to look at these questions and say something meaningful about the differences between the equilibrium states in these two settings.

David Alderson also gave a very interesting talk on modeling the Internet topology using optimization methods, and using it to study the scale of the damage an adversary could do to the Internet. His point of view is very different (and I think much more compelling in the Internet setting) than the scale-free analysis popularized by for example Barabasi. I recommend checking out his papers (most of them co-authored with Doyle and Willinger, among others) on the theme.

Finally, I ended the day nicely by getting to talk with UCLA faculty Rafail Ostrovsky and Amit Sahai. Rafail expressed his usual enthusiasm insisting that we find a problem to all start working on -- so now I even have homework to keep me busy on the plane ride home!

IPAM Networks of Networks (Day 1)

I spent Monday at an IPAM meeting on Networks. (I'll be here today, too.) I gave a very "high-level" talk, covering my taxonomy of networking papers (just replace "power-law" papers with the more general "networking papers" in this editorial of mine) and discussing my thoughts on how we get closer to validation/control in the Internet with a universal hashing architecture. Slides are here; it's sort of a mix of some other longer talks here, here, and here.

The highlight for me for Monday was listening to Tim Roughgarden (who always gives excellent, crisp, clear talks) on comparing FIFO vs. FairShare queueing policies using the worst-case price of anarchy over all possible utility functions (and, unfortunately, only quadratic cost functions). This is work in progress, so I don't think Tim will have slides up, but it's a very nice analysis. [Spoiler: FairShare wins!]

David Clark gave a public lectre in the afternoon on the Internet -- what's wrong with it, and his ideas for fixing it. It got a big audience, and he was certainly interesting, amusing, and provocative. I've got to admit I have a lot of skepticism so far about how we get to a better Internet, one with security and designed with the economics of the creature in mind from the start instead of developing by accident. Which reminds me, what's the state of the NSF GENI project? I've been hearing rumors that it's headed to an early rest, but I'd be curious if anyone more in the know can comment anonymously.

Bugs

In a recent review of a journal paper, I got the following comment:

In explaining why you are presenting simulation results, you say, "First we wish to check our theoretical analysis..." I don't understand this motivation. Your theoretical analysis is substantiated by mathematical proofs. What more evidence do you need of its validity?

Please keep this statement in mind.

I've stated frequently that theorists should actually implement their algorithms. I have some further recent anecdotal evidence to back up this claim.

I have students working on a variety of projects, and recently, I've had a strange convergence: in several of the projects, the students have found what appear to be non-trivial errors in recent theory papers (2 in CS theory, 1 in EE theory). It's not clear that any of the results actually break -- indeed, I don't think any of them do. I don't want to exaggerate. In one of them, a constant factor seems to be messed up -- not disastrous, but it is an important constant factor in context. And perhaps in one of the papers we could chalk it up to really just a sequence of confusing typos rather than an actual error.

Now I'm not naive enough to expect conference papers (or even journal papers) without errors. But the key here is that these errors were either easily found and/or proven to me by the students by having them implement the algorithms described in the paper. Then they sort of jumped out.

Implementing your own algorithm is a good way of checking your work. If you aren't implementing your algorithm, arguably you're skipping a key step in checking your results. Why should a reviewer go to the trouble of checking your work carefully if you're not?

Moreover, imagine not some student but an actual real systems-builder who decides your algorithm is worth implementing for their system. Imagine how they feel when they find things don't quite work as you've stated. That doesn't exactly inspire confidence, and is the sort of thing that discourages someone from using your algorithm. More globally, it gives systems people the feeling that theory (and theorists) aren't important. Why should they be debugging your proofs? You should give them evidence your algorithm can be implemented and works as claimed by actually implementing it if possible or reasonable to do so.

I'm not stating that you personally have to implement your algorithm. Hire a student to do it! It's good experience for them. You can apply for an REU, or other student research funding.

So, to answer the anonymous reviewer, I think there's always plenty of good reason to check our mathematical proofs by experiment and impelementation, and it's suitable to put those results in the journal paper as an aid to the reader (and reviewer!).

Computer Science AP Test

Harry Lewis went to a meeting about the future of the Computer Science AP Test. He wrote a short report for our faculty, and gave permission for me to put it on the blog. There are a few Harvard localisms in the report, but I think the information is important for everyone to see.

Guest post by Harry Lewis:

Over the weekend I attended a meeting in Chicago about the future of the AP Computer Science test, along with about 70 other faculty and a few high school teachers.

The College Board announced last year that it was dropping the CS AB test after the spring 2009 administration (the more advanced version, with more on data structures and algorithms) for business reasons (only 4000 test takers). The A test isn't that popular either -- 18,000 I think. By way of comparison, 180,000 take Math, 80,000 take Physics B, and 75,000 take the Statistics exam, if I wrote down the numbers accurately.

The CS AB test is the only one for which Harvard gives credit for advanced standing purposes (and then only if you get a 5). I don't think it would be wise for a student who has taken only the CS A course to skip CS50. So as far as I can see, Harvard won't be offering credit for AP CS after this year.

The numbers of students studying AP CS are pretty bad. Only 12% of American high schools even offer a CS AP course (even the A).

There were rumors that some effort going on to line up industrial support to keep the exam alive. (The Italian-American community got their act together when the College Board cancelled the AP Italian exam, and managed to keep it going by pledging to pay the College Board, even though even fewer take it than the CS AB test.)

The economics of this are interesting. Most of the costs of administering the exams are in grading the free response (non-multiple-choice) questions. But a psychometrician gave an interesting presentation -- it seems that the FR questions contribute nothing to the validity or reliability of the test scores that was not already there in the answers to the multiple choice questions. (Which is not to say that teacher and student behavior might not change if they knew the exam were nothing but multiple choice questions. And the psychometrician cautioned that he hadn't calculated the nontrivial and perhaps different costs of developing the MC and FR questions themselves.) On the other hand, the College Board is probably basing its cancellation judgment on trend lines. CS seems not to be growing in high schools, AP or not. (Whereas Chinese and Japanese, also currently small tests, are.) (The College Board made the decision to drop the AB test and has no plan to reconsider it. It announced its decision without any consultation with the faculty groups involved in developing the exam, by the way.)

A great many people at this meeting don't like the A test, and after taking and grading a few questions, I can see why. It's a Java programming exam. Graders are trained not to take off for trivial syntax errors (confusing commas and semicolons in written answers). They do an incredibly good job developing grading rubrics and keeping the grading consistent between graders. But there are more questions about OO programming concepts than anything else, which doesn't seem right (on the exam we looked at, there were 18 questions on inheritances and whatnot, and 2 on "logic"). Several people said, "if you wonder why kids are turned off on CS in high schools, just look at the questions on the AP exam to see what image of the field is projected." Some proposed, half-seriously, that we ask the College Board to rename the existing course and exam "AP Computer Programming" (no dice, it has to be the name of an academic department in colleges and universities).

There is a movement afoot not to tweak the A exam, but to develop an entirely different kind of intro course to be the basis of the CS AP, something broader, emphasizing both the principles and the importance of the field. A couple of prior working groups drafted proposals. They are right on target IMHO, though maybe better goals for what our majors would understand at the end of 4 years than at the end of the first course. We spent Sunday discussing these. (The College Board is emphatic that there will be only one exam in CS, so this would substitute for the A exam, and we are not discussing a substitute for the AB exam in addition to keeping the A exam.)

These 2 proposals have no operational content at this point. The discussions we had were not premised on there being any programming, or on what programming language might be used (though there certainly seems to be a consensus that the first programming course should use a light-syntax language). And the discussants yesterday were divided on that question. So we are still in early stage development.

We didn't even talk about testing. The two boundary conditions are that the course has to be testable (though not necessarily multiple-choice testable), and has to draw people into the field (the NSF was represented by Jan Cuny, who was evangelical on the subject of drawing more people into the field, especially groups now underrepresented in the field, and has curricular development money to put where her mouth is). Obviously the easiest things to test are programming and math, so this is a nontrivial feasibility issue.

A critical question was whether colleges would give the imagined new AP course credit. I found that hard to answer. With no programming, or very little, it certainly couldn't place people into CS 51 or 61. It seems to be in the CS 1-QR48 space (though maybe not, if there is significant programming after all). Harvard doesn't give Advanced Standing credit for courses that no department would count toward their major, and you can't place out of your Gen Ed requirements on the basis of high school work anyway. In the end I decided there were too many ill-defined variables for me to claim that Harvard would give credit for this course, though I think it has huge potential.

This would take a minimum of 5 years to mature into textbooks, teacher training protocols, etc. Probably more.

There is, by the way, some worry that the College Board's decision will (a) lead to a general dumbing-down of high school CS, marginal as it now is, and (b) deprive us of a small but important part of the flow of new majors (though the number of students taking the AB test nationally is tiny, 40% of them become CS majors, and I'll bet a significant number of our CS majors are in that number).

It will be interesting to see how this develops and there will be opportunities to be involved in the development.

Harry