One of the smarter students I've known at Harvard, who has been a
Teaching Assistant for me the last two years (and who I'm hoping will be
back again this year), recently made the following argument to me,
encouraging me to hand out written answer keys:
There are two approaches to giving quality feedback on student work:
writing good comments for each student, which is O(N) for N students, and releasing
answers to the pset problems, which is close to O(1). In classes that
release pset answers, TF comment-writing decreases to almost nothing. We
tried, last year, to implement an alternative O(1) solution, the
"answer review sections" that I ran for each pset. I think that students
who had been confused on the pset had trouble following these sections,
and I don't think that they were nearly as effective as written
solutions.
Here he's cleverly played a novel argument: I should be
providing answers not because it helps the students, but because it
helps the teaching assistants, who are woefully overworked. He knows I
agree on this last point. He also knows that I'm unsympathetic to the
argument that it helps students to be given written solutions.
(Parenthetically: Since this comes up often, I feel it necessary to be clear that it's not that I'm unsympathetic to students
who have trouble on my apparently very difficult homework assignments;
it's just that I don't feel that handing out written solutions is the
right response. First, by what I consider necessity, I re-use problems,
and experience shows that, given opportunity, even students who you
would not imagine would copy old solutions run into situations where
they believe they need to copy old solutions, which generally leads them to
getting kicked out for a year at Harvard. I don't like having to turn in students
and then seeing them get kicked out for a year. Second, even if you
feel that cheating-prevention is not a suitable reason not to hand out
assignment solutions, I'm not a big believer that written answer sheets help
students learn better than other methods, such as getting more
individual feedback and working out the problems afterwards from the
assignments with TAs or others. Or, if you want to see worked-out
examples, buy one of the recommended textbooks and read it as well. The
fact that so many students fail to take advantage of opportunities to
obtain and/or work through answers after the fact in the absence of
written answers being handed out remains troubling to me.
Maybe
I should just use MOOC techniques and have students turn in numerical
solutions electronically that can be script-corrected. And videotape my
lectures and just replay them each year so I can stop showing up.)
This
year, however, I'm entertaining his proposal... primarily because it's possibly
my last year teaching the undergrad algorithms class for some time.
(With any luck, I'm on sabbatical next year -- and I'm likely to be
passing the course on, after what will be the 15th year in a row
teaching it, to one of our newer faculty.) If someone else is teaching
it, the calculus changes from my perspective. (Though I suppose I
should check that the incoming faculty are OK with the idea.)
What's your take? It seems like an interesting way to change things up.
Subscribe to:
Post Comments (Atom)
11 comments:
Since you mention MOOCs, do you foresee there being an edX version of the class in the future? Because that would be awesome.
Don't use MOOC techniques. I think that written answers can be helpful, but perhaps you shouldn't give written solutions to all questions.
I think that writing up proofs well on homework assignments is excellent preparation for writing up papers as a researcher. (At least, this was the case for me. Unfortunately, most student papers I see are miles away from this standard. Students are usually happy with getting down an answer that a generous TA can interpret as being correct.)
I give out solutions, and rather detailed rubrics, for my algorithms homeworks.
Partly this is economy of scale. It's simply impossible to give detailed individualized feedback to 175 students for each of the 30-ish homework problems they attempt in a 15-week semester, even with the help of graduate, undergraduate, and administrative support.
Partly this is explicit signaling to the students. *THIS* is what we expect from your future homeworks, in terms of correctness, completeness, rigor, balance between intuition and formality, and (hopefully) clarity. And it's also signaling to the TAs, who often come in with very different expectations, sometimes _much_ higher, sometimes _much_ lower.
Partly, it's for my own benefit. I write out my own homework solutions for the same reason that I write out complete proofs before I trust a new research result. I'm pretty good at solving problems, but I make lots of stupid mistakes along the way. Better to be sure.
But most importantly, I think the solutions, revealed _after_ they've struggled with the problems themselves, actually _do_ help them learn the material better.
Do solutions help as much as working out the solutions for themselves? No, of course not. But realistically, most of the students don't work the problems out for themselves. Some of them _think_ they've worked out the problems, but don't recognize their own mistakes. Others think they've worked out the solution, but really only have an outline or a reasonable approach, with none of the necessary and subtle details. (Show them the details.) Others think they don't know how to solve the problem when they actually do; the problem is confidence, not ability. (Show them they were right all along.) Others struggle for a while and eventually give up; the more often they have to struggle before they give up, the easier it becomes for them to give up. Others try but just make progress to slowly and so can't figure it out by the deadline. Others don't even try, because who's going to use this stupid theory crap anyway? Erickson's a tool; screw him, the horse he rode in on, and the horse his horse rode in on.
Better, I think, to meet the students halfway.
In principle, I agree that "other methods, such as getting more individual feedback and working out the problems afterwards from the assignments with TAs or others" might be better still. But first, I really don't know how to do that at scale, and second, even if I could, why shouldn't I give the students both? Do you believe that providing solutions would lead to your students learning less?
I am also troubled by "The fact that so many students fail to take advantage of opportunities...", but that's not an argument against providing them more opportunities.
Do I reuse problems? Of course. Do I have to deal with cheaters? Of course. Is the small number of unpleasant cheating cases a reason to change how I teach? No.
I agree with Jeff. (Found myself nodding my head so often that I was afraid of it falling off.) I hand out solutions to HW problems in my algorithms course. My primary reason is that I do not even have close to enough TAs to implement Michael's suggestion of the students going to the TAs to learn about the solutions. (If we're lucky we get 1 TA for ~50 students.) Also in my experience, most students do not think about the HW once they are done with it: hopefully when they get back the solutions, they'll actually go over it and maybe some light bulbs will go off. It does mean a bit more work on preparing new problems but I have found that even superficially changing the problems seems to avoid most of the cheating possibilities (though of course this is not perfect).
I don't teach. Not in the traditional sense anyway. But, I was a grad student at one of the prestigious schools in India. Even after three decades, the subject of assignments and tests stirs up quite a lot of emotions in me.
Here is the thing I don't understand. Crack open any academic text book or attend any course. Assignments are more than likely to be demanding both in time and skill. In many cases, the problems required significant effort from those who worked on them and produced original solutions. Now, a student who has to pass exams with the highest possible grade he can get, has multiple courses each asking him to solve problems that may or may not resonate with her. While this is all good fun for the teacher (as a sadistic teacher at my school told us so), I continue to believe that there are better ways to teach and more importantly learn. Instead of making students do problems that they cant relate to, isn't there another way to teach them the fundamentals of algorithms through interesting problems? The hard and dry theory of algorithms maybe fine for training future sadists (smile) but not the right stuff for training engineers who take delight in solving real problems. I have a hard time believing that your students are not smart or hardworking. What really separates us is mostly motivation. I happen to believe that how well the student body does is a direct measure of how well a teacher was able to teach and motivate it (barring the misfits, of course). I think schools and teachers should take that a bit more seriously.
I wonder what you will end up with if you throw a bunch of practical and interesting problems at your students which will make them learn and master the tools needed to solve them.
Kurt --
Thanks for the kind statement. I would definitely entertain the idea of constructing an edX version of CS 124. I do have some concerns, however.
First, photogenically or vocally speaking, I'm no Tim Roughgarden.
Second, I think it would be a great deal of work to transfer the class to that format. Mostly I'm thinking about the homework assignments; I'm not clear on the effort involved in turning them into machine-gradable format.
Third, my version of algorithms/data structures is apparently harder than average. I'm not clear what the market size is for my version of the course.
But it's something I could easily see doing if backed by Harvard to do so.
Jeff --
I agree with many of your points,so let me focus on disagreements. I would differentiate between not giving the students solutions, and not giving them written solutions. I'm not against providing students solutions -- I do so in class (when asked) and have the TAs do so (in section and office hours). So I try to meet students halfway. However, this does take more effort on the students' part -- sometimes they have to ask to see a solution. Giving them written solutions, however, changes the dynamic.
Specifically, you ask: Do you believe that providing solutions would lead to your students learning less?
Well, yes, I absolutely do, for a large number of them. I believe that if I hand out written solutions and re-use those problems, a substantial fraction of the students will examine those written solutions in order to solve the homework problems instead of working it out themselves (ostensibly they're smart enough to not copy enough so that it doesn't look like cheating; or, worse, they're not, and then I have to deal with them cheating). Thus they're not working to solve the problems themselves, which is the reason I assigned the problems. If I wanted them to read and copy over solutions, or if that's what they want to do, there are plenty of problems worked out in the textbooks that I don't assign as homework or have time to cover in class.
Indeed, I would argue that FAILING TO SOLVE A HOMEWORK PROBLEM and not having an answer or getting a low score is a better learning experience for them than determining the problem is too hard, finding the written solution from a previous year, and copying it down.
You might argue that students won't do this if I tell them not to. Or, going the other direction, you might argue that students are doing this anyway, getting answers from students in previous years or from the Internet (even though I tell them not to), so what's the harm in providing my own solutions. My belief is that providing written solutions substantially changes the dynamic, greatly increasing the chance that the students find it OK to look at the answers instead of working out the problems themselves.
Again, this is not to deny your many excellent reasons in favor of handing out written solutions; but rather to point out my belief that there's a substantial other side.
Anon #5: I do attempt to "vary" the problems in my class, with both programming assignments as well as proof-type assignments, and (where possible) presenting real-world algorithms/applications in class an in exercises. So I hope I'm meeting some of your desiderata in my class.
I am strongly against posting online solutions (except in password protected domains) where others can easily stumble upon them with internet searches but I will always provide printed solutions. Students particularly find printed solutions useful when they study for tests and exams. (Their value for TAs is also significant, though that would not require that they be distributed.) In the quarter system and without a tutorial section in my algorithms course, there is simply not be enough time to take up solutions in class. I feel that if I ask students to solve problems, I owe it to them to convince them that the problems are solvable.
One place where I do not give out solutions is optional extra credit problems which I always include on my assignments. Because these are optional, I do not feel the same responsibility.
Speaking as a student myself, I essentially agree with Professor Mitzenmacher's stance on how much _students_ benefit from online solutions. I have self-studied several books in the past, and at first I was eager to find solutions to the exercises as a guide on my progress. But I fell into the trap of using the solutions manual whenever I got stuck, and I eventually realized that struggling with a problem for days and then finally solving it was not only a crucial process for problem-solving development, but also very satisfying in the end.
As I said, this was just my reading books in my free time. When your grade is on the line, and you have assignments from four other classes bearing down, the temptation is much higher.
However, you do have to appreciate the dichotomy of backgrounds in CS124. In my experience, the class consists of a lot of computer science concentrators who have never written a proof (and a handful of math concentrators who find proof-writing to be second nature). What about providing model answers to some example problems, so that the students who have never written proofs before know what to aim for?
Another strategy that's been used in some of my other classes at Harvard is to pass out hard copies only, as Paul Beame suggests.
Moving on to the issue of handing out solutions for the benefit of TFs: as someone who has TFed other classes here, I have a lot of sympathy. But I don't think that giving out solutions excuses one from giving feedback on individual student's work, at least at the level of "this is incorrect, because ..." And to be honest, when I took the class I rarely got more than a couple sentences' feedback on a problem set. I'm sure the 124 staff puts in quite a lot of time, but it seems to be general opinion that, after dividing this time by O(n), students feel like they're getting very little attention. Maybe you need more TFs.
The previous poster's comment about CS concentrators not being as comfortable with math proofs resonated with me. The trick for the CS person without a heavy math background is figuring out why some two sentence "proofs" are completely OK among math luminari, while similar attempts by the less math-informed generate a scoff, scold or null. Regarding reusing problems, it seems insufficient, unless they are of the same classic nature as well-known chess positions. I've always been impressed how MIT people manage to come up with new problems and publish their solutions. As for working out problems, the answer is time. Students who work out problems either had or made time. Students who don't may not have time for a variety of reasons. Taking the time to find out why people don't have or make time could be educational and remediative for professor and student alike. As for your not being photogenically or vocally suitable for an edX version, nonsense, you would be fine.
Post a Comment