One change I guess I'll have to make is to officially label one textbook Mandatory. I suggest both the standard Introduction to Algorithms
and Algorithm Design
as useful optional textbooks in the syllabus, and instead provide students with my lecture notes. But several complain they find the lecture notes insufficient, so I'm guessing they don't understand that optional does not actually mean "unnecessary for everyone", and the best solution I can come up with is to make a book mandatory. I can see other uses for this, such as assigning problems from the book as good review problems or problems to cover in section, so perhaps it's really a win for everyone. If anyone cares to offer any insight into which of these two are most useful (keeping in mind I'll still be using and making available my lecture notes) please advise in comments. (This subset of students also complains that the lecture notes haven't really changed since I first wrote them about ten years ago, which is true enough, although I don't believe that, for example, Prim's algorithm has changed substantially in that time either. The latest edition of Intro to Algs seems to be from 2009, while Algorithm Design now goes all the way back to 2005, so maybe Intro to Algs should win based on newness.)
A number of students also complain about the midterm. Not just that it's too hard -- which is fine for them to complain about, it is indeed quite hard, although I personally would label it "just right" -- but that it's the same day an assignment is due, just before spring break. Somehow, having the midterm after spring break, or before they've done an assignment on the material, has just always seemed like a bad idea to me, but it's clearly not working for them.
I think I've found a creative solution, which is simply to abolish the midterm. This has many advantages beyond dealing with the complaints above. For example, the past couple of years another faculty member has complained about my choice of midterm day, because they wanted to schedule their midterm that day, and didn't think it would be appropriate for the students taking both classes to have to take two midterms. So now I can yield this midterm day.
Further advantages come from my midterm replacement idea. I still want to have in-class assessment, so instead of a midterm, I'll break it up into shorter in-class quizzes. I'll get a more timely idea of what students aren't understanding; we can go over the quiz problems, giving the students more feedback (partially solving a different complaint, that because I don't hand out answers to problem sets, students don't get enough feedback); and now class attendance will be essentially mandatory. I'll let students drop their lowest one or two quizzes, so if they miss a class quiz, they can use a drop, but students will have to get to class more often. (Besides giving lecture notes, my lectures are taped, so currently some students often don't come to class and catch the video later.) The biggest downside is that I'll have to drop some material to make time for this approach. I have some fun lectures at the end about a Least Common Ancestor algorithm and suffix trees (maximal palindromes in linear time!) that I'm sure I'll have to drop at a minimum, but even I admit those are a bit esoteric for a first algorithms course. I can always make the notes available for the interested.
I imagine these changes will fail to please everyone. But I think they're worth a try. As I reread this, I recognize this might all sound sarcastic, and while I've sprinkled in some sarcasm in the discussion, these do seem like things I'll plan to do next year. There are also other changes I'll try to implement with the TAs -- a common complaint is that they can be too picky on details in grading (true enough) and that more math review is needed (certainly possible, best done by splitting up sections into super-advanced/standard/math-phobic?). Then we'll see if any of this effort pays off in next year's reviews.
25 comments:
Short Quizzes (like 3 or 4) is the best idea. I don't know why in US we don't have quizzes as a part of coursework. In an IIT an algorithms class would be
50% endsem exam
20% midterm exam
20% 4 quizzes
10% assignment
(The weightage for the assignments is less because we don't have TAs and honor code is not strictly enforced in the assignments).
But yes, the big picture is that quizzes are really useful!!!
I recommend "Algorithm Design" over "Introduction to Algorithms" (CLRS). The latter is a great book, but it is too big and heavy, partly from spending too much effort proving obvious "lemmas", confusing some students and misleading them about what algorithms really are about.
In my algorithms class we used the Kleinberg/Tardos textbook and I found it to be wonderful (as a student, I should mention). It's rare to find a textbook that, while informative, remains accessible and well-written. I also liked the problems in that book, I would say that homework based on this textbook was most instrumental in my own learning. As an added bonus, it is specifically requested that solutions are not posted for public to see which means students can't just go ahead and do a web search for a problem with solutions (mostly) :-)
It's important to keep in mind that a lot of undergraduate students (most, in fact) come from an environment where they were given very specific instructions on what to learn and where to learn it from, too many have this mindset that if something is not "required", then it is truly optional. As in, they can skip the textbook altogether since you, as an instructor, did not say it's useful enough to be "required". It's just what they are used to.
I use KT in the graduate class and CLRS in the undergraduate class, partly for compatibility reasons with other instructors, partly because some of the very basics (sorting, etc.) are not covered in KT. The last time I taught the undergrad class, well over 50% of the students had as their number 1 complaint that they strongly disliked CLRS, and couldn't we use KT instead? In my grad class, most students really like KT, though a few claim that they don't like all that "intuition" stuff, and would prefer just seeing proofs. (No, I'm not making this up.)
Scheduling a midterm, by the way, has been shown to be an optimization problem with an empty feasible set.
I remember reading an article that one should not read too much into these reviews. The point is that students mark a course higher in the review when they feel better about it, e.g. they enjoy it more. Fixing small things like these complaints is good, but I would not expect a dramatic change. I have less experience in teaching that you do, but I think that making the course more fun is what really matters here. The students should feel that taking the course and attending the lectures is not a waste of their time, just another course that they have to take. This is even more true about the assignments.
I approve of the quizzes too, as more regular feedback (for the teacher as well as for the ones being taught) helps with the teaching and the learning.
Yet, the midterm usually tests a different level of assimilation of the material of the course, and I don't think that the quizzes replace adequately the midterm. In a simplified way, I expect three levels of learning from my students: (1) do they remember, (2) do they understand, (3) can they apply. The quizzes typically test the first one, the assignments the second one, the midterm and exam the second and last one. But of course, one must do with whatever time is available, I often leave the quizzes as mere oral questions in class.
I am curious about the use of quizzes in written form: is cheating a problem? During my postdoc, convinced that all students would cheat during the weekly 10 mns of written quiz, I spent an extravagant amount of time developing a generator of random multi choice quizzes which would be easy to mark (the positions of the corrected answers were encrypted in a random string at the bottom of the page). Was this superfluous or would it serve if such a service was made available on the cloud?
I will vote in favor of Algorithm Design.
Algorithm Design definitely is a better fit with my course in both content and philosophy so it's my natural choice -- I was joking about the whole "from 2005" thing -- but it is nice to hear that others have had good experiences with it.
I'm a bit surprised at the clear positive responses to quizzes. My take is that a reasonable fraction of the students will hate quizzes, particularly if they're not announced in advance or given roughly every class, in no small part because then they are forced to attend class. (Especially with taped lectures, lots of students don't come to class.) I think of it as a dramatic change, but it will be fun to try.
I have also had a very positive experience as a student with K & T.
Maybe you want to consider announcing when the quizzes will occur. I personally learn best by reading, and have therefore have considered the lectures of very few classes to be worth my time, no matter how good the professor (the exception is when the reading -- either notes or textbook -- fails to convey intuition that the professor does a good job describing in lecture).
My point, which others may disagree with, is that it should be totally acceptable for a student to skip lecture as long as the student still absorbs the material on his/her own time in a way they find preferable -- and finding ways to limit the flexibility of students to skip lecture if they find it unnecessary/not the best way for them to learn will just lower their enjoyment of the class and material.
Even if you announce the quizzes, students will still have to show up to lecture when they are given, so you still gain a bit of increased attendance :)
Not to put words in anyone's mouth, but others in Harvard CS might agree with my point. Matt Welsh said in his last blog post "Dan Huttenlocher took a chance on this long-haired crazy kid who skipped his class a lot." Seems to suggest skipping class is ok :)
I'll balance the feedback by expressing displeasure with mandatory attendance. I hated sitting bored for an hour going over information I already understood. I had much better experiences with classes where I could sit in on interesting/new/hard lectures and skip the ones where I grokked a topic faster than the rest of the class.
Some classes just kept giving new information and expected people to attend office hours/discussions if they didn't immediately understand. Those were fine, I'd attend every day. But having to constantly wait for the median of the class to catch up really drained my enthusiasm for material.
My take is that a reasonable fraction of the students will hate quizzes, particularly if they're not announced in advance or given roughly every class
I think that students, almost regardless of their 'caliber', will seek the bottom as best they can. If you are unpredictable about your assessments of them, they will have to be vigilant as well as diligent about their learning. It will severely complicate their calculation of how to minimally 'get by'.
I believe it to be a bad thing when a final exam is filled with styles of questions that are completely different from what students have been asked to do the rest of the term. An example might be an algorithms class that asked "design" or "analyze" questions all term, but then asked for proofs on the final exam. (I'm not saying you do this, but I have seen variations happen many times.)
One useful function of a midterm then, is to serve as a preview of the styles of questions on the final exam.
Quizzes certainly can serve this function just as well, as long as they are deliberately constructed that way, but they often aren't.
Here's a suggestion that's too late for this year, but which you can implement next year.
Every year I hand out my own course review form. I ask the students to tell me what they think about specific ways in which I'm considering changing my course. I also ask them for suggestions on how to improve my course. I find their suggestions to be very helpful.
K&T is a great book.
Yet I favor CLRS.
But you knew that.
Seriously, one comment we get about CLRS is that students are glad to have it after they've taken the course, and after they've graduated. Once they're out doing real stuff, they often find themselves referring to it, and sometimes looking up material that's in the book but was not covered in their algorithms course.
One other point in favor of CLRS: it may be physically larger than K&T, but its list price is lower.
If only a minority of students find the lecture notes insufficient then why not just stress to the class that in this case they should consult such and such textbook and give pointers to the specific relevant material. If you make it mandatory then everyone will think they must buy it and end up spending a lot of money.
Tom -- funnily enough, when I suggest both books in my syllabus, I do say that CLRS might be more useful if you go on to take further algorithms courses, and mention its beneficial "encyclopedic" qualities. I'd rather let the students choose whether and which book is best for them... but again, that doesn't seem to be working.
Dan -- a good idea. I do think, though, that our current forms provide students a good way to suggest changes. I also talk to students (often at lunch or dinner) during the semester, and get feedback directly on ideas for changes. To me, the issue isn't one of getting feedback -- it's that, at the level of specific recommended changes, I often disagree. (For example, many students would like to have "late days" they can use to turn in assignments late. I don't want to deal with the administrative hassle of it. Strangely, when I've left it up to the TAs, they generally don't want to either -- even if they've thought it would be a good idea as a student.)
Adam O -- Your reasoning is precisely, in the past, while I've made textbooks optional, figuring students who needed the book would get one, and students who didn't could be spared the trouble and expense. I'd prefer to leave this up to the students, but my takeaway is that the students themselves feel they need to be told to get a textbook. I don't think I need to give page numbers -- the students can use an index and table of contents, I hope. My point was there's apparently, now, a non-trivial minority of students who feel it important to say to me the notes are not enough and that it's wrong to make the textbook optional on their class feedback. As I already strongly suggest students might need a textbook, I'm taking this to mean I need to make a book mandatory.
I'd rather let the students choose whether and which book is best for them... but again, that doesn't seem to be working.
Wouldn't a student have to read a substantial amount of each book in order to choose one over the other? They assume that you have already done that, so from their point of view, why shouldn't you recommend one book over the other? (As long as you recommend CLRS, of course.)
Tom : I don't give a recommendation of one book over the other because I don't think it's a one-size-fits-all sort of question. In particular, since the book (in the past) has been optional, and BOTH are suitable to help with course material, I think they're better suited to pick the book for them. Here's exactly the quote from my syllabus (which will obviously be updated somehow next year):
You may find it useful to obtain one of the following:
* Introduction to Algorithms, by Cormen, Leiserson, Rivest, and Stein. This book is probably worth buying if you are going to study algorithms beyond this course. It is primarily a theoretical text, and it is quite encyclopedic in nature. If you are looking for help with the proofs and mathematics, this is a good book to purchase.
* Algorithm Design, by Kleinberg and Tardos. This is also an excellent book, with a different style. It follows the course quite closely, but it is not as encyclopedic as the other book, and in particular assumes a lot more background.
* Algorithm Design, by Kleinberg and Tardos. This is also an excellent book, with a different style. It follows the course quite closely, but it is not as encyclopedic as the other book, and in particular assumes a lot more background.
From the perspective of a soon to be grad student in theory...
I think you could do a better job promoting the Kleinberg book. I prefer it to CLRS when learning a new area of algorithms because it tries to impart algorithmic intuition to the readers. In particular, I like the discussions of trying to develop algorithms and why some of them fail.
The mathematical approach of CLRS is important for students but if I had to choose one, I would rather develop the intuition and then fill in the messy mathematical details later. Having the math without with the intuition is a little confusing and also doesn't inspire people nearly as much.
My course used CLRS and I wished we had used KT.
To me the textbook issue strikes at the heart of what the goal of an algorithms course is, no matter what the level. Is the goal to introduce students to a collection of algorithms for problems that they might encounter (classified either by kind of problem or by kind of solution technique) or is it to teach them how to go about the process of solving algorithmic problems? The majority of algorithms texts over the years have focused on the former, encyclopedic aspects. These encyclopedic aspects are a necessary part of the latter since there is a matter of triage based on known problems. However, I find teaching only a collection of problems and algorithms to be particularly unsatisfying.
Very few textbooks over the years have focused on the algorithm creation process. K&T is the best text with this focus which is why I have strongly preferred it since I used it in notes form. The only two others I know of are Manber's "Introduction to Algorithms: A creative approach" and Skiena's "Algorithm Design Manual", neither of which is suitable for a graduate course. Manber's book is better than K&T or CLRS on divide and conquer for algebraic problems like FFT and Strassen's algorithm (which K&T doesn't) but it has very little advanced material. It is a bit idiosyncratic (no Kruskal's) and emphasizes an overarching theme of "design by induction" that may be too general. (It could have used a second edition but this never materialized because of a dispute with the publisher.) Skiena's book is very elementary and a lot of the book is a problem catalog but I used it a couple of times for a non-majors algorithms course.
Here is the article:
http://logicmatters.blogspot.com/2009/09/student-evaluations.html
and if you permit me, I would like to mention that I like Paul Beame's comments.
I admire your self-assessment and honest look at your teaching practices.
Dropping the midterm entirely seems like a weird overreaction to the student comment. Why not just schedule the exams and homeworks so there are a few days after the homework is due for students to study before the exam? It seems like an entirely reasonable request, and a good way to respect the students' busy schedules.
I'm not wild about in-class quizzes as a way of assessing how well students are understanding; they take up time. Here's an alternative that I've used, effectively: I give a short reading quiz once a week. The quiz is very short: designed so it should take only 5-10 minutes if they've done the reading (e.g., 1-2 questions). Then I tack on a question that says "What's the #1 thing you're most confused about or would like to see changed?" and I get lots of thoughtful feedback every week. I make the quiz due half a day before the lecture, so that I can adjust lecture based upon how well students are understanding.
On textbooks: I much prefer Dasgupta-Papadimitriou-Vazirani: I find it simply superior. The students seem to like it too. The writing is very clear, straightforward, and down-to-earth. Much better than CLRS or "Algorithm Design". (Yes, I learned out of CLR when I took algorithms myself, so I have a fondness for it, but Dasgupta is just better. For instance, just try comparing the exposition on DFS in Dasgupta vs in CLRS.)
Another idea is to hand out an anonymous feedback form halfway through the course that asks a few questions about how the course is going and what you can do to improve it. Ask students to fill it out and return it anonymously at the end of class (without writing their name on it). This way you'll get early feedback while there is still time to address the student concerns.
Anon 22: It's also a good idea to think if there's a natural way to re-schedule to have the midterm. But there isn't -- unless I have the midterm AFTER SPRING BREAK, which as I said, seems like a bad idea.
I'm not understanding how you do the reading quizzes. Do you do them on-line, since you say you do them 1/2 day before lecture? Do you actually time them?
My notes are already pretty similar to DPV. I want a textbook that fills in the holes -- in that sense, CLRS is actually quite a good book. I'm not clear that I need a book that provides intuition -- I find that where students have problems is in turning intuition into actual arguments.
Michael, I don't get what prevents you from keeping the midterm. I see a number of options, any of which seems reasonable to me. (1) Hold the midterm after spring break (at least 2-3 days after spring break). What's the big deal? Yes, I know you said it seemed like a bad idea to you, but you didn't say why. (2) Make the homework assignment due 2-3 days before the midterm. (3) Hold the midterm earlier, and change what's covered on the midterm so it doesn't include material that they haven't done an assignment on yet. It seems to me that any of these could work.
I don't see why any of these are hard to do. What are the constraints? I'm assuming you have regular homeworks; you control the content and timing of the homeworks; you control the content and timing of the midterm. It's just a simple co-scheduling problem: you make sure there are at least 2-3 days between the homework due date and the midterm date (so students can spend those 2-3 days studying), and you make sure that if it's after spring break, the midterm falls at least 2-3 days after the end of spring break (so students can spend the 2-3 days before the midterm studying). This all seems straightforward and easy to implement to me.
Quizzes: Yes, I have students do them online. For each quiz there is a simple web page with the questions and a few textboxes. They fill in the textbox with their answer and then click "submit". No, quizzes are not timed; students can take them at any point of their convenience, as long as it is before the due date. Quizzes are designed to take only 5-10 minutes if students have done the reading, but I don't time them.
Anon -- Certainly I COULD keep the midterm. But again, I think your suggestions aren't goo ideas.
Holding the midterm after spring break just seems bad form. They've just spent a week forgetting everything, and need time to get back into the swing of things. (I'd hate a prof who gave the midterm the week after break.)
I could make the assignment due earlier, but then they wouldn't really have time to do it. I could, naturally, make everything due earlier, but I don't think I'm allowed to start the class two days earlier. What you're really suggesting is to change the material on the sets in order to fit a schedule rather than my idea of the didactic importance of the material. Again, not something I want to do.
For your third suggestion, certainly that's possible, but doesn't seem best (to me) didactically. The midterm covers up to dynamic programming, which is a good break point.
So I'm not saying I couldn't follow your suggestions. And they are, as you say, straightforward, if I ignore what I think they might do to the learning aspect of the course. But in that case, I can just give everyone an A and make assignments optional. I imagine my reviews would readily go up with much less work on my part (and the students'!) with that plan.
Post a Comment