Initial reflection on Bowman-style grading

I had my first batch of submitted student work last week, and I would like to share some reflections on exploring a new grading system. As I mentioned over the summer [1,2], I have revised two of my courses to use a new grading scheme. CS222 Advanced Programming and CS315 Game Programming are both using a technique that I have lifted from Joshua Bowman's work. This technique looks at each goal and assesses a student's contribution into one of four categories:

• Successful
• Minor revisions needed
• New attempt required
• Incomplete

The first and the last are clear, but I found myself tripping up between the middle two. I think this is in large part to an important distinction between this technique and Rapaport-style triage grading, which I have used for years. In that model, you have four categories as well:

• Done and correct
• Done and partially correct
• Done and clearly incorrect
• Not done

The distinction between "partially correct" and "clearly incorrect" is very clear to me, and these are the second and third categories for Rapaport. I started using that as a heuristic to differentiate between "Minor revision needed" and "New attempt required," but I don't think that's right. With Rapaport's approach, "partial correct" captures a huge category of errors that one would put into the "C" letter grade bin: such a submission has some elements of correctness but significant flaws. I think Bowman's "Minor revisions needed" is much closer to Rapaport's "Done and correct." Clearing up the differences between these two rubrics caused me to have to re-grade many submissions.

Bowman's philosophy, which I am also bringing to bear in my classes, is grounded in mastery learning. Hence, recognizing the affordance for resubmission is fundamental to understanding the system. I knew I wanted to throttle my students' resubmissions, so I set up a two-tier system. With minor revisions needed, students could make the necessary tweaks within three business days, then get full credit for their submission. With new attempt required, or if they didn't make minor revisions within three business days, they could resubmit at most one per week.

I switched to Bowman's model in an attempt to clarify evaluation, and I'm already confused. I think this kind of system could work brilliantly if there were any tool support for it, but every gram of this technique fights against Canvas. Not only does Canvas lack robustness to anything but the least interesting of point-based pedagogic models, it and its LMS ilk breed an intellectual laziness among the students. The student usage pattern is to look at how many points were earned and then ignore any formative evaluation. My conclusion so far is that doing this on paper would be a great improvement over using Canvas if it weren't for the fact that my students' submissions are often inherently digital and not just accidentally digital.

It is early in the semester, but I have yet to see that Bowman's approach is going to be any more clear that Rapaport's. I've been using Rapaport-with-resubmissions, and that fills the middle ground between a clear representation of points and clear feedback about which parts are wrong. I will have to give it another two or three weeks to see how students respond before I make any systemic changes: there hasn't been ample time to get complete submit-evaluate-resubmit-evaluate loops from enough students yet.

Last year, I experimented with EMRF grading and ended up quickly dropping it. Canvas had no clear way to express this system either, and I did not see any clear benefit from distinguishing between "excellent (E)" work and "meets requirements (M)". It's easy to blame the tool for its shortcomings, and in this case, that's exactly the right thing to do. I know folks who "make it work" with tricks and hackery, but in my mind, there is no excuse for having a system that demands that the only real part of a class is something that has points and contributes to a pool of points. It's not how learning works, and it's never been how teaching should work.