Grading rather than improving

I talked too much today. I had back-to-back 75-minute class meetings, first of CS222 Advanced Programming and then of CS315 Game Programming. Both times, I spoke almost the whole time. I would much rather have had structured exercises to help teach what I wanted to show. It wouldn't have been that hard to set them up, just an hour or two each of setting up a template project that demonstrates what I want to show. I don't have an hour or two for each meeting for each class. I have filled my allocated class time with grading. This is partially due to the new grading system I am using. I'm having a lot of back-and-forth with my students. Turns out that getting them to mastery is a lot harder than giving them partial credit. I believe it's bearing fruit. But it's also taking all or more of the time I can give to a class. 

I am not sure what the path forward is. I will do less grading later as both classes move from individual lessons to large project integrations. Then, however, it's too late: we will have passed the point in the semester where a strong introduction is better than 75 minutes of my talking.

CS222 and CC17

It has been many years since I have required my CS222 Advanced Programming students to read chapter 17 of Robert Martin's Clean Code. This chapter is entitled "Smells and Heuristics," and it contains a wonderful collection of common code problems and potential solutions. This year, I had my students read the chapter just before starting our two-week project, and I gave them the challenge to pick three items from the reading that were particularly interesting to them. These were fun for me to read, displayed thoughtful reflection on programming, and to top it all off, were easy to grade.

Some of my favorites showed up in the students' responses, such as the advice to extract conditionals into named functions, to replace magic numbers with named constants, and to avoid selector arguments. Feature envy showed up more than once, which surprised me. Students recognized that some of their previous courses actually habituated them to these smells rather than their cleaner alternatives.

I need to remember to keep this assignment. I plan to ask my students today whether they think this chapter would have made a good introduction to our reading rather than a capstone on it. Because the chapter is so accessible, it's possible that reading it first might help them get better faster, and to do so before they get into the trickier distinctions such as SRP (Chapter 10) and the distinction between objects and data structures (Chapter 6).

Docs is code

Clint Hocking's birthday blog post led me to look at the EXP tabletop roleplaying game, and in turn, that got me looking at AsciiDoc and the Docs as Code movement. I understand completely the arguments that AsciiDoc makes against Markdown. Regular readers will recall that I experimented with converting my course plans to GitHub-hosted Markdown and almost immediately backed away from it: Markdown almost immediately requires a polyglot approach for anything significant. However, I don't see AsciiDoc nor Docs as Code as addressing what I consider the most important tool for technical writing: the ability to embed scripts.

I have been using lit-html for years (and Polymer before that). What it lets me do is separate the structure of my writing from its display. For example, when I write an assignment for my students, I might conceive of it as having a list of objectives. In Markdown, AsciiDoc, or even HTML, I could easily represent that information as an ordered or unordered list. Later, however, I might decide to change the representation, instead showing it as a definition list, or making sure the name of the objective is bold, or generating unique links to each individual objective. In any of those plain markup environments, I have to do this by hand or, worse, with a regular expression.

What I don't see from Docs as Code, although I admit I haven't done more than a cursory search through their materials, is the observation that docs is code. If I separate my model and my view, I gain a robustness that any journeyman programmer understands. For example, using lit-html, I can create a simple JavaScript data structure that represents a goal, with a name and a description. Either or both of these can be html templates, not just strings. With that structure defined, I can create a list of them for an assignment. Now, on my first pass, I show them as list by iterating through the list and dropping the data into list items in an ordered list. When my requirements change—as they always do—I can modify my script and make the same data into a definition list, section headings, etc. If I need to change the actual definition of an assignment goal, I can make that change explicit.

Of course, the whole thing is in version control with sensible commit messages.

I have taken a similar approach in the past to build documents using LaTeX, coordinating the execution of multiple scripts through GNU Make. That works when LaTeX is needed for document output, but it feels less elegant to me than being able to generate the HTML directly from the Javascript.

If you know of an approach in the AsciiDocs or Markdown vein that gives the same level of robustness as what I can do with lit-html, please let me know.

A Morning with Scum & Villainy

After writing about my first experience with Blades on the Dark, I heard from a friend who recommended that I also look into Scum & Villainy. It is a sci-fi interpretation of the Blades in the Dark rules following the Forged in the Dark license. I use "sci-fi" intentionally since the rules and setting lend themselves to space westerns or space operas—anything with scoundrels on spaceships—but it would be difficult to do science fiction with them. The rulebook makes it clear that it's drawing on the "rag-tag group of outlaws traveling across the sector" trope as seen in Firefly and Cowboy Bebop. Both are clearly space westerns.

This theme is a good thing, and those two shows are among my favorites. It is a shame, then, that one of the first things one notices on opening the book is that it doesn't mesh with these themes. Blades in the Dark sings out its theme in graphic design and illustration. In contrast, Scum & Villainy feels like it cannot decide what it wants to be. This is exacerbated by the initial impression given when the structure of the book and much of the copy itself are taken verbatim from Blades in the Dark. None of this is inherently bad, but it gave a negative first impression after having been given such a strong recommendation to read it.

The few mechanisms added to Blades are quite good. Developing your ship rather than your headquarters captures the theme well, also lending the feeling of an episodic series. Getting bonus dice from gambits is a welcome addition to my group, since they have a penchant for beating the odds by rolling consistently low.

My favorite addition consists of the three starting scenarios, one for each of the ships. Playing Blades in the Dark, or even just reading and imagining it, it wasn't quite clear where to start. Scum & Villainy gives more tightly scripted introductory scenarios. At least two of them boil down to simple chase sequences, but there is nothing wrong with that. Each of these scenarios has just enough background to fill in details as needed, and each one provides clear hooks into the next episode. On top of that, there three outlines for other, unrelated jobs that are fit for the theme of the selected ship. 

We got the game to the table yesterday morning, and I played with my three eldest sons. My third son had previously expressed disinterest in tabletop roleplaying games, having not enjoyed whatever fantasy game we had tried together once years ago. I convinced him to try this one, knowing that he's a storyteller at heart, and he and his two older brothers had a great time. They chose to be bounty hunters on a Cerberus ship, playing a Scoundrel, a Pilot, and a Mechanic. We played the recommended started mission: tracking down a member of the Ashen Knives gang with multiple bounties on his head. There was a little hiccup due to an ambiguity in the scenario description, but once we got into that, we had a blast. The two older boys had a handle on the action resolution protocol as well as the role of flashbacks. They used flashbacks much more successfully as part of the storytelling than in our two Blades games, using them to set up a two-pronged assault on the mark's location, and then using one to set up and soup up hoverbikes for big chase scene. A glorious failure by the Mechanic led to a potentially disastrous desperate situation for the Pilot, but he used a gambit and pushed himself to ace it. It was exactly the kind of thing one wants out of a chase scene.

There are a few places where Scum & Villainy falls short of its august predecessor, doomed perhaps by its own lineage. For example, in Blades in the Dark, there are sensible limits on how much coin (or value in coin) a person can carry or that one can stash. This is actually quite interesting, a point that I don't remember seeing before: you can only carry so much money, and you can only have so much liquid cash, particularly in a Victorian setting. Scum & Villainy borrows this mechanism despite it describing money as being kept as software on credsticks. Also, while both games admit that any given action may be sensible under multiple action ratings, the action articulation in Scum & Villainy feels more forced than Blades'. I suppose, due to my career, I am particularly puzzled by how the Doctor action rating is for "doing science" and the Study action rating is for "doing research." It makes me wonder about how I would take the Forged in the Dark idea and put it into a setting of my own choosing, as many others have done. For example, making Kapow! years ago was a great exercise in understanding how Powered by the Apocalypse ideas could apply to campy 1960's superhero action.

I find the setting of Blades in the Dark to be more intriguing, but the setting of Scum & Villainy appeals to me personally while also being a "safer" space to explore with my boys. One could do a sci-fi criminal gang drama, but maintaining a ship vs. expanding gang turf really pushes toward the Firefly vibe. Scum & Villainy certainly stands alone, although there are parts whose rationale would make more sense if one is familiar with Blades in the Dark. Reading the Blades book was enough to see why there has been so much excitement about it,  even though I know I'm late to the party. The Scum & Villainy book may have lacked some of this pizzazz, but the table doesn't lie, and we had a great time. 

Initial reflection on Bowman-style grading

I had my first batch of submitted student work last week, and I would like to share some reflections on exploring a new grading system. As I mentioned over the summer [1,2], I have revised two of my courses to use a new grading scheme. CS222 Advanced Programming and CS315 Game Programming are both using a technique that I have lifted from Joshua Bowman's work. This technique looks at each goal and assesses a student's contribution into one of four categories:

• Successful
• Minor revisions needed
• New attempt required
• Incomplete

The first and the last are clear, but I found myself tripping up between the middle two. I think this is in large part to an important distinction between this technique and Rapaport-style triage grading, which I have used for years. In that model, you have four categories as well:

• Done and correct
• Done and partially correct
• Done and clearly incorrect
• Not done

The distinction between "partially correct" and "clearly incorrect" is very clear to me, and these are the second and third categories for Rapaport. I started using that as a heuristic to differentiate between "Minor revision needed" and "New attempt required," but I don't think that's right. With Rapaport's approach, "partial correct" captures a huge category of errors that one would put into the "C" letter grade bin: such a submission has some elements of correctness but significant flaws. I think Bowman's "Minor revisions needed" is much closer to Rapaport's "Done and correct." Clearing up the differences between these two rubrics caused me to have to re-grade many submissions.

Bowman's philosophy, which I am also bringing to bear in my classes, is grounded in mastery learning. Hence, recognizing the affordance for resubmission is fundamental to understanding the system. I knew I wanted to throttle my students' resubmissions, so I set up a two-tier system. With minor revisions needed, students could make the necessary tweaks within three business days, then get full credit for their submission. With new attempt required, or if they didn't make minor revisions within three business days, they could resubmit at most one per week.

I switched to Bowman's model in an attempt to clarify evaluation, and I'm already confused. I think this kind of system could work brilliantly if there were any tool support for it, but every gram of this technique fights against Canvas. Not only does Canvas lack robustness to anything but the least interesting of point-based pedagogic models, it and its LMS ilk breed an intellectual laziness among the students. The student usage pattern is to look at how many points were earned and then ignore any formative evaluation. My conclusion so far is that doing this on paper would be a great improvement over using Canvas if it weren't for the fact that my students' submissions are often inherently digital and not just accidentally digital.

It is early in the semester, but I have yet to see that Bowman's approach is going to be any more clear that Rapaport's. I've been using Rapaport-with-resubmissions, and that fills the middle ground between a clear representation of points and clear feedback about which parts are wrong. I will have to give it another two or three weeks to see how students respond before I make any systemic changes: there hasn't been ample time to get complete submit-evaluate-resubmit-evaluate loops from enough students yet.

Last year, I experimented with EMRF grading and ended up quickly dropping it. Canvas had no clear way to express this system either, and I did not see any clear benefit from distinguishing between "excellent (E)" work and "meets requirements (M)". It's easy to blame the tool for its shortcomings, and in this case, that's exactly the right thing to do. I know folks who "make it work" with tricks and hackery, but in my mind, there is no excuse for having a system that demands that the only real part of a class is something that has points and contributes to a pool of points. It's not how learning works, and it's never been how teaching should work.