La Fin du CS222: An Assessment in Three Acts

In December 2010, I wrote about an experimental approach for assessing a course's effectiveness, using convergent and divergent exercises and mind maps as a replacement for traditional written exams. Since then I have used this approach a few times, with only minor alterations.

For last Fall's CS222: Advanced Programming final, I was inspired by a recent purchase to present the assessment to my students as La Fin du CS222: An Assessment in Three Acts.

Act I, Scene I
Each student constructed a mind map on the topic of programming. They had done the same exercise on the very first day of the semester.

Act I, Scene II
In this scene, I distributed to the students their original mind maps back to them. The main source of conflict in this scene was that it was the final assessment of how well I had learned their names.

Act I, Scene III
Each student wrote an essay comparing and contrasting the two mind maps.

The essays were generally well-written and insightful, especially those where there was not much change. One student commented, "Now I know that this random assignment we did at the beginning of the year was not so much for you but for us." This is great evidence of metacognition, that the student has learned to think about the value of reflection to himself rather than to some arbitrary authority.


Act II, Scene I
I hung my giant, flip-chart-sized sticky notes and told the students that we were going to list, in twenty minutes, all of the things we learned this semester that were somehow connected to the course. I briefly explained that this was a divergent thinking exercise, meaning that anything could be listed and there would be no debate or criticism at this stage. We ended up with 119 items, which may be a new record.

Act II, Scene II
I asked the students whether there were any items on the list that should be consolidated. Many suggestions were made, though not all were chosen by the majority. After this process, we were left with 113 items.

Act II, Scene III
Each student was given four stickers, and they were asked to place these by the four items that they found most significant. It was made explicitly clear that significance was subjective.

Act II, Scene IV
We identified at the top elements by vote, which are given in the table below. I have provided clarifying links for the curious.

Item	Votes
Object-oriented design	19
Coding conventions	14
Agile Manifesto	14
Effective Java	11
Test-Driven Development	7
Stack Overflow	7
User Stories	6
Mercurial	6

Act II, Scene V
The students were challenged to write an essay about what they actually did to learn these items that the community had chosen as the most significant.

I believe that it is important to help students understand the connection between what they personally do and what they have learned. Last year, the first time I did an exercise like this, we did another round of divergence and convergence to determine what actions helped learning. I was not very happy with the results because they became too generic: I was hoping for more specifics than "the project," for example. Unfortunately, most of the essays were written in generalities rather than specifics. In part, I think this is human nature: if we write in specifics, we make statements for which we are accountable and that may be shown to be false. Writing in established generalities, such as "I learned a lot by working on a team," is certainly easier than identifying specific times, places, and situations with the team that led to learning about a particular topic. I realize now that, if I want the students to write about specifics, I need to provide more explicit guidance to this end. I also need to narrow the focus: taking the top ~10% of items is too many: even though students were told they could pick from among the eight listed above, many wrote vague statements about all eight or a majority of them, as if they would get points for coverage rather than quality.

Some of the essays did contain interesting insights. My favorite comment was on the hassles of merging in Mercurial. We had previously discussed in class how tool-supported merging could be frustrating, but that it was a necessary evil—and actually not evil at all compared to the manual, ad hoc alteratives. This student identified how the frustrations of merging made him focus on learning encapsulation, because he realized that the better encapsulated and more modular his team's system was, the fewer times they would have to merge. Excellent!


Act III, Scene I
Cinnamon Rolls, courtesy of my lovely wife.