Reflecting on CS445/545 Human-Computer Interaction, Fall 2020

This is the third in my sequence of end-of-semester reflections [1,2], and today's topic is my Human-Computer Interaction class. This was the course that I ended up designing twice this past summer. The end result was a great experience. Out of my three different courses in Fall 2020, I think this is the one where I was most consistently impressed with students' dedication to tackling the ideas of the course. And how many ideas there were! With the removal of face-to-face discussions, and knowing the limitations of discussion boards, I presented a wider variety of ideas to the students this semester than in the past. I will proceed by giving some brief thoughts on these additions.

One of the most obvious, sustained additions was adding Open Mind Platform, which I wrote about in October. It went as I had hoped, with my seeing evidence that students learned from the content of Open Mind itself but also thought critically about its implications for HCI design. I have also been in touch with the team that manages the platform, and I have reviewed some revisions that they are making to further improve the teacher's experience. Their dedication to improving Open Mind excites me, and I am glad that I was able to play a part in it. In fact, I recently added Open Mind Platform as an Achievement in my sketch of CS222 plans for Spring 2021, which will be the topic for another day's blogging.

I think I made good use of Scott Klemmer's lectures, balancing them against several of my own lectures that gave my perspective on various HCI topics. Whereas Klemmer's presentations were generic, I was able to focus mine on the specific assignments and tasks that I had in mind for my students. I do not know when next I will be asked to teach this course, but I will need to consider how I might reuse some of this course design for face-to-face classes.

Another valuable addition was Nielsen's Usability Heuristics. Given the challenges of empirical testing in the face of a global pandemic, performing heuristic evaluation was much more practicable. Nielsen's approach combined well with the ideas in Norman's Design of Everyday Things, Gestalt principles, and the accessibility heuristics we explored, showing students that there were multiple valid and useful ways to evaluate a system.

I am pleased with how the final project went. I assigned them a challenge that I myself faced earlier this year: taking a traditionally in-person Student Showcase from the CCSC Midwest conference and making it an online asynchronous event. Rather than just throw them at the problem, I scaffolded a series of six weekly assignments that walked through a reasonable, human-centered process. Only two weeks out of the six were spent on building, which I think surprised many of the students. The technical artifact was not the primary outcome of the process but rather a secondary one: the primary outcome was the report that students built on week by week. In the end, these ranged from about 15 to 50 pages long, depending on students' use of images and their attention to detail. After reading their final project submissions, I recorded a response video in which I encouraged them to consider the extrinsic value of this document, given that it provides clear evidence to potential employers that they themselves can follow best practices of design—not just parrot a few definitions from a textbook.

The final project was not without problems. Some students fell behind, and there was little recourse for them. I provided verbal instruction on how to progress; for example, if someone was unable to get a build working, I told them to reach out to classmates and continue the project using someone else's build. However, no one did that: they either made something up or gave up on the project. It is worth noting that the view counts on my weekly feedback videos were noticeably less than the course enrollment, so I am not even sure that the students in trouble knew that I had given them instruction. 

The best of the reports explicitly compared their results to the one that I used. I did not require this, but I was surprised that only of the students thought to use my solution as a benchmark. I was a little disheartened to see how many students praised their solutions based on the limited understanding and testing, but most seemed to be honest about their strengths and shortcomings.

While planning for this semester, I went back and read my post about the Fall 2019 offering, which regular readers may recall was particularly challenging. Seeing the high quality work of these students, I wish I had them in a semester with a properly community-engaged project! The CCSC Midwest Student Showcase was a fine case study, but it was still primarily about my work as opposed to a community member's. Ah well, if my biggest regret is that I could not get these students working on something even more public, then I think that means it was a great semester.

Wrapping up OpenMind in HCI

My HCI students wrapped up their experience with The OpenMind Platform this week. Back in September, I wrote a bit about how it was going. Since then, they completed modules three through five. I gave them reflective activities after each of the first four modules, tying the module to the week's activities:

OpenMind	Course Topic
Irrational Mind	Goal-Directed Design
Moral Matrix	Cultural Constraints
Intellectual Humility	Errors
Value of Diverse Perspectives	Heuristic Evaluation
Constructive Disagreement

For the final module, rather than tie it into the week's topic of accessibility, I asked them to reflect on the OpenMind experience writ large and relate it to their study of human-computer interaction. This reflection was submitted confidentially to me rather than as a discussion board post, where practically everything else has lived this semester. I wanted to ensure that I was getting as honest feedback as I could expect from them.

The results were overwhelmingly positive. Everyone said it was useful to their studies, and most agreed that its impact reached well beyond the course. The most commonly cited portions of the program were the importance of a growth mindset—particularly in the face of receiving feedback on designs—and the elephant-and-rider metaphor. One student adroitly recognized the connection between OpenMind's themes and the Norman's themes in The Design of Everyday Things. Another pointed out how his background in social science made him skeptical of the program, but that going through, he found nothing questionable nor overreaching about it. One of the more curious but interesting responses was a student who said he wasn't sure that he got anything out of it, yet he enjoyed the exercise of tying a module's ideas to the week's topics.

One student argued that we didn't go far enough with the ideas brought up in OpenMind, that they have deeper connection to HCI than we gave them credit. He pointed out that OpenMind prepares you for meaningful interaction with different others, but that we stayed in conversation with the same classmates all semester. This is quite an observation, and I'm proud of this student for pushing in this direction. I had two themes in my response to him. One is the easy one, that the global pandemic complicates a lot of things, but I could certainly consider rearranging other parts of the course to make room for such activities in the future. The other is that I retain a fear of tokenization. Disability is diverse just like everything else, but the observation leaves me a stake in the ground to consider for future offerings of the course. Also, he may not yet have looked ahead at the just-published final project specification, which does actually require interviewing people as part of the empathy-building process, although not necessarily along lines of disability.

Overall, then, I call the integration of OpenMind an unqualified success. All those who mentioned course integration lauded it and encouraged continuing it. One suggested that it would be even more valuable if given in high school, to counteract the tribalism that arises there. This leans into my continuing consideration of OpenMind in my teaching: would it be valuable to roll this out in my CS222 class next semester? I need to give it serious consideration once I have a chance to start planning that course. (I am still waiting final word on the room and meeting arrangements, which is of crucial importance to planning, of course.)

My main criticism of OpenMind from a teaching point of view is that it's very difficult for me, as an educator, to get in and browse the material. I went through it myself, but memory is imperfect. They provide a dashboard of sorts for teachers who assign the program, but I had no use of this: I wanted to be able to browse or even dump out the contents of each module so that I could review it and mention specific parts in my feedback videos. I understand that they don't want people accessing the material outside of their platform, for research and analytics purposes, but it would still be quite valuable to me. I would rather not go through it again myself each time I am planning a course and trying to remember the details of how a student interacts with it.

Initial thoughts on OpenMind in HCI

I am taking a moment this morning to capture a few thoughts about integrating the OpenMind platform into my HCI course. I wrote a bit about the initial decision in my post about the course re-redesign. Yesterday, I read my students' second batch of submissions, and I can say that incorporating this platform may have been one of my better decisions from the summer.

In the first week, the students completed the first module, and I had them write about whether they could identify the use of motivated reasoning or confirmation bias in past project work. Most were able to come up with good examples, a few did not, and a few did something much more interesting. These students responded first by saying they didn't think they saw either one, then they explained lucid and clear examples of using these. That is, even in writing about it, they tried to deny this as if it were shameful, but then admitted to having done it. In my feedback, I tried to point out that the important thing was to recognize it, not to feel bad or shameful about it.

This past week was even more interesting. They completed the second OpenMind module, this one on moral foundations theory. At the same time, they read Norman's Design of Everyday Things presentation on constraints, which includes a taxonomy of logical, semantic, cultural, and physical constraints. I asked them to consider how moral foundations theory influenced cultural constraints of design. I have to admit a certain pride in that question, and the students' responses by and large showed that it was both challenging and thought-provoking. A few students were only able to give very broad, abstract answers, essentially answering the question by restating, or trying to answer the easier question of whether it does rather than how it does. Here, it was an opportunity for me to remind them that human experience is almost all about abstractions and assumptions, while design is all about specifics. Indeed, that is a major theme of Design of Everyday Things' discussion of human behavior. Some of the students were able to come up with excellent insights that related the theories of moral foundations and design. Most noteworthy, however, were the few students who made generalizations about all people based on their own moral foundations. That is, they made claims that all peoples preferred safety or liberty, and I was able to push back on the distinction between how one sees the world versus how the world is.

They will be completing the remaining modules over the next three weeks, and I am eager to see how this trend continues. At the end, I want to talk to them about the holistic experience of completing the OpenMind modules, and maybe at the end of the semester try to come back to it and see if they think it impacted their designs.

Summer Course Re-Revisions 2020: CS445 Human-Computer Interaction

Yes, you read that title right: this is a re-revision of my Human-Computer Interaction course, which I wrote about revising back in early July. Since then, I learned that all my courses would be asynchronous online (after being told that we would almost certainly be able to do them with synchronous online meetings). As I mentioned in my game design revision notes, then, I have had to come back to the HCI course to figure out what to do about it.

Like basically all of my in-person course plans, my HCI course traditionally relies upon in-class activities to help students understand the concepts and build a community together. The original synchronous plans assumed that we would have time to work together, even if distributed physically. Removing that constraint meant that I had to revisit the design to make all the required conversations asynchronous, which means more formal structure around discussion board posts.

After some consternation, I decided to follow a model similar to what I am using in my game design course. I will set up small peer groups who are required to read and comment on each other's work, in the hopes that this not only helps students sharpen their skills but also build some community and get to know each other. I thought about adding a requirement that each group get together online to discuss the readings and work, which is something I've seen in a colleague's online graduate course, but I decided that I wanted to keep this in fairly small, contained steps.

Along those lines, I copied over and lightly edited some of the additional prose I have been adding to course plans back into the HCI one, including several paragraphs about commitment. This may be one of the hardest parts for the students. It has always been the case that I have designed my courses to take nine hours of effort per week, which some students struggle with anyway; without the valuable three contact hours per week of structure, discussion, rapid feedback, and encouragement, it is not clear to me how many of my students will be able to keep their heads above water. Yet, it seems to me to be the only honest option: if we're going to take their money for an education, and federal financial aid law makes these demands on the effort-hour-cost equivalency, we cannot pull back on expectations without a concomitant decrease in tuition and fees. Assuming I can keep my head above water, my plan is to record a short talking-head video each week to release to my students, talking about what excites me about their work and where I see challenges. I hope that will at least help them remember that I'm rooting for them.

I reviewed all my notes and did some additional research as part of my re-redesign, and I added two new elements to the course which I hope will work well with the distributed mode. I added several videos from Scott Klemmer's series of lectures based on his Stanford course. I watched a lot more of the series than I will be assigning. It certainly seems prudent to use his nicely polished videos rather than record inferior ones myself. Incorporating his content made me rethink the ordering of some of the topics I wanted to cover as well.

The other formal element I added was a requirement to complete the five-step OpenMind tutorials. I heard about these through Heterodox Academey I think, and this past week I took some time to actually go through them. There was nothing in it that I had not heard before, but I suspect there was quite a bit that my students would not have. Some of it is relatively simple contemporary psychology, such as the idea that the emotional system is fast and dominant but trainable (the riding-the-elephant metaphor). Other parts mix pragmatism and ethics, such as the importance of intellectual humility. Taken as a whole, I liked the presentation, my only major criticisms being that it starts on the wrong foot and has an imbalanced obsession with Buddhism. Knowing the unprecedented breakdowns of discourse in modern times, and how the Internet seems to be fueling the fire of ideological demonization, it seemed like the HCI course was a good place to inject some of this content. I am eager to hear what my students have to say about it.

We are less than two weeks from the start of classes, and I find myself nervous about the semester. I am grateful that I can do my work from home, but I am concerned about my students' experiences. I still don't really have a plan for how I will do anything like office hours aside from a passing fancy about livestreaming my course-related relevant work. At the same time, I've put in an unprecendented amount of effort this summer to revise my courses for Fall, and honestly, I feel like I could use a vacation, just as the meetings are kicking up for the semester's start.

Reflecting on the Fall 2019 CS445/545 HCI Course

I have to start by saying that this was one of the strangest classes I have ever taught. We were continuing my series of collaborations with the David Owsley Museum of Art, and as I wrote about in July, I had a great meeting with them to set up some tighter constraints around how we work with the students. There were only nine students in the class, which I thought would be really exciting: small team, a couple of graduate students, working with a partner on exploratory software to enhance the visitor experience. It seems like the recipe for an excellent learning experience, but in truth, this was one of the most frustrating classes I have ever taught.

I have never had a class where I have to "pull" so hard to get them to do, well, anything. On the very first day of class, there were about ten people in a room that seats over thirty. When I walked in, they were all sitting apart from each other, staring into their phones. I commented on how quiet it was, and I encouraged them to move in toward the front. Nobody moved. We repeated this ritual basically every class meeting. It became something of a joke after a while—a sad, sad joke. On a few occasions, I forced them to rearrange the furniture so that we could sit in a circle for discussion, and these meetings were always better, but they didn't seem to have any impact on the de facto standards. One time, I came into class, and two or three students were talking to each other. I heaped praise on them in hopes of some positive reinforcement. I got a few smiles, but again, no real change.

When we moved into working as one big team, I let them follow their own path for the first two week sprint. After a structured reflection, I told them that I would be scaffolding their improvement by providing a methodology. I used one based on my immersive learning teams, which have been roughly the same size. One of the rules of the methodology is to work in pairs whenever possible. They ostensibly read the methodology and we got to work... everybody working silently at their own laptops. I paused for a minute or two, then I interrupted, pointing out that they were all in violation of the methodology, and that they should pair up. Some of them still did not. At that point, I feel like I just have to throw up my hands.

With only nine people, attendance irregularities are easy to notice, and many people missed many class meetings. It's not like it didn't hurt their grade either: they had work to complete and then discuss almost every class meeting. I think it's fair to say that it's disheartening for everybody in the room to look around and see that only five or six out of nine people are there. In the latter part of the semester, we worked as one consultancy, and there was work for everyone to do; even here, people missed critical planning and reflection meetings.

The point of this writing is not just to complain, though. We did have some real stand-out meetings. In one of them, we talked honestly about their past team experiences. They acknowledged that none of them had ever really been on a non-dysfunctional student team before, and also that they didn't really know what a successful team looks like. This is invaluable to me as an educator, because it makes me realize that it's not just enough to give them guidance: I think we have to work harder to show them examples of successful teamwork to model. I am still not sure how to do that, except maybe by filming one of my high-functioning immersive learning teams. Another important part of this discussion was their acknowledgement that in the prerequisite course (CS222), they learned to fear feature branches and pull requests and, generally, GitHub. That is, they saw these tools as impediments to their success rather than what they are: critical parts of a healthy and productive work environment. This again points to some specific actions, to make sure that the prerequisite course is not accidentally teaching counter to its purposes. Conveniently, I've been assigned to teach CS222 in the Spring, so I will be able to pilot a few interventions here.

One of the frustrating outcomes of this semester is that I am not really sure whether the students learned anything or not. We studied some of my favorite theories of HCI during the semester, always embedded within the context of our collaboration with the art museum. My plan was, near the end, to return to those theories and frame our work within them. We ended up having to declare a failed sprint a few weeks from the end of the semester in order to produce a barely-testable digital prototype. This ate up the time that I was hoping to use to close the loops. Looking at the work that the students did on the project at the end of the semester, while technically competent, I didn't see any consideration for any of the theories we discussed earlier in the semester, like Don Norman's action cycle or Gestalt vision principles. Instead, I saw what it looked like they would have done if they had never taken the course. This is disheartening.

In our final meeting, our museum partner was kind to point out that the prototype these students created was the highest quality of any that students made in the previous semesters. I generally agree, and this is in large part because we had one group and one consistent series of conversations. From a teaching point of view, it's a completely different thing to have one team of students to engage with than it is to say, "Get into your groups and talk about this." Also, because we all worked together, I could give more direct guidance on some of the technical issues of the implementation as well, which prevented them from getting caught in amateur's dead-ends. For example, there's always someone who thinks copying and pasting code will do no harm, and then you end up with an unmaintainable mess that collapses under its own weight; I was able to work with students to refactor such solutions, teaching both process and techniques for refactoring along the way.

Although our partner was positive and right to be so, I remain concerned at how it seemed the students didn't really think about what they were saying or doing—they were not critical. For example, they liked to mention that they used journey maps, but they didn't do this well. I graded all the journey maps and provided feedback about the parts that were good and bad, but in their presentation and final essays, they wrote about their journey maps as strictly virtuous. I mentioned Falk's theories of museum visitor motivation in class, and the students latched on to parts of this; in the presentation, though, they made it sound like they had actually studied and applied these theories. In fact, they had done the equivalent of a standard contemporary undergraduate practice: hear of something, Google it, put some buzzwords in, and call it satisfactory. Truly, their presentation of their knowledge of Falk's model bordered on lying to the client, and I just wasn't prepared for how to react.

If you got this far into this essay, you can understand why this class that seemed like it should be so good would actually be so frustrating. I'm still not sure about the root cause, but I'll share the best thought I have. I don't know why the department administration thought this was a good idea, but we've offered HCI as an upper-level elective for something like five straight semesters—including summers—when it used to be a biannual course. The number of students in my classes has gone down each time, and with only nine completing this time, I have to think that a majority of this small group didn't really want to be there. I don't think they had any motivation to either take HCI or to take a class with me. (Not to toot my own horn, but there are some students who just want to study with me, regardless of the course.) In the absence of motivation, the course is perceived like the methodology I wrote up for them: a hurdle to be cleared rather than an idea to explore. I suspect I could have just given them readings and assignments, and they still would have skipped a bunch of classes and earned their C and B grades, and I would have been able to sleep at night. But you know, that's not how I roll.

I had one day where I was feeling particularly frustrated as well as low on physical and mental energy. I don't remember where we were on the project, but I asked the class to give an update on their progress. Nobody in attendance had made any. It was one of those times when I had to seriously think about just leaving the room, just walking away and letting them do whatever it was they thought they should be doing instead of contributing to the class. I even said out loud, "I have done more work on this project than you have collectively," which is probably not exactly true with a class of nine people, but I think it was dangerously close. I took a minute or two to collect my thoughts and, with some grace, turned it into a discussion of how to move forward. After this, one of the students—one with whom I had some struggles earlier in the semester but with whom we grew into mutual respect—started occasionally thanking me for my work. He would say something like, "Thanks for taking the time to put this collaboration together," and he really meant it. This may seem like a small thing, but it wasn't. It's possible that some of the students still see faculty generally as some kind of automata. I think this student saw that I cared, and because I cared, I hurt. We talk about helping students build empathy, and here's a case where it actually worked, not in some abstract social justice sense, but in a very concrete, local-community sense.

This post is a bit long, but I have had a strong desire to try to capture these stories. In truth, I'm not even entirely sure why, because in conclusion, when I consider what I would do differently next time, I honestly don't know. One thing I would do is absolutely put my foot down on the "nine people spread around a classroom" on day one, though. That was just ridiculous.

Thanks for reading. Feel free to share your thoughts and suggestions. The next one will be more cheerful, I promise.

Reflecting on the Spring 2019 CS445 Human-Computer Interaction Class

Regular readers may recall that I was given the Spring 2019 HCI course to teach on rather short notice, so I only made a few structural changes between it and the Fall 2018 section. The most relevant to this post are the increased attention to software architecture and the switch to specifications grading. I also gave the teams nominally more time for their final project, but not enough that it was noticeable from my point of view. We retained our collaboration with the David Owsley Museum of Art (DOMA) and the overall theme that student teams would identify and address real problems they face.. Yesterday, I shared my sending-forth message to the students, and today, I would like to share my reflection on the semester's experience. Feel free to reference the course page, which provides the policies, procedures, assignments, and assessments for the semester.

I used a similar approach to specifications grading as I did in the Fall 2018 Game Programming course, in which there were discrete criteria for each level of grade. I added a separate category of criteria for the project reports, which were designed to provide the process documentation that corresponded to the technical artifacts produced. As before, students had to submit a self-assessment along with their source code and report, the self-assessment's consisting of a checklist of criteria that were met. Unlike the game programming class, where there was rarely disagreement between the students and I about whether a criterion was met, there was a lot of friction this semester. This was especially the case on the final project's two iterations. As I wrote to several students in my formal feedback, I have serious doubts that many of the teams honestly conducted a self-evaluation at all. Consider, for example, one of the most missed criteria asked students to explain how their projects manifested particular design principles from Don Norman's The Design of Everyday Things. Teams submitted a list of examples, with no explanations of them. It seems to me that if a team sat together and worked through the checklist, as I expected them to, someone would have said, "Have we explained this?" I don't think they had anything approximating such a discussion: I think they surveyed the checklist, said "Good enough," and marked the box. That is, I think they defaulted to the "hope for points" model rather than the "ensure success through unambiguous choices" model. Of course, the idea of self-assessment is not to save me grading time but to foster reflective practice and remove ambiguity. When students do it honestly, it does save me grading time, and when it is done dishonestly, I suspect that students might learn something about the importance of self-reflection. I need to think about what I might change in future uses of specifications grading to get around this.

An honest student approached me near the end of the semester, in part to share what he claimed was a voice of many students who were frustrated with the specifications grading system. I explained to him that the goal of the system is to remove ambiguity, both for students and for assessors. I honestly never got a good explanation of what precisely he or other students did not like about it, except that there were standards at all. I think the status quo is that students believe they start a project with full credit, and then I take away points for mistakes. One of the things I like about specifications grading is that it follows my contrary philosophy, which is that students start with nothing and must earn their credit. I think it is this idea, not specifications grading in particular, that students are upset about, because it holds the accountable to demonstrating understanding to earn credit. The fact that I get complaints about grading regardless of the scheme I use is probably testament to students' pushing back against having expectations rather than the particulars of the system. However, foreshadowing some of what is to come below, part of why a subset of students complains is likely that I actually draw upon knowledge they should have from prerequisite courses—knowledge that they may not actually have.

In the first half of the semester, I used a running demo project ("archdemo") to demonstrate some ideas of how to separate the layers of a user-facing software system. In the previous semester, I had done something similar, but using a context separate from our class collaboration with DOMA. Many students that semester did badly with the "warm-up" project, and so in order to help with on-boarding and consistency, archdemo showed a sample use of the DOMA data via the ContentDM database. The resulting application was called "Naïve Search," named thus because it didn't really solve any reasonable search problem: it just showed how to separate the layers of a system. While this worked in the short term, I think it also caused problems as students perceived more value in the example than it was meant to have. It was never intended as a template, but only an example of very specific course concepts.

One of the changes I made from Fall 2018 was that I required final project teams to use a subset copy of the ContentDM database in their projects. My intention here was that each team would have to demonstrate that they could separate the layers of a user-facing software system, regardless of what creative direction they wanted to take the project. The result, however, was that nearly every project looked a lot like archdemo with an added bell or whistle. Last semester, we had a broad range of concepts on a plethora of platforms; this semester, it was dullsville, as the teams just added some minor idea to archdemo. One team even consistently referred to their solution as "an improvement over Naïve Search," despite my repeatedly telling them in their formal feedback that this was not even close to our goal. I have no doubt that our partners at DOMA were uninspired by this semester's projects, although we have not had our wrap-up meeting yet. I would be remiss not to mention one exception, which was a clever interactive map that tied into the database in interesting ways despite the tight project timeline; those guys really nailed it, so if you're on that team and reading this, kudos to you.

Throughout the semester, we returned to five principles of design brought up in Don Norman's The Design of Everyday Things: affordances, signifiers, mapping, feedback, and conceptual models. Despite this being a theme of the class, there is scant evidence that students understand or applied these principles. Instead, my professor's eye tells me that they designed whatever they wanted, and then they tried to shoehorn those designs into these principles, or to justify their work after the fact. Although they had several assignments and much formative feedback about these principles, students continued to show misunderstandings through the final exam.

What was missing? I believe a big part of it was that they didn't follow my advice. This is exemplified with one key example: taking notes. The course plan says explicitly that students should always have their notebooks available for taking notes and jotting questions, and furthermore, that they should not have their distraction machines (laptops and phones) in their way during class discussions. Taking a friend's advice, I even made a first-day quiz in which students had to answer questions about this aspect of the class. Yet, very quickly (and for some, instantaneously), their old habits took over, and I would stand in front of class looking at the backs of open laptops rather than faces. Almost no students took any notes on any of our discussions, and is if to drive the nail of the coffin, some of them only got out their pens when I wrote something on the board. Even if they had a glimmer of understanding about affordances and signifiers during class discussion, there is no way that they held on to this fifteen minutes after class unless they actually expended the effort to do so.

I wrote on Facebook the other day about how I was feeling conflicting emotions about this class. On one hand, I am unsympathetic that they did not learn the material because they chose not to follow my advice on how to do so. The advice is not complicated: it primarily involves reading and taking notes. However, at the same time, I pity the students, because I think a large majority of them—if not all of them—know neither how to read for understanding nor how to take notes while reading or discussing. To me it begs the question, "Where does the buck stop?" If I get undergraduates in my upper-division elective Computer Science courses who lack these skills, is it my responsibility to teach them or just to assess what is in the master syllabus?

There is a related puzzle, which was foreshadowed in my sending-forth message to the class. An uncomfortably large proportion of the class showed very little proficiency in fundamental programming skills. When I brought this up in honest, private conversation with trusted undergraduates, they showed no surprise: they said that it was fairly easy, and common, for students to "cheat" on assignments. This manifests in two ways: either copying the work of a peer and submitting it as their own, or stringing together bits and pieces of code found online. Neither approach forces the learner to confront the useful struggles required to build firm understanding. It reminds me of the advice from Make it Stick that I wrote about last December, and that in its absence, students really don't know how or what it means to learn.

Going a little further, I witnessed a curious phenomenon several times during a guest presentation by a CS alumnus and successful professional. The speaker is currently in a position with a lot of creative flexibility, and he has his own team of programmers to implement parts of what he designs. However, many students seemed to misunderstand his story, thinking that this meant one could just "have ideas" and tell others to program them. They missed the part where he worked on rather tedious programming tasks for ten years to prove his capability, vision, and leadership. Instead, they rejoiced, saying things like, "I cannot program, but I want to tell programmers what to do, so now I see that there's a job for me!" This sentiment was shared primarily among CS minors despite their having taken at least three programming courses in the prerequisite chain to this course.

These students don't seem to see a connection between the HCI goals of the class and the fundamental skills of software development. It appeared to me that these students were not being tripped up by the accidental complexity of software development (such as the placement of braces or the quirks of a UI framework) but rather by its essential characteristics, which include precision and sequential reasoning. How I frame HCI as a Computer Scientist is essentially that it is user-centered precision and sequential reasoning. What happened on the students' final project teams seemed to be that those who had programming skill were relegated to doing persistence- and model-layer data manipulation tasks—required tasks for the program to work at all—while those who could not program worked on the UI. The result is that the UIs were badly conceived and executed, because those working on them couldn't conceive of the problem as requiring precision and sequential reasoning. Part of my evidence for this manifestation is the difference between teams' paper prototypes and their final products. Every team decided upon a paper prototype that was developed from a user-centered design process, but practically no team's final product looks at all like their prototype. Instead, their products looked like the archdemo sample, but with a few more widgets added via SceneBuilder. One could say they did what they could rather than what they wanted, or more pointedly, they decided to fail conservatively rather than succeed differently—which was exactly one of the human failure modes we discussed in class.

A student confided in me that, when he signed up for the course, he expected he would learn how to design a good user interface. By this, he meant that there would be some thing I could teach him that would suddenly make him good at it—a silver bullet. He pointed out that some students seemed to think that it was all in the tools: if they learned the tools, they would be able to make good UIs. I am grateful that he took the time to share this with me. I asked him if, after studying this topic for fifteen weeks, he understood why I could not meet his desires, why I could not dump ideas into his head that would suddenly make him good at UI design. He indicated that he did understand it, but he also sounded disappointed.

As I wrote about yesterday, I had forgotten about the emotionally powerful reflection session I had with my students at the end of the Fall 2018 HCI course. I didn't schedule for it this semester, and so it didn't happen. I think it would have helped all the students to frame their difficulties and challenges within their authentic context: yes they struggled, but they did so because what we are doing is legitimately hard.

The puzzles I face in considering how to change the course for Fall 2019 are significant. I expect this week to be able to meet with representatives from DOMA to talk about their take on the experience. Our primary contact is their Director of Education, Tania Said, and she has intimated that she would like to see the class work together to produce something with more staying power rather than a series of prototypes. This makes me a bit nervous, given my previous experiences having entire classes taking on a project, but there may be a possibility to set it up like competing consultancies rather than trying to do a whole-class team. One of the reasons I wrote this up now is that it has helped me serialize and articulate some of my thoughts in preparation for meeting with her. I hope that conversation will help me turn some of these reflections into actionable course plans for Fall. As always, I expect I will be able to share my summer planning activity in a blog post in the coming months. Until then, I think it may be time to spin up my summer project.

Sending forth the Spring 2019 HCI class

I spent the morning computing final grades for my Spring HCI class. Last night, I came across a post I wrote at the end of the Fall 2018 section that I had forgotten about: my post about how powerful our final reflection session was. This, along with other thoughts and desire for closure, inspired me to compose the following final announcement for this semester's class. I have another post I am working on which is a reflection about the semester, but I also wanted to share this as an open letter here. If I didn't, it would be trapped away on Canvas where I wouldn't be able to reference or reflect on it later. What follows is the announcement in its entirety, except for the first paragraph, which simply dealt with the final grading formula and reporting mechanisms.

---

I regret that we didn't schedule a day for a semester wrap-up discussion. Because of the problems scheduling our final presentations with Tania Said, we went right from working on the final project, to technical presentations, to formal presentations... and now time is up. Even though this is a one-dimensional stream, I would like to share a few of my final observations, in hopes it is inspirational to you as you move forward in your studies and career.

A conversation just fifteen minutes before the final exam made me think of the perfect question for the final exam. Of course, it was too late to make any changes at that point, but I did post it up on Facebook for some of my friends and alumni to comment upon. It looks like this:

Many of you signed up for this course expecting that I could pour some special knowledge into your heads that would make you good at designing user-interfaces. Now that we've studied design methods for 15 weeks, explain why what you wanted is not possible.

The resulting conversation on social media has been interesting. A colleague who also teaches undergraduate HCI mentioned how he regularly gets students in his class who conflate visual design with HCI. A successful alumnus from about five years ago posted, "It's a field without universal truths except for, 'Know thy user, and thy user is not you,'" which I think is a brilliant summation. Another alumnus who does a lot of user-facing development pointed out that there are always accessibility issues in any design, responding to a nested conversation about how accessibility is important but also platform- and application-dependent. A friend who teaches game design in Michigan simply said, "People are broken in interesting ways," which also ties in to many of our discussions of design processes, working in teams, and confronting our human biases.

I share this with you to help you compare what you know now to what you new before the semester started. Undoubtedly, there is something you can do to take the next step of improvement in your HCI skills, and my hope is that this course helped you identify what that is. After all, the goal of a liberal education is not prosaic job training but helping people understand how much there is yet to learn. Successful professionals are those who engage in reflective practice: thinking carefully and critically about their work and continually improving. Keep close to the heart of agile.

It would be irresponsible of me not to share with you an observation I made from working with you on the final projects. For many of you, the thing that is stopping you from creating innovative and useful systems is programming. I was disappointed to see so many in this class struggle with fundamental topics from the prerequisite courses, such as variable scope (CS120), parsing a one-dimensional data structure (CS121), and naming variables and methods appropriately as verbs and nouns (CS222). There are prerequisites because a solid background is required to succeed at the level of discourse appropriate for a 400-level course. How a student may have gotten to this level of the curriculum without such knowledge is a question for their own personal reflection; prudence demands considering what to do next. What ought one do who wishes to improve their software development skills? As I shared with you from Brown et al. Make It Stick, "To achieve excellence in any sphere, you must strive to surpass your current level of ability."

A creative mind may come up with no end of possible hobby projects; for such a person, the problem can be identifying a reasonable scope. You can always look up the kinds of questions that are asked on technical interviews, participate in the mathematical challenges on sites like Project Euler, or join a community of practice around a passion area such as game development or humanitarian open source projects. If you're not sure what else to do, I suggest working through this semester's projects again. The course plan will stay online. You can start again on the short project, but do it from scratch: don't fall into the trap that every team did for the final project and assume that my naive search architectural demo had any implicit value. Build up something for which you understand every part. Hold yourself accountable to best practices, which you can refresh yourself on by re-reading Clean Code. It's possible that our image server will be taken down some time this summer, but that doesn't have to stop you: you can find another data source to draw from, as the first part of the final exam implied. In fact, the university is moving to a cloud-hosted ContentDM database, and I'll be talking with Ms. Said and Mr. Bradley about how future students might take advantage of that.

A. J. Liebling wrote, "Freedom of the press is guaranteed only to those who own one." Andy Harris said, "You want to be a game designer? You write the code, or you write the check." Steve Jobs said, "To me, ideas are worth nothing unless executed. They are just a multiplier. Execution is worth millions." Kyle Parker told us that he was proving himself in the trenches for a decade before he was given a position of creative authority within the university. We are Computer Scientists—whether majors or minors!—and we are the ones who create value in the 21st century.

I hope that you have had a formative experience in this class. I wish you the best in future endeavors and will pray for your success.

Two missing specifications in HCI

I finished grading my students' final projects for the Spring 2019 HCI class (CS445, used to be CS345). Before the start of the semester, I wrote about how I would try specifications grading in the course. After the afternoon of grading, I realize that I missed two important specifications. I will share them here so that I have a better chance of remembering when planning Fall's class, since I've been assigned to teach the course again.

I should have had a specification requiring all non-trivial processing to be done off of the event thread. This is, of course, a requisite for any kind of multi-threaded UI programming. I specifically chose a data source that would require handling slow load times and long processing times so that my students could practice this technique. I developed a sample project in the first half of the semester based around this common practice, and I explained to them why it was important. However, I neglected to have a specification about it. Three hours before the final project was due, I had a student ask for some last-minute troubleshooting. He said that he added a spinner while some images loaded, but it wasn't showing up. Of course, it wasn't showing up because he was loading the image on the event thread. I showed him (again) the example from earlier in the semester and explained (again) why this pattern was necessary. From their final technical presentations, it was clear that he was the only person in the class of roughly twenty students who understood this crucial point. I believe this is an instance of the old standard motto: if it's important, make it worth points. I simply missed it in my specifications.

The other specification deals with acceptance testing. There are two relevant specifications in the evaluation plan, one at B-level and one at A-level. Specification B.5.R says that the final report " describes the methods by which the solution was evaluated," and A.2.R says that "The documented solution evaluation includes both quantitative and qualitative elements that explicitly align with this semester's readings." The B-level specification is designed to be broad: you can earn a B on the project by doing any kind of acceptance testing. The A-level specification is designed to be more focused: do a mixed methods evaluation based on a theory we studied this semester. None of the five teams explicitly aligned their evaluations with the semester's readings. This didn't stop two of the groups from marking that specification as complete in their respective checklists, casting serious doubt on the implicit claim that they had conducted the required self-assessment for which the checklist is a result. (Perhaps, then, I need to add more rigor to the self-assessment itself, requiring them to link their claim to the artifacts.)

The problem with the acceptance testing actually goes much deeper than dishonest claims of completion. Among those who conducted any kind of acceptance testing, there was no evidence of their having learned anything from the assigned readings and exercises relating to Steve Krug's and Jakob Nielson's theories. Instead, they followed ad hoc approaches that were poorly designed and yield unreliable results. They did actually use quantitative and qualitative approaches, in keeping with A.2.R, but they did not do these well. For example, many groups asked questions like "What did you think of the application?" and then reported "3/6 users say they liked it." I pointed out in my feedback that 50% of users claiming they liked it is different from 50% of users liking it. More importantly, "liking" the application was not one of our design goals: we were designing systems to be used. Yet, only one of the groups conducted a task-based evaluation, where the user was asked to accomplish a goal using their system. Task-based evaluation is what I expected, and task-based evaluation is what I wanted. However, I wanted the students to realize that this kind of evaluation was the right choice, so I left the specification open to other options. The other options were demonstrably worse. Hence, in the future, and particularly in this introductory HCI course, I should just require them to follow the best practice rather than give them the choice to shoot themselves in the proverbial feet.

I have to wonder if the students would have spontaneously met these criteria if they had taken notes during our discussions

"Who Would YOU Rather Work With?" A classroom game show for my HCI class

On Tuesday last week, my HCI class gave their informal reports for the first iteration of their final projects. I neglected to post any real guidelines about the presentation, and so it was ad hoc. We did have two guests in the room, though, one being an expert user for the projects and the other being a software development professional. There were several points in the students' presentations when I cringed at how they presented their status and ideas. I considered for a while how to address this, and I ended up coming up with a classroom game show called Who would YOU rather work with?

This is a rhetoric game along the lines of The Metagame and CARD-tamen. There are two teams, each of which sends up a player for the round. The two contestant are shown a pair of fictional quotations, and based on these, must argue for who they would rather hire onto a team. We alternated which team went first, and each player had 30 seconds to make their cases. The winner was determined by popular vote. Even though a team could just vote for their own representative, I think the students understood that it should be a fair vote.

The original idea for the game was Is it User-Centered... or Not?, but I realized as I started working on the game systems that user-centeredness was only part of the equation. More of the problems were with students' rhetoric than with their technical or procedural approaches, and hence my choice to make this a rhetoric game.

We played the game last week Thursday. I had the students count off by twos in order to shuffle them around, so they would not be on a team with the person they usually sat with. The two teams named themselves The One Before Two and Los Dos. (Important note: The One Before Two insisted that the short name of their team was TOBT, pronounced "tot" with a silent "B".)

Here are the prompts that I gave them:

1.
"The images take a long time to load, so the user just has to learn to wait."
or
"We will show a spinner to signify to the user that the system is still working."

2.
"JavaFX is a pain. It's really fiddly."
or
"We thought we knew the JavaFX API better than we actually did. We talked about it and identified our main misconceptions, and we are working to overcome them."

3.
"We were all really busy so we did not work on that feature."
or
"We documented that feature as a user story to consider for the next iteration."
[or, in a bonus round, "That feature is out of scope."]

4.
"Git kept on destroying our files. It doesn't do what it is supposed to do."
or
"We had problems managing version control, and one of our impediments is our own understanding. We have prioritized finding the root cause so that it doesn't happen again."

5.
"We are making this application using Java and JavaFX because Dr. Gestwicki provided an example with these."
or
"We have analyzed our target population's needs and have chosen a development platform that allows us to deploy a solution that meets their needs."

6.
"We did a lot of work that you cannot see exposed through the UI right now, but it will be incorporated into the next iteration."
or
"Our source code provides the simplest possible solution for this iteration that maintains our standards of quality."

7.
"We are not sure if this is an SRP [Single Responsibility Principle] violation or not."
or
"We have evaluated our code for compliance with SRP and, where we were unsure, we consulted with the professor to evaluate the structure."

8.
"We know that what we have planned for iteration 2 is going to solve our problem."
or
"We learned from iteration 1 to help us improve our product and processes for iteration 2."

9.
"We based our work on Dr. Gestwicki's example."
or
"We built a solution for this problem based on our users' specific needs."

The last one is quite a bit like #5, but that was in part to handle the case where everyone actually showed up to class and would get a turn. Of course, not everyone showed up to class, so we had enough questions that some people got to answer two.

One of the students hypothesized out loud, in round three or four, that the bottom answers were always "right." I pointed out that this was a rhetoric game: there was no objectively right answer, but rather a challenge to persuasively argue your position. Once the game was over, I explained to them that there was indeed a pattern, of course. The top item was more or less what I heard people say during their presentations, but the bottom is what I would have like to hear instead. This caused a murmur in the crowd as they considered what this means. It gave us a good excuse to talk about how you present yourself when applying for a job or internship: that the applications will be sorted into two piles, and you want to make sure you end up in the "Let's talk to this person" pile.

For the most part, students argued that they would rather work with the second person on each slide, using the kinds of arguments you would expect. The statements on the bottom tend to show capacity for reflection, eagerness to learn from mistakes, honesty, professionalism, accountability, and user-centered rather than developer-centered. However, it was not the case that everyone argued for the bottom. On #3, a student argued that the person on top was simply being honest, not really making excuses, while the person on the bottom was being political to the point of dishonesty.

In the end, it was a close game, with TOBT winning 5 to 4 over Los Dos. I think the students appreciated the interactivity and how everyone got a chance to speak their piece. When it was all over, I walked through the slides again to talk through some of the issues that came up, so I guess they didn't really get a reprieve from hearing me lecture to them. However, I think they felt like they had some skin in the game now, since they had already shared their thoughts about each issue.

Reflecting with the Fall 2018 Human-Computer Interaction Class

The truth is that I had been a bit down on my HCI class. I set up what I thought would be a wonderfully inspirational cooperation with the David Owsley Museum of Art, and I gave them the challenge of prototyping real solutions to some of DOMA's problems. As with my Spring 2018 HCI class, I wanted these solutions to be firmly grounded both in human-centered research methods and in the general design theories we studied in the first half of the semester by reading Don Norman's The Design of Everyday Things. However, as we moved through the stages of the project, I was not seeing teams doing either of these. In the first iteration reports that they submitted before Thanksgiving, it was obvious that their designs were not really rooted in their research, nor were their designs intentionally applying the principles we discussed. This left a heaviness in my heart, both because I doubted the efficacy of the class and because we were going to be showing our results to DOMA at the end of the semester.

Our primary contact at DOMA was their Director of Education, Tania Said, who has been a gracious and kind partner in this endeavor. Due to the schedule of docent training, we scheduled our final presentations for Tuesday of last week, which was the penultimate day of class. I was impressed by how well my students presented their work and the effort they had put into revisions in the 1.5 weeks since Thanksgiving. It helped that Ms. Said has a grace and wisdom that made her questions uplifting to the students: her questions were fair and honest, yet always supportive and encouraging.

I think we would all call this meeting a success, yet there was one more meeting yet in the semester—Thursday, December 6, which was also the deadline for their projects. I had originally planned to have them present their final projects to each other, but this clearly would have been redundant after the meeting on Tuesday. I decided that my goal for the meeting would be to try to wrap up some of the loose ends, to share with them some of my joys and frustrations from the semester, and to prime their reflections to prepare for the final exam. To this end, I decided to present them with three questions and an activity: What went well this semester? What did not go well this semester? What still puzzles you? The activity would be to write final exam questions.

We pushed all the tables against the walls so we could sit in something like a circle, doing the best that the room permits. I opened the class with a few remarks about my perspectives, and I got them into groups of three or four students to answer the first question. Here is a transcription of the notes from the board after we openly shared our results, slightly expanded from my blackboard shorthand:

• Project pride & satisfaction
• Presentation to Tania Said
• Getting class feedback
• Using the Click-Share system from our seats promotes discussion 
• Second round of paper prototyping helped clarify the difference between sketch and prototype
• Museum visits
• Connecting with a real place / real partner
• Two-week warm-up project provided a trial run before working with the real partner
• Relationship between design principles and Clean Code
• Reading Design of Everyday Things

It gave me satisfaction to see them bring up many of the items that I too thought were our greatest successes of the semester. Of course, we didn't vote or rank these, so some may only be important to one or two students, but that's fine for our purposes. One of the biggest surprises came from the comment about the second round of prototyping. This was a particular exercise where everyone dropped the ball, so we agreed to reschedule literally the rest of the semester so that they could do this homework exercise again. At least the student who spoke up, this gave her the opportunity to really understand it, which is infinitely more valuable than just plowing forward. I was also glad to see students reflect positively on the two-week project warm-up rather than frame it as wasted time and effort.

The next question to cover was, "What did not go well this semester?" I had them get up and move around the room to sit with different people for this conversation—something I had warned them I would do in my introductory remarks. For many of the items, I asked a follow-up questions, whether the conversation included ideas about how to address the things that didn't go well. I used two markers for this: black for the original item, and green for the suggestions. Again, my notes are below, with the green parts offset in square braces.

• Project reports
• Balancing the need for functionality against the practical application of design principles
• [Emphasize making high-fidelity executable paper prototypes]
• Justifying designs as meeting the principles rather than intentionally applying them
• Struggling against "just make it work" vs. a good report
• The former is enforced by department culture
• One commented that I am the only faculty who grades on anything besides whether the software "works"
• [Focus on design (look, feel, operation) over implementation]
• [More practice exercises to practice implementations]
• "Two-class" phenomenon, where first half of the class focuses on DOET and second half on the project, without a clear bridge.
• [Again, high-fidelity executable paper prototypes might help here]
• There were more conversations in the first half than the second
• [Sitting in a circle would have increased conversation quality]
• [Use an Interactive Learning Space for this class]
• Test-Driven Development
• Still confused about types of testing (unit, integration), test-friendly architectures, and how to test first.
• Time management and problem slicing
• Other faculty do not emphasize Agile slicing approaches
• Continuous communication with DOMA
• We had to work around their schedules, which held us back on our tight timeline
• The collection database we used was inconvenient

I will quickly add comments about those last two points. Regarding working around their schedules, I explained that this phenomenon was one of the reasons why the granddaddy agile methodology Extreme Programming calls for an on-site representative of the client, so that they are always there to answer questions and work alongside you. Regarding the database we had, I pointed out that our data was a smaller copy of a live system in use by the University Libraries. That is, it showed what a "real" database looks like, rather than a fabricated one for a class example. 

It was delightful for me to hear students sharing so candidly about their struggles during the semester. Again, they identified many of the elements that had been on my mind as well, but it was more powerful to hear it come from them. I was particularly pleased at their comments about how the architecture of the room was an impediment to our activity.

At this point, we only had about twenty minutes left in the meeting. I told them that I was cutting the third question but still wondered if there was anything that still puzzled them. One student posed a really interesting epistemological question: how do we know if Don Norman's rules are the right ones? This got us into a quick conversation about how his rules in DOET are like Robert Martin's rules in Clean Code: they are not universal rules, but they are frameworks that help us consider what the truth might look like. In retrospect, I should have spoken about shuhari here, but I was trying to push ahead and it didn't come to me in time.

I moved on to the last question, again shuffling the groups. Here are their ideas of what might be a good final exam question for the course.

• What does good design mean to you?
• What are five of the seven Universal Principles described in DOET?
• What are the four stages of the Double Diamond?
• What went well or what did not go well?
• If you were to take this class again, what would you do differently?
• Give examples and applications of the seven Universal Principles
• How would you change the organization of this course?
• Reflect on something valuable you learned and how you will use it in the future.
• Reflect on the outcomes of the course.

When the first item was shared, I asked how the student would assess responses. I think they didn't expect this question, but I assured them it was a good question, and that I was trying to understand the nature of it. A few students suggested that it would be enough to see if the hypothetical test-taker could answer it coherently, to show that they had thought about it. 

The second and third questions really surprised me. Those are so unlike the kinds of questions I give. I wish I had noted at the time whether these were students who had ever taken a class with me before, but I didn't. By contrast, that fourth item is exactly the kind of question I like to ask on a final exam, and the student who suggested it pointed to our notes on the other boards and said, basically, "Just ask those. Those are good questions." 

I had been feeling a bit down in the dumps on Thursday morning for a variety of reasons. The way I put it on Facebook was, "I feel emotionally dead inside, and now I have coffee. I can be apathetic FASTER." At the middle of the day, though, my Game Programming students showed their final projects, and most of them did a great job. That lifted my spirits, but after this meaningful, honest conversation with my HCI students, I felt almost giddy. Of course, maybe it was sleep deprivation, but I'll give them the benefit of the doubt. Maybe I let myself grumble too much about those students earlier in the semester when I was feeling uncertain. They were really a good group, maybe an uncommon mix, but they were paying attention and they were learning.

My original plan was that I would work with DOMA to choose a project they really wanted to see polished up, and then use that in my premiere graduate-level course on Software Engineering. As I recently reported, however, those plans have gone up in smoke. Instead, I will be teaching another section of Human-Computer Interaction. This gives me a chance right away to incorporate some changes, although it has to be done in the all-too-short winter break. But first, I better actually write my final exams. Thanks for reading!

The Paper Metaphor and the Brainwashing of Writers

I had a parenthetical phrase in my previous post that was about as long as the paragraph that contained it, so I decided to extract and reform it into its own post. It's something that's been on my mind the last few days in two of my classes, specifically game design and human-computer interaction. I'm using Google Docs in these classes, as I have done for years in many classes. Google Docs has some excellent affordances for learning, perhaps the most obvious being that student teams can collaboratively write in a convenient way. To me, however, this feature is secondary to the ability to highlight and comment on specific parts of a document and then to transform that comment into a conversation in the margins. If I see something interesting or confounding or insightful in a student submission, I can highlight it and leave a comment. The comment might be a question designed to make a point, an honest question of my own curiosity, a reference to relevant work or other student work, or really anything else I can express in text. Whoever wrote that section of the document gets a notification, and anyone who reads the document can join in this comment thread.

The problem is that my students are not responding to the comment threads. In fact, I believe that this entire semester so far, no student has responded to or even resolved my comments in any of their submissions. I paused to wonder why this was the case, and I came up with two answers. The first is the simple pedagogic answer that I had not incentivized them to do so. Students, like many of us, are busy—some are even busy with with their studies. If there is no incentive to respond to comments, then why bother?

When I teach CS222 (Advanced Programming), I often use a resubmission policy through which students can rework old assignments, learn from their mistakes, and resubmit for course credit. If a student resubmits something and they have not responded to my comments, they should expect me to simply kick it back to them. I believe that this policy is generally good for students, since they have a real incentive to learn from their mistakes, although it also has the negative consequence that some students submit substandard work knowing they can resubmit it later. That aside, the resubmission policy requires a lot of effort on my behalf: not only do I have to grade a submission more than once, I also have to try to understand and comment on the differences between the original and the revised submission. The burden on my time is one of the reasons I am not using a resubmission policy this semester.

I think there's something going on here besides just the incentive structure, however. Conventional educational practice involves students "turning in" work to the teacher, who then evaluates it, assigning a grade and giving some feedback. A student gets the paper back, looks over the comments, and then discards it. Online writing environments like Google Docs draw upon a conceptual model of writing on paper, in part because the legacy of text editors is often tied to the concept of printing onto paper. It's worth noting, however, that "plain text" editors make no such pretense. While it's possible ti know what "page" you're on when programming in your favorite programming environment, nobody does it. Concepts like "page" are purely metaphorical in a digital writing environment: there is not a "page" at all, not unless work is printed onto said page. Because rich text editors draw upon the conceptual model of paper, however, students get drawn into the same one: whether a student gives me a URL or a printed sheet, the culturally expected behavior is the same. This phenomena rose its head earlier this semester when I noticed how many students were putting hard page breaks into their Google Docs documents that are intended to collect all their work. This makes it tedious for me to scroll through their work, because to me, the conceptual model is "chronological log of work," but to them, the model appears to be "series of pages of work."

What I intend, when I leave comments into a student's document, is the conceptual model of conversation, not submitting paper to a teacher. Imagine sitting with a student who makes a claim such as, "I don't think Don Norman gives enough credit to the role of amateurs in his discussion of the future of design," and you say to him, "Why is that?" and then they simply walk away. Cultural constraints around conversations tell us that this is not only strange but rude. However, I have left, oh, let's say thirty questions in students' submissions already this semester, and they have all essentially got up and walked away. I think it's a mismatch of mental models: I think we're having a conversation, where they are in a transaction.

Unfortunately, it's not clear to me how to encourage students to use the feedback-as-conversation mental model without adding incentives such as resubmission. This means that once again, it breaks away from being an exchange of ideas and into a transaction around points. There's always the opportunity to turn it into an achievement in a course that uses it, although I'm not using many achievements this semester as I experiment with specifications grading instead.

My observations beg the question, "What would a digital writing environment look like that fosters the conversation rather than transaction mental model?" I hate to beat a dead horse, but I think this is exactly what Google Wave was getting at, and perhaps is one of the reasons why it didn't catch on. It was solving a problem that people didn't know they had, because they were locked into a different conceptual model of how writing and conversations manifest. For programmers, the answer is clear: digital writing looks like GitHub, which integrate writing and conversation along with version control and issue tracking. If GitHub supported commenting on text without needing a pull request, then perhaps using it and Markdown would be a viable alternative to Google Docs for the kind of learning environment I want to foster.