Initial thoughts on OpenMind in HCI

I am taking a moment this morning to capture a few thoughts about integrating the OpenMind platform into my HCI course. I wrote a bit about the initial decision in my post about the course re-redesign. Yesterday, I read my students' second batch of submissions, and I can say that incorporating this platform may have been one of my better decisions from the summer.

In the first week, the students completed the first module, and I had them write about whether they could identify the use of motivated reasoning or confirmation bias in past project work. Most were able to come up with good examples, a few did not, and a few did something much more interesting. These students responded first by saying they didn't think they saw either one, then they explained lucid and clear examples of using these. That is, even in writing about it, they tried to deny this as if it were shameful, but then admitted to having done it. In my feedback, I tried to point out that the important thing was to recognize it, not to feel bad or shameful about it.

This past week was even more interesting. They completed the second OpenMind module, this one on moral foundations theory. At the same time, they read Norman's Design of Everyday Things presentation on constraints, which includes a taxonomy of logical, semantic, cultural, and physical constraints. I asked them to consider how moral foundations theory influenced cultural constraints of design. I have to admit a certain pride in that question, and the students' responses by and large showed that it was both challenging and thought-provoking. A few students were only able to give very broad, abstract answers, essentially answering the question by restating, or trying to answer the easier question of whether it does rather than how it does. Here, it was an opportunity for me to remind them that human experience is almost all about abstractions and assumptions, while design is all about specifics. Indeed, that is a major theme of Design of Everyday Things' discussion of human behavior. Some of the students were able to come up with excellent insights that related the theories of moral foundations and design. Most noteworthy, however, were the few students who made generalizations about all people based on their own moral foundations. That is, they made claims that all peoples preferred safety or liberty, and I was able to push back on the distinction between how one sees the world versus how the world is.

They will be completing the remaining modules over the next three weeks, and I am eager to see how this trend continues. At the end, I want to talk to them about the holistic experience of completing the OpenMind modules, and maybe at the end of the semester try to come back to it and see if they think it impacted their designs.

Students' preference for discussion over prototyping, despite instruction to the contrary

Monday afternoon was a perplexing one, forcing me to look back at my goals and direction for a variety of reasons. What kicked it off was my one o'clock meeting with my HCI class. After Spring Break, we started in on our final project, and given their surprising reaction to our pre-break mini-project, I decided we would use the final project to gain a better understanding of the double diamond approach rather than try to introduce a different model. Briefly, before break, we did a quick run through the double diamond; in evaluating their results, it was clear that the vast majority of students did not invest the time to understand the context, let alone to identify a real problem. Indeed, what seemed to happen is that they chose to do something they could do rather than trying to solve a real problem. That shook me pretty hard—hard enough that I realized I couldn't let that be their broken understanding of the process.

We spent a week on the discovery phase, and I pushed them out into the field to talk to real human beings. Based on this, they had to make a few empathy maps and personas. They then interpreted these into journey maps, almost all of which were identical—not because of academic dishonesty but because of collective myopia. From this, they identified the problems they would work on, and that brings us to Monday: the first day of the "develop" phase, in which we would review and practice making low-fidelity prototypes.

I started by asking the class to list the tools of low-fidelity prototyping. They quickly came up with paper, markers/pens/crayons, and PowerPoint, along with other lightweight drawing tools not specifically designed for prototyping but certainly amenable to it. Their next answer was people, which surprised me but I think is appropriate. I added whiteboards, and I pointed out that one of the students had previously deployed a system specifically for UI prototyping (uxpressia by name, but that's just one example among many).

I created a second column, which—since I had sort of backed myself into a taxonomic corner—we called the "Anti-Tools" of low-fidelity prototyping. What sorts of things should we avoid? One student quickly mentioned code, which I agreed is exactly right: avoid code until it's the best tool. Another mentioned templates, which at first I didn't understand, but as he explained it, he was really talking about locking yourself in to particular approaches too early: a template is a reusable abstraction, but we don't know a priori that the given abstraction is appropriate. They paused here, and it was my chance to introduce two critical ideas; indeed, the primary reason for the exercise was for me to share these two points. I added brainstorming, which required me first to define the term—I am regularly frustrated by how students want to use this as a trendy synonym for "thinking." I briefly explained to them that brainstorming in a group will tend to push early convergence rather than divergence: that the best approach was to just start making. The second one I put up was related: analysis paralysis and discussion. A kissing cousin of brainstorming, I explained that I have seen practically every team I have mentored fall into this trap, thinking that sitting and talking about a problem will help us solve it. It won't. Primed by these observations, I returned to the positive column and suggested that timeboxing is one of the greatest tools of creative prototyping.

With that, they voted on a 15-minute timebox, I set the timer, and they got to work.

Or something that looked like work to them, anyway.

There was one group whose only discussion was about distributing index cards—I had told them a low-fi prototyping exercise was coming up, and so a few people brought supplies—and then they set to work, cutting, drawing, crumpling. Another group did a brief powwow before going in what appeared to be a similar direction, although that might be up for interpretation. The rest of the class, roughly 70%, engaged in discussion. One group took to the whiteboard to draw something like a flowchart, the rest sat in their clusters and discussed while they drew.

I observed all of this happening, of course, and as the timer kept ticking, I kept thinking, "Any moment now they will break and start actually prototyping, right?" At about ten minutes in, it was clear that this was not going to happen, so I wrote two questions on the front center board: What makes it a prototype? and What makes it a good prototype?

When the timer went off, I invited them to look at these questions. Honestly, I wasn't very hopeful in getting good answers, based on what I saw, but we went to the board anyway. In answer to the first, a student (from that first group) said, "It's testable." That's exactly right: a prototype has to be testable, otherwise it is something else. I pointed out a secondary part of the definition is that the prototype points to a possible future. I couldn't think of the word for this, but a student suggested, "portent." In retrospect, the word is "portentous," but still, that's a 10-cent word. We moved to the second question, and right away a student (from a different group) said, "It gives you good feedback." That's the right idea here too, in my opinion, although I tell you what I told them, that I prefer to frame it as, "It answers a design question."

This is interesting, isn't it? They seem to understand the theory. I asked them, then, how many of them had prototypes from the 15-minute timeboxed exercise? The students in that first group, they all raised their hands, and rightfully so. The only other hand that went up was from one student in a group of three. I asked them to dive into this: they had all worked together, on one artifact, in discussion, during the timebox, and only one of them characterized it as a prototype. I invited them to describe their disagreement, and one of them attempted to justify that what they made was a prototype because it had arrows and indications explaining to someone how it would work. It was clear to me, and I think to the rest of the class, that this was not a prototype at all, but some kind of sketch or schematic.

My next question to them was, "What did you notice that was different about how the groups worked?" They all seemed to recognize that the people who came up with prototypes used their fifteen minutes to create their prototypes, while everyone else engaged in discussion. I explained to them again how this phenomenon was something I had seen many times before, particularly on immersive learning teams, where I advise working on prototypes and instead, students talk to each other—at length, with no real output.

I ended class, then, with a challenge: first, that they actually follow the instructions I give; second, that perhaps they consider breaking their teams in half, with half doing timeboxed prototyping and half focusing on brainstorming and discussion, and compare the results at the end. Honestly, I would rather they do the first, but I'd be satisfied if they did the second.

As this post was bouncing around my head this morning, I realized that I have seen a similar phenomenon before, regularly, in my teaching. It is the phenomenon of CS222, where I tell the students that they need to start with CRC cards, then make a list of tasks, then use test-driven development to approach those tasks. From many years of teaching the course, I my best estimate of what actually happens is that teams get together, talk a bit, and then start programming. This comes, in part, from student essays admitting to it. This inevitably leads to failure of the two-week project or the first iteration, and I provide vociferous feedback about what is wrong and how to improve. The "how to improve" is, essentially, to follow the steps. Yet, students don't. Even to the third iteration, I regularly have 20-40% of teams who are still not following the steps that I have laid out and that they know they will be evaluated on. Whether or not they ever read the requirements is moot here, since they all look at the feedback I provided and claim to be seeking to improve; yet, if we judge intention by results, the real intention seems to be maintain the status quo rather than learn something new.

I am left with a burning question about how to push my role as a mentor. Should I be interrupting their 15-minute timebox to point out what I see, in the hopes of pushing them more quickly in the right direction, or do I need to let them make this little mistake so that they can learn from it? I am afraid of them treating me like an exam proctor: if only the proctor weren't here, we could just collude on the exam and get out of here. I have been thinking about how this applies in CS222 as well, and I have been thinking about having more formal check-in points; for example, if teams had to turn in their CRC cards two days into an iteration, it would show them that I mean business. However, it would also mean that they would do it because I was collecting it and not because they should in order to learn what is being discussed.

My next meeting with the HCI class is coming right up. I think I may begin class by asking them to share the processes they used to construct their prototypes, and perhaps I will push them a bit further into a root cause analysis, to consider why they didn't follow the instructions they were given.

Empathy, Listening, and Hearing

I have more stories I want to share here than I have made time to share them, but here's one that I find my idle mind keep returning to. I want to capture it here to make sure I don't forget it.

We are reading Design of Everyday Things in my HCI class (CS345/545), and several meetings ago, the students read Norman's presentation of the UK Design Council's double diamond model for design. After some thinking and Googling, I developed a series of in-class exercises to help students understand the model. Particularly helpful was Heffernan's overview of activities associated with various stages of design. I decided that an exercise on empathy mapping might be just the right way to start.

Of course, to build empathy, you need a focus and a context. I decided to use, as a running example, students' experience dealing with the triage grading system that I use. This is a brilliant grading system that I learned from William Rapaport at University at Buffalo when I worked as a TA with him. It is coherent, philosophically-sound, and unfamiliar to almost everybody. I get the occasional question about it and the student whinging in course evaluations, but by and large, student experience with it is unrecorded: it happens in the shadows or in passing conversations. It's also something that all of my current HCI students are experiencing, and practically all of them have either experienced it before in one of my other classes or know someone who has.

I introduced the context in class and asked students to work in small groups to develop a map, writing them on the classroom whiteboards. We used the conventional map labels as shown in Heffernan's overview: what do students think & feel, hear, see, and say & do in their experience with triage grading.

It's helpful to have just a passing understanding of triage grading before we move on. Whereas conventional grading is based on percentage correct, triage grading is based on discretely measured quanta. For example, if you assume that 90% is an A, then as a test designer, you would design the exercise so that A-level work is attained by completing 90% of the prompts correctly. Notice that this is the tail wagging the dog: the fact that you are using conventional grading determines your test structure. With triage grading, any given item is scored out of three points: essentially correct (3 points), essentially incorrect (1 point), or somewhere in between (2 points). The letter grade is determined by weighted linear interpolation across scores, assuming that "A" means correct, "D" means incorrect, and "C" means middling.

Almost every group wrote down that they see percentages when they first encounter triage grading. That is, they see "1/3 points" as 33%, which they interpret as "Low Failing Grade" even though in triage grading it is a low passing grade. (You might consider 33% in triage grading as having a qualitative interpretation like 65% in conventional grading, right on the border of poor and failing.) There was broad consensus about this in the class.

I pointed out that the maps appeared to come from initial experiences with triage grading, but one of my bright students—who has taken my classes before—noted that his was more of a mid-semester view. He had recorded in his map that he became able to see the feedback as qualitative, as coarse-grained values that drove him to change his behaviors. I do not remember exactly the words he wrote on his empathy map, and indeed I didn't understand what he meant by the words he chose, but our conversation came back to the concept "seeing quanta" rather than "seeing percentages."

That was pretty interesting in itself, but here's where it kicks up a notch. A student in the back chimed in, saying essentially, "But it's still a percent." The first student acknowledged that mathematically it was, but that's not what he saw, and the one in the back insisted more strongly, essentially, "But it's some number of points out of a total, and so it's a percent, and so you're still seeing it as a percent."

Wow! What a teachable moment for empathy! I pointed out, treading carefully, that this was an example of the second student showing no empathy for the first student. The second student saw the world in his way, insisted that it was the right way, and that everyone else must also see it that way. I introduced the idea that whether or not there is an objective reality, perception drives a person's lived reality, and perception is subjective. Two people can look at the same thing and "see" two completely different things. The expression on the second student's face told me that he understood what I was saying, but he was still working on the implication; of course, maybe I observed this wrong.

We are continuing to work with this example, and I am finding it a rich context for discussion. I hope to share a few more stories here on the blog, but for now, it's time to head off to class. Thanks for reading!

Metacognitive learning through a design thinking framework

I have written a few times about the design thinking model that I use, lightly adapted from George Kembel's:

Brian McNely and I wrote a paper about how we observed a team falling into a failure mode of looping like this:

That experience inspired me to start using this model as an exercise in CS222. Around the beginning of the third three-week iteration on the final project, I take a day where I begin by introducing this model. Then, I ask the teams to annotate their paths through the phases, starting with their project pitch, and annotating major milestones such as the ends of iterations. This is a great metacognitive exercise that gets students thinking about how they have been proceeding. Inevitably, I get a student who reports that they did it "wrong" by starting in somewhere other than empathy or by jumping between phases, but then I explain that we're not using this in a prescriptive way, but as a tool to help us think about our processes.

In this semester's CS222 class, we did this exercise last week Friday, which was the first week of the final iteration. This is the only photograph I have of the event, but rest assured that this is representative of the kinds of diagrams the students generate.

I usually tell the story about the study that McNely and I did, pointing out that sometimes we software development types get stuck in "ideate-build-test" loops without stepping back and remembering for whom we are building this thing. The arcs in students diagrams tend to have a similar shape, often skipping over the empathy step entirely (which is understandable since it's not really connected to our course learning outcomes).

On Friday, though, I saw something I had never seen before. The last team to present their models showed that they had a strong arc like this:

That is, they observed that they identified the real problems within a specific demographic, but they never actually decided which of those problems they would solve, and how they would do so. Instead, they went right from identifying the problems to programming. Having looked at their code in two formal evaluations and some information evaluations, I think this is astute: it perfectly describes the failure mode that they fell into. The team is pulling itself together in this last iteration, and I hope that this exercise is formative to that process.

As I was talking to my class about this model, I had a moment of inspiration where I realized there may be disciplinary differences in how one gets trapped in subcycles. This is only a hypothesis at this point, but I think I've seen this before in my multidisciplinary classes:

I see programmers fall into the trap where they think of something, build it, think of something else, build it, and so on, without ever meaningfully evaluating it or really returning to the user's needs. The humanists--such as English and History majors--tend to think about the problems that exist and think of solutions, then think of other problems, then think of solutions, and so on, without actually building anything to validate their ideas. I wonder if teaching this model early would help students to frame their processes in a useful way and help them catch themselves unproductive spiraling.

Revising Courses, Part III: Advanced Programming

Inserting reflective essays into my game programming course was straightforward, and the augmented achievement-based grading system of my game design course was an incremental improvement. Now, I will share the story of the biggest renovation in my Summer of Course Revision: complete renovation of my Advanced Programming course.

CS222: Advanced Programming has been around for about four years now, and I have taught it more often than any other. Several posts on my blog reflect on this course, including this one from the first semester's experience and this one reflecting on this past academic year. For the past several offerings, I have used the same fundamental course structure: studio-based learning, using daily or weekly writing assignments for the first several weeks, gradually turning toward a focus on a pair-programmed two-week project and then a small-team six-week project. I have been basically happy with the structure, but reflecting on my experiences, I identified the following pain points:

• The course was sequenced along a particular path of learning that did not resonate with all the students.
• There was not enough mastery learning: students would often do poorly on an assignment and clearly not revisit it, since weeks later, they still didn't understand the concept. This caused especial pain when these assignments covered technology that was core to my pedagogy such as distributed version control.
• It was easy to slip into a mode where I would talk for the whole class meeting. The studio orientation—through which students show artifacts that represent their learning, and these artifacts are subject to formal and informal peer and expert critique—was not guaranteed in my course structure: it relied on daily ingenuity rather than codified form. (The time and space constraints imposed by the university negatively affect studio orientation as well, but these are beyond my control.)
• It was not clear that all the team members were engaged in reflective practice when working on their projects. That is, I felt that there were people falling in the cracks, not learning adequately from the team projects, sometimes dues to lack of reflection and sometimes due to lack of practice.
• Students enter the course with high variance in knowledge, skill, experience, and motivation.

In addressing these pain points, I wanted to make sure I kept all the good parts of the course. Students work in teams on projects of their own design, using industrial-strength tools and techniques. Teams have to incorporate some kind of management techniques and give multiple milestone presentations, both of which remind students about the "soft skills" of Computer Science. There is a good overall flavor of agile development, pragmatism, object orientation with patterns, and the importance of reflective practice and lifetime learning.

If you've read my last two posts, you probably see where this is going! I decided to replace the daily and weekly assignments with a system of course achievements (a.k.a. badges), and students will write reflections that relate the achievements to essential questions of the course. The complete course description is available online, or you can choose to view just the achievements.

I have developed the following set of essential questions:

• What does it mean to be a "professional" in software development?
• How do you know when a feature is done?
• How do small teams of developers coordinate activity?
• How does a Computer Science professional use ubiquitous and chaotic information to be a lifetime learner?

As in my game design course revision, students' grades will be based on number of achievements earned and written reflections on those experiences. There are also four "meta-achievements" that can be earned by completing sets of regular achievements. These are leveled achievements and reflect four different potential paths of mastery: a Clean Coder has documented evidence of applying ideas from each of the first twelve chapters of the book; a White-Collar has demonstrated savvy at project management and presentation; an Engineer is moving toward an understanding of software architecture and patterns; and a User-Centered Designer has designed, evaluated, and iterated on user-facing systems. The introduction of these meta-achievements was partially inspired by an alumnus who, when I told him about this redesign, said that most courses were like a Call of Duty game, but that this was more like Deus Ex, and that it would benefit from having more quests.

I want to help students succeed in this kind of learning environment, and so the first month is designed to help them understand what it means for them to have more ownership over course activity. In the first week, I will provide a review of important CS1 and CS2 topics, focusing on the mechanics of object-oriented programming in Java. Knowing that student experience and comfort with this material is highly variable, I can use this week to focus on how to navigate the course structure. In fact, I will strongly recommend that their first achievement is Studious, which requires them to read William Rapaport's excellent How to Study guide and write a study plan for the semester.

We will still use Clean Code as a shared focus, but I have changed how I am scaffolding students' experience in integrating its expert tips. Previously, I had identified specific tips or chapters for students to read and then asked them to apply these tips to their projects—past or current. There was little evidence that this "stuck" with the students, as later coursework would violate the concepts supposedly learned. This was, in part, due to lack of mastery learning, where students would accept a low grade and move on without having learned the material. Furthermore, because we paced Clean Code readings and assignments through the semester, I would frequently encounter examples in class and in critiques that afforded application of a tip that the students had not yet studied. In the revised course, we will read the book relatively quickly, but then keep returning to it in informal expert critiques, adopting an iterative approach that is more apropos to how one learns and remembers these tips: a bit at a time, and a bit more next time.

In my game design course, students are required to present to the class to earn their achievements, but the enrollment in that course is half that of this one. I toyed with the idea of having my CS222 students post their work on the walls, a portion of the class each day, but I feared that this would have too many negative consequences. Instead, I am requiring students to post their artifacts to a wiki, and my intention is to review the wiki changes between class meetings to identify notable entries. This way, I can bring up the wiki on the projector and model an appropriate critical process. I can also insert mini-lectures as necessary to clarify misunderstandings. I am eager (or perhaps anxious) to see how the density of misunderstandings in this redesigned course, which encourages mastery learning through reflection, compares to that of my current status quo, which encourages correctness up front. We'll be using BlackBoard's wiki only because it's easy for all the students to find; it was frustrating to me during my experimentation that it has no wikitext editor. If the wiki turns out not to work for us, we can always move to an in-person poster-style presentation.

Since I will be doing more just-in-time teaching—reacting to students' insights and confusions—I have decided to expand the conventional six-week, two-milestone project to three milestones over about eight weeks. The students like this part of class the most, and I like the idea of adding another milestone, since this gives them another opportunity to learn from mistakes in tools, design, organizational structures, and presentation. For both this large project and the small project, I will provide common requirements that everyone must follow, such as using distributed version control. Previously, these things were simply worth some points on the project; however, if I want to focus on assessing their reflections while also requiring some shared technological experience, it's fairly easily done with a hard-and-fast requirement. With multiple milestones, there will be an opportunity to ensure each team is following the requirements. For example, if I see a team not using DVCS, then I can remind them.

I am eager to see how students take these changes. I expect to be able to blog about some of the day-to-day experiences, and I anticipate writing some academic papers on aggregate student performance in all these modified classes.

Here are the critical links again:

• CS222 Course Description
• CS222 Achievement List

Revising Courses, Part I: Game Programming

I spent the lion's share of the last two weeks revising my three courses for the Fall semester. They are the same courses as last time, although some of the themes have changed. After a trepidatious beginning, I am now quite pleased with the results. In today's post, I will describe the revision to my game programming course, an elective for Computer Science undergraduate and graduate students. The actual change to the course may appear small, but it represents a significant amount of research and learning on my part.

I have been structuring this course as a project-intensive, team-oriented experience. For example, last Fall the students implemented The Underground Railroad in the Ohio River Valley. I have also used this course to experiment with various methods of grading. I wanted the grading to be as authentic to the work as possible: students are evaluated on their participation and commitment only, not on quizzes or exams. For example, instead of midterm exams, I held formal face-to-face evaluations with each team member, modeled after industrial practice.

These methods work well, but reflecting on these experiences, I identified two potential problems. First, these methods fail in the case that a student refuses to participate or keep commitments: in particular, these methods produce little that could be considered evidence in the case of an appeal. Realistically, sometimes I get a bad apple, and so I want a grading system that allows me to give the grade I feel is earned. Note that while I admit to having given grades that are higher than I thought were earned, the assessment failure may be twofold: some students may require more concrete evidence of their own progress in order to improve or maintain performance, especially if such students lack intrinsic motivation.

The other potential problem stems from my wanting the students to engage in reflective practice, not just authentic practice. I wonder if some of my high-achieving team members have gotten through these production-oriented courses without having deeply considered what they learned. My model for encouraging reflective practice is based on industrial practice—agile retrospectives in particular—and is documented in my 2013 SIGCSE paper. This model, called periodic retrospective assessment, requires a team to reflect on its successes and failures intermittently during the semester, and at the end of the semester, to reflect on what it has learned. This sociocultural approach to assessment is appealing, and again, it seems to work in many cases, although it affords scant individual feedback.

While at this summer's GLS conference, I attended a talk about game-inspired assessment techniques given by Daniel Hickey. His model is called participatory assessment, and a particular aspect of it—which you can read about on his blog—is that it encourages evaluating reflections rather than artifacts. During his talk, he made a bold claim that resonated with me: writing is the 21st century skill. After having worked with Brian McNely for the last few years, I have come to understand “writing” in a more deep and nuanced way. (See, for example, our SIGDOC paper that takes an activity theoretic approach to understanding the writing practices involved in an agile game development team.)

Putting these pieces together, I decided to keep the fundamental structure of my Fall game programming course: students will work in one or more teams, organized around principles of agile software development, to create original games in multiple iterations. We will continue to use periodic retrospective assessment in order to improve our team practice and consider what we learned as a community. Now, I have also added individual writing assignments, to be completed at the end of each iteration. I want these reflections to be guided toward fruitful ends, and so I have brought in another pedagogic element that has intrigued me for the last several months: essential questions.

I first encountered essential questions (EQs) on Grant Wiggins' blog, and I blogged about this experience in the context of my advanced programming course. The primary purpose of EQs is to frame a learning experience. EQs have no trite or simple answers, and they are not learning outcomes, but they inform the identification and assessment of learning outcomes. With a bit of crowdsourcing, I came up with the following essential questions for my game programming course:

• How does the nature of game design impact the practices of game programming?
• How does game software manage assets and resources effectively?
• How do you coordinate interpersonal and intrapersonal activity within a game development team?

In reading about participatory assessment and the badges-for-learning movement, I came across Karen Jeffrey's HASTAC blog post. What she called “Really Big Ideas” seem isomorphic to EQs, and so I adapted her ideas in defining a rubric for evaluating reflections. I will be looking for reflections that provide the following:

• A characterization, with supporting evidence, of one or more essential questions of the course.
• The consequences of this characterization on individual and/or collective practice.
• The potential critiques of this characterization.

Deciding how to guide student attention is one of the most challenging parts of course design, and I recognize that by introducing these essays, I am reducing the number of hours I can expect students to spend on the development tasks at hand. However, these essays will afford conversation and intervention regarding reflective practice. They respect the authenticity of student work since, if done right, they should yield increased understanding and productivity from the students. This reasoning is similar to that given my proponents of team retrospective meetings as part of an agile practice: by reflecting on what we are doing, we can learn how to do it better. I have been encouraging my students to write reflectively, especially since starting my own blog; these reflective essays codify the practice and reward student participation.

The official course description for Fall's game programming course can be found at http://www.cs.bsu.edu/homepages/pvg/courses/cs315Fa13. I am happy to receive feedback on the course design, particularly the articulation of the essential questions, since they will be central to the students' learning experience.

Next time, I will write about the redesign of my advanced programming and game design courses, both of which involve turning to badges to incentivize and reward student activity.

Rearranging RB104

My Advanced Programming (CS222) class meets in RB104, which is an unremarkable classroom, as evidenced by the following photos.

Student view from the corner

Instructor view facing side

Instructor view facing back

On Monday, I told the students that they had five minutes to rearrange the room to better suit our needs. We have been working together for over ten weeks, and they know what kind of thing I like to do. I suggested that they begin by explicitly identifying their design constraints, then moving into implementing a plan. I offered that after five minutes, they could get another five by simple majority vote, but only once. Then, I left the room.

After five minutes, I poked my head into the room and asked if they needed more time. I was greeted by a clever little barricade built from an overturned table, but it did not stop me from seeing that they were asking for another five minutes.

Another five minutes passed, and then I made my official return. The first thing I noticed was that most of the chairs were pushed into the back of the room, and in their place, the students had constructed four circular arrangements of desks.

View from instructor's station

Detail of center group

One of the clusters had no one sitting at it, and it was the only one with a trash can in the middle.

View from front center, looking left

The next thing I focused on was the odd construction at the front center of the room. Notice the eraser inuksuk, which was just completed as I entered.

Front center

I have a grad student, Jason, who is sitting in on the class. He always sits in the back center of the room, behind the last row of chair-desks. The students pointed out that they had left his chair accessible to him. I pointed out that it looked a bit like the kind of fort my boys would build in the family room:

Rear center

With this revised arrangement, I proceeded with a fairly normal class. First, I brought up the rubric I used to evaluate their first milestone submission on our six-week project, and I discussed the trends that I saw in the submissions. Then, I led a discussion of Alistair Cockburn's articulation of Human Success and Failure Modes, which I have written about before. I distributed a handout that summarized the key points and then explained and interpreted them. Then, I asked the students to get into small groups with people not on their project teams to discuss how they have witnessed these success and failure modes. This kind of structure—my introducing an idea and then asking them to discuss it in small groups—is fairly standard for the class.

I will share a few of my observations of the students' room design. First, because the students were facing each other, about half the class had a physically awkward time listening to me: they had to twist awkwardly in their seats, and in some cases, they simply kept their heads bowed and faced away from me. Second, many students could not see the whole projected screen, where I was showing the project rubric: the sculpture at the front of the room obscured their view, as the shadows in the photograph above demonstrate. Third, and perhaps most difficult for the students to observe, I didn't have a table to sit on, and this made me uncomfortable. When I am explaining concepts to the students and not using the projector—as with the Human Success and Failure Modes discussion—I like to sit on the table. It puts me just a little higher than them so they can all see me, and I can be in the middle of the room and talk to all sides with an informal air. Without that, I found myself stuck at the teacher's station, where I could set down and pick up my coffee cup easily. I felt like I was talking directly to the circle of students before me, and less so to those further away. Finally, when I did move around the room, I noticed how much easier it was to move from group to group compared to having twenty unused chair-desks in the way.

Toward the end of the meeting, I asked the students to individually write evaluations of their design. Most of them mentioned foregrounding the small group work, knowing that it was something they did regularly. As one student put it, "it's much easier to have a discussion in a cluster formation already facing your group." Several of them pointed out something I had not noticed: they had purposefully positioned the clusters next to the chalkboards, knowing that I regularly have them write notes and diagrams on the board. The only individuals to write about how the "art" at the front of the room blocked the projector were those in the group closest to the instructor station—those whose view was actually blocked. It makes me wonder if the other students didn't notice or if they simply didn't find this worth writing.

I find it interesting that they created four clusters of desks and also rationalized these as being for group discussion. There were six teams created for the six-week project, and since these groups were formed, we have used these teams as discussion and activity groups during class meetings. However, some of the groups have very poor attendance, and so their one or two attending students tend to get swept into another group for the sake of the discussion. So, frequently, we have three or four discussion groups even though there are six teams. The students did not comment on this phenomenon in their reflections, but their design reveals a focus on what actually happens in class, not what could happen if everyone showed up.

One of the students explained that the group of chairs around the trash can provided a "symbolic receptacle for bad ideas." While no one was sitting at this cluster originally, it was used once I had them split into interteam groups for discussion. 

At the end of the meeting, I asked the students to help rearrange the room into its conventional form for the next class. Someone pointed out the last alteration, which I had not noticed. Several students commented on a mix of serious and silly design ideas, and I'm glad to see that they weren't afraid to get their hands dirty.

Side Board

This was my first time using this exercise in class, but I plan to use it again. It was a manageable design and evaluation problem. In fact, it worked better than some of the paper prototyping exercises I have used, which I attribute to the physicality of the classroom space and concreteness of the problem: it's a real design problem that we face three times a week, Monday-Wednesday-Friday at 8AM. This exercise also gave the students an opportunity to control their environment, to exercise agency in a setting that is too often formal and stuffy. Finally, it was memorable—when else do they get to create eraser art in a Computer Science classroom?—which means it is something I can draw upon later when talking about design, evaluation, and what we learned this semester.

The Design Thinking Game

This morning in my game programming class, we started a short unit on game design. I brought in some prototyping materials and gave the students ten minutes to whip up a board game in the roll-and-move genre. This was inspired by Ian Schreiber's Game Design Concepts Level 1 activity and its primary purpose is to get students over the invisible barrier, "what if I cannot make a game?"

To explain what "design" is, I had started with a brief presentation of design thinking. Once I got the students started with the activity, I decided to sketch out a game as well. Glancing at the chalkboard, it struck me that the design thinking cycle could be modeled as a board game, so I made this:

The captions on the boxes are:

• Empathy: an audience
• Identify: a problem
• Ideate: a solution
• Build: a prototype
• Test: a strategy

Each player starts at Empathy. Each turn, you roll a die and move that many spaces along the cycle. Grab a blank card and write down something that happens in that phase, using the caption to guide you. For example, my first turn, I rolled a three, and ended up on Build, so I made a build card, "Sketch." Then I rolled a two, putting me on Empathy, so I made an empathy card called, "Elderly." As soon as there is at least one card for each category, that player takes one card at random from each category and has to make narrative that explains how these five things fit together. I didn't get far enough in my own design process to decide on an initial scoring or elimination process, but I imagined that the other players would somehow score the story, and there would be a time or point threshold to trigger the end of the game.

That's it. A ten-minute game, most obviously inspired by design thinking, 1000 Blank White Cards, and Dixit. Is it fun? No, not really. You fly around the board and are forced to invent artifacts separated from any context, so you can only be very generic. Is it good? No, not really. It completely misses the point of design thinking, that you have to go through each phase to get to the next. So, why write it up here? It was actually a fun thought experiment. I used this ten-minute design exercise at the AIM conference a week ago, but I did not actually participate, I just observed. It was much more fun to actually get in and do it. I also enabled me then to present my flawed game design first, demonstrating the importance of metacognition, specifically learning from the process rather than judging the artifact too harshly.

Brontosaurus and BDUF

My 4-year-old son loves dinosaurs. He draws them, he pretends to be them, we read about them, we tell stories about them, and today we even we ate pancakes shaped like them. Many contemporary children's books on dinosaurs include types I'm sure I never saw as a child, such as carcharodontosaurus, deinonychus, and one of my favorites, therizinosaurus. My son could also tell you—since it comes up a lot—that there really is not such a thing as a brontosaurus. Turns out, what was called "brontosaurus" was really an apatosaurus, and since the latter name had historic precedent, it was the one that stuck.

Yet, in our family furor for all things dinosaur, we frequently find references to the brontosaurus. My first reaction to such occurrences is usually, "Why don't they know better?" A good follow-up question is, "How would they?" That is, why would the designer of, say, dinosaur-shaped pasta even think to investigate whether or not there really is such a thing as a brontosaurus? Second-order ignorance remains the malady of the ignorant.

My son's love of drawing recently blossomed, and it was initiated by a visit from my father—who, I should mention, is an excellent artist. My dad got my son into drawing, unlocking what must be an artistic skill and passion that skips generations. However, every time my dad drew a tyrannosaur, it was standing upright with its tail on the ground. These were fine drawings in their own right, but they also happened to be wrong. How would my father know that his archetypal dinosaur's posture is scientifically passé?

I was idly considering dinosaurs this morning when an analogy popped into my head: teaching children about brontosaurus is like teaching novice software developers to do Big Design Up Front (BDUF). For years, the status quo assumed software development was like manufacturing, and hence one can measure productivity in man-hours and lines-of-code. As long as you design it right from the beginning, all will go well. Of course, as far back as the 1970's, Fred Brooks was predicting agile methods. Despite the popularity of his essays, I have seen little evidence of their impact (q.v. the CHAOS report) until relatively recently, as "agile" is less frequently used as a profanity. From a modern perspective, we know that software development is the process of learning how to develop that software. By necessity, you know the most about a system when you have finished building it. This is why you always feel like you could do it better if you could do it again, or why you should always build one to throw away.

I look forward to a day when software engineers take a page out of modern children's dinosaur books: any time BDUF is mentioned, there's a an explanation that well-meaning people used to think it was a good idea, but now we realize that its really just a shadow of iterative and incremental design processes. The positive side is that I don't foresee BDUF-shaped pasta ever being included in Software Engineer Mac & Cheese.

Sandwich

When I conceptualize a sandwich, I generally think of it the same way that I perceive it, combined with memory of the flavors, textures, and emotional connotations. However, when I design a sandwich, I approach the problem from the inside out.

What is the feature of the sandwich? Roast beef. The point of this whole sandwich is that it is a roast beef delivery system. Now that I have decided on the feature with the most value, I can approach the rest of the design challenge as one of finding the best complements to the feature.

Cheese? Swiss. Roast beef and swiss just sounds right.

Veggies? Of course! Tomatoes, lettuce, some bell peppers, and for salty complement, a pickle on the side.

Condiments... horseradish sauce. Mmmm.

What's the best way to wrap this up? The bread is a key component, but it is a servant to the rest of the sandwich experience. Sure, sometimes the bread is the feature, like when my wife makes peanut butter and banana sandwiches on graham bread, but not this time.

Regardless of whether I was ordering this sandwich or building it myself, I would have to consider these ingredients in a different order. As a sandwich builder, I need to select my bread first, then spread the condiments, then stack up the fillings.

When you go to The Atrium at BSU and go to the deli sandwich counter, they have a touchscreen display for selecting what you want. It's screen-oriented, with the first (pertinent) screen showing breads, then meats, cheeses, vegetables, and condiments.

This sequence of screens is builder-centric. The order of selection is exactly the order that the staff behind the counter will use to build the sandwich. From each screen, you cannot see the options on the other screens, because they are not pertinent to the assembly-line approach to creating the sandwich.

The touch screen interface is less user-friendly than the paper-based system that preceded it. Previously, any number of clients could grab a sheet, check the meats, cheeses, breads, etc. that they want, put it in the bin, and wait for the sandwich. Now, you have to wait in line behind the single touchscreen station. I find that this pressure adds to my frustration, but I usually approach the counter without knowing what I want. That is, I use the sandwich design experience to explore both the design space and my own desires.

Also, they forgot my horseradish sauce.