SIGCSE 2018 Talk: Design and evaluation of an undergraduate course on software development practices

For many years I have been saying to myself, "I should write a paper about CS222." Well, I did, and I just finished giving the oral presentation at SIGCSE 2018. Check it out, standing room only!

(The empty seat in the front row is my own.)

I was the third speaker in a post-lunch session, and I enjoyed the other two talks. Nicole Herbert from Tanzania was especially interesting, as she was describing a capstone structure very much like where I think we could take ours at Ball State with a bit of curricular and potentially administrative revision.

My paper can be found on the ACM Digital Library. I think the talk was well received, and a few attendees asked for a copy of my slides. I prepared all the slides using Google Slides—which I've never done before—andthat makes it very easy to generate a public link. You can find all the slides here.

Thanks to all the attendees, especially to those who came up to ask questions and share stories afterward. I'm truly heartened to hear how much this message resonates with you. If you are new to the blog and want to read more about the Advanced Programming course, search for the "cs222" label. My most recent post on the topic is in my "What we learned" series, in which I share the outcome of the somewhat unconventional final exam format that I use.

You can find the most recent course site, with assignments and evaluation schemes, at https://www.cs.bsu.edu/homepages/pvg/courses/cs222Fa17. I actually have a break from teaching the course this semester—the first such break in many years!

My CS222 YouTube channel is publicly available. It is a collection of tips and tutorials. Some videos are extensions of in-class activities, particularly the earlier ones; I've tried to transition to more encapsulated presentations for improved modularity.

This is a quick after-presentation post. I will try to share some more about the talk later, but I wanted to get this up for any visitors who come looking for the slides and links. I am happy to share my course materials, instructional videos, and reflections. If you do end up using them in your courses and designs, I'd love to know about it. Consider the material something like CC BY-NC-SA.

If you came for the miniature painting, search for the "painting" label instead :) Cheers!

Empathy, Listening, and Hearing

I have more stories I want to share here than I have made time to share them, but here's one that I find my idle mind keep returning to. I want to capture it here to make sure I don't forget it.

We are reading Design of Everyday Things in my HCI class (CS345/545), and several meetings ago, the students read Norman's presentation of the UK Design Council's double diamond model for design. After some thinking and Googling, I developed a series of in-class exercises to help students understand the model. Particularly helpful was Heffernan's overview of activities associated with various stages of design. I decided that an exercise on empathy mapping might be just the right way to start.

Of course, to build empathy, you need a focus and a context. I decided to use, as a running example, students' experience dealing with the triage grading system that I use. This is a brilliant grading system that I learned from William Rapaport at University at Buffalo when I worked as a TA with him. It is coherent, philosophically-sound, and unfamiliar to almost everybody. I get the occasional question about it and the student whinging in course evaluations, but by and large, student experience with it is unrecorded: it happens in the shadows or in passing conversations. It's also something that all of my current HCI students are experiencing, and practically all of them have either experienced it before in one of my other classes or know someone who has.

I introduced the context in class and asked students to work in small groups to develop a map, writing them on the classroom whiteboards. We used the conventional map labels as shown in Heffernan's overview: what do students think & feel, hear, see, and say & do in their experience with triage grading.

It's helpful to have just a passing understanding of triage grading before we move on. Whereas conventional grading is based on percentage correct, triage grading is based on discretely measured quanta. For example, if you assume that 90% is an A, then as a test designer, you would design the exercise so that A-level work is attained by completing 90% of the prompts correctly. Notice that this is the tail wagging the dog: the fact that you are using conventional grading determines your test structure. With triage grading, any given item is scored out of three points: essentially correct (3 points), essentially incorrect (1 point), or somewhere in between (2 points). The letter grade is determined by weighted linear interpolation across scores, assuming that "A" means correct, "D" means incorrect, and "C" means middling.

Almost every group wrote down that they see percentages when they first encounter triage grading. That is, they see "1/3 points" as 33%, which they interpret as "Low Failing Grade" even though in triage grading it is a low passing grade. (You might consider 33% in triage grading as having a qualitative interpretation like 65% in conventional grading, right on the border of poor and failing.) There was broad consensus about this in the class.

I pointed out that the maps appeared to come from initial experiences with triage grading, but one of my bright students—who has taken my classes before—noted that his was more of a mid-semester view. He had recorded in his map that he became able to see the feedback as qualitative, as coarse-grained values that drove him to change his behaviors. I do not remember exactly the words he wrote on his empathy map, and indeed I didn't understand what he meant by the words he chose, but our conversation came back to the concept "seeing quanta" rather than "seeing percentages."

That was pretty interesting in itself, but here's where it kicks up a notch. A student in the back chimed in, saying essentially, "But it's still a percent." The first student acknowledged that mathematically it was, but that's not what he saw, and the one in the back insisted more strongly, essentially, "But it's some number of points out of a total, and so it's a percent, and so you're still seeing it as a percent."

Wow! What a teachable moment for empathy! I pointed out, treading carefully, that this was an example of the second student showing no empathy for the first student. The second student saw the world in his way, insisted that it was the right way, and that everyone else must also see it that way. I introduced the idea that whether or not there is an objective reality, perception drives a person's lived reality, and perception is subjective. Two people can look at the same thing and "see" two completely different things. The expression on the second student's face told me that he understood what I was saying, but he was still working on the implication; of course, maybe I observed this wrong.

We are continuing to work with this example, and I am finding it a rich context for discussion. I hope to share a few more stories here on the blog, but for now, it's time to head off to class. Thanks for reading!

Open Letter to Fifth-Grade Game Designers

I was recently contacted by an old friend of mine who teaches fifth grade. His students became interested in designing their own card game, and he reached out to me to see if I could give them some feedback. With his agreement, I decided to post my feedback in an open letter on my blog, following the philosophy that if it's worth typing, it's worth sharing.

Greetings!

I was very excited to hear from Mr. Dietzen that he has a group of students who are eager to design games. Mr. Dietzen and I met in high school, and we are from the same small city in Western New York. I am now a Computer Science Professor at Ball State University in Muncie, Indiana; I was drawn into Computer Science in part because it gave me the tools to transform my creative ideas into real working systems.

Shortly after becoming a professor, I started looking for ways to use my love of games—and my students’ love of games—in my classes. I was able to teach a few courses on game design in addition to my Computer Science courses, and for several years I have been able to mentor teams of college students in designing and developing original educational games with local museums and schools. You can play some of my students’ games online, such as Travelers’ Notebook: Monster Tales (travelersnotebookgame.com) and Social Startup Game (socialstartupgame.info). You may enjoy some of the ideas and experimental gameplay that my students have developed, even if the games don’t have the graphical pizzazz that comes with a multi-million-dollar budget.

Game Design is Hard

By now you have realized that game design is hard. There is a world of difference between having an idea for a game and making an actual game that people can play and enjoy. Just like making a movie or writing a book, making games involves a lot of time, patience, and effort. Kudos to you for not just sitting on your ideas and actually making something out of them! I always tell my students, “Ideas are a dime a dozen; it’s the execution that matters.”

#1 Rule: Make Decisions Interesting

The number one rule of game design is this: make decisions interesting. Think about games you enjoy, including video games or card games or sports. They involve a series of interesting decisions. A game where there’s an obvious next best move is not a very interesting game. A game where your choices don’t matter at all is also not interesting. The decisions your players make need to be interesting and meaningful. If you stopped reading now, but you made all of your decisions meaningful, then you would be improving as a game designer!

Game Design is a Process

I was glad to review some of the artifacts that your teacher shared with me. One thing I’ve learned about game design, though, is that the artifacts are only part of the picture: it’s the process of making them that is even more important. Obviously, I cannot see your process, so instead I will share with you some of what I have learned.

I always teach my students that game design an iterative process. That means that you work in cycles or loops. Here is a common way to describe this loop:

Let’s break down what kind of thing you might do at each of these phases. During design, you develop the ideas you have about the rules, the art, the stories, and the systems of the game. It’s a good thing if you throw away more than you keep, so that only the strongest design ideas remain. To implement, you create a model or a prototype that you can test. The purpose of building a prototype is to test it. How do you know if the prototype is any good? You playtest it, of course! Next, you need to take serious time to evaluate your playtesting. Every playtest can teach you something, but that “something” can be… well, anything! The evaluation helps you make decisions as you return to the design step. You are back where you started in the process, except now you are smarter! You know more about your goals and your players from going through the loop, and so you are ready to go through the loop again.

The more often and the faster you can go through the loop, the better your design will be. Designing a game is a learning process, where you are learning to design that game. Going through the loop gives you the feedback you need to improve, in the same way that a good teacher or a good coach will give you feedback on your work so that you can always get better. One of the best ways to go quickly through the loop is to keep the design materials as lightweight as possible for as long as possible. I like using index cards and sticky notes as my primary tools of prototyping. I can quickly build and modify prototypes since it’s very fast to scratch out new notes on paper. These “low-fidelity prototypes” are just simple enough to be played.

Prevent Players from Doing it Wrong

A more specific tip came to mind as I was reading through the material you shared with me. I noticed that there were some card effects that said, “You can only use this once.” This kind of rule means that players have to remember a little bit of knowledge each time they play, but relying on players’ memories is dangerous. Memory plays tricks on you, and you want to avoid cases where players might remember things differently and get into an argument. It’s better to take this kind of “knowledge in the head” and turn it into “knowledge in the world.” How could you design the game to prevent someone from doing it wrong? For example, perhaps the player has to discard the card to do that action: if the card is discarded, they clearly cannot do that ability more than once! You can do similar things with tokens: using an ability may require you to spend some resources, and if you don’t have those resources, you cannot use the ability.

A Moment to Hear; A Lifetime to Master

What I have shared above are some of the most important things I have learned about game design, and they are the core lessons I teach in my game design courses. It’s the kind of advice that’s easy to hear and hard to follow, like “Speak only the truth” or “Practice your guitar daily.” After a semester of working with me, my college students begin to understand it, but I think it takes much more time to really understand it and longer yet to make it a habit. 

Wrapping Up

In closing, I want to say again how glad I am to see fifth-graders excited about game design. Game design draws on so many different areas: math, business, computer science, art, philosophy, language, history, psychology, and more! Who knows, maybe your interest in game design will lead you to being a Computer Science Professor in the American Midwest someday---stranger things have happened, or at least things exactly as strange.

I hope that you found this to be useful and that it will inspire you. If you’re interested, please feel free to write back over email or in the comments. We could also set up a video call if you wanted to follow up on some of the points I’ve shared here.

Happy designing!