Essential Questions of an Advanced Programming Course

I have been following Grant Wiggins' blog for a few months now and familiarizing myself with the fundamentals of Understanding by Design (UbD). It has not been a rigorous scholarly investigation, but I have enjoyed reading Wiggins' posts and thinking about how they connect with my experience and expertise. A fundamental tenet of UbD is what they call "backward design," which is the idea that you start by defining your outcomes and then build lessons and curricula to support that. While this may seem obvious to some, my experience resonates with the claim of Wiggins and McTighe, that many (or most) teachers start with the content and focus on how to cover it. This is certainly true in higher education, where the "covering content" is tends to dominate curriculum discussions, and "teaching a book" is an established part of our discourse.

Two days ago, Wiggins posted On geniune vs. bogus inquiry—using EQs correctly, the continuation of a series of posts, this one focusing on Essential Questions (EQs). I found the essay particularly thought provoking as I am currently looking over several syllabi in my department: Computer Science 1, Computer Science 2, Advanced Programming, Discrete Mathematics, and Introduction to Algorithms. The Foundations Curriculum Committee is looking over some assessment data to determine whether we need to shift emphasis in any of these courses or articulate their objectives more clearly.

The Advanced Programming course in particular falls under my purview. For those who don't know, this is a home-grown course designed around a specific set of problems: students coming from the intro courses (CS1–2) are generally ill-prepared for medium-scale collaborative project-work in the upper level of the curriculum. All students take a year-long project-based senior capstone course, in which they work in teams with a community partner, but this is at the end of the curriculum, so there's a gap in the middle where faculty who want to do project-based learning had been left in the lurch. Enter CS222: Advanced Programming. The syllabus for the course identifies learning objectives regarding interactive debugging, testing, build configurations, and projects of significant size; a revision proposed for next academic year includes more specific learning objectives regarding test-driven development, distributed version control, code review, object-oriented design, and model-view separation. We have been teaching this course for about three years, and I think the results have been positive. The major hurdle we face, internally, is educating the other faculty about how they can take advantage of what the students learn in this course, but I'm hoping to address that in a colloquium presentation this semester.

Reading Wiggins' essay made me reflect on the fact that the tools and techniques described in the syllabus answers. What are the essential questions of our Advanced Programming course? I was so inspired by this issue that I posted a link to the essay and some provocative questions for alumni to Facebook... on a Friday night, where it was quickly lost, and hence my migration to a longer form here.

In a bit of casual reflection, here are some essential questions that I am considering:

• How do small teams of developers coordinate activity?
• Is it sufficient for a program to be syntactically correct and meet functional requirements, or is there more to the definition of rightness?¹
• What does it mean to be "professional" in software development?
• How does a Computer Science professional use ubiquitous and chaotic information to be a lifetime learner?

This is my first time trying to articulate Essential Questions in the tradition of Understanding by Design. I would love to hear what students, alumni, and software development professionals think about these questions. If you have a contribution, please leave a comment!

---

¹ I chose not to use the word "correctness" because I'm dealing with practical development issues, not theoretic program correctness.

One and a half days of rules design

On Monday, the as-of-yet-unnamed Spring 2013 Game Development Studio approved a concept document   that will serve as a spike in the ground. Tuesday morning, we started with detailed rule design. Aside from a few afternoon meetings, I was able to spend the day with the team, from about 9:30 to 4:30, working on rules articulations. They kept on going in the early evening, and when I came in this morning, we picked up the process and finished the core rules articulation by around 2:00 P.M.

Game rules, one of four

I should be clear about who "we" are. The team has a shared space, thanks to the generosity of the Computer Science Department. However, we don't have a shared time that everyone is there. Rather, people come and work for a few hours when they can. Over the last few weeks, we've seen a "morning crew" and "afternoon crew" emerge, but that's a very rough description: many students come to parts of both, or even put in evening hours. As a result, much of what I do is help facilitate communication across these groups. A few times in the last two days, I've been the only person who was present in a morning conversation when a later group is looking at the rules.

Game rules, two of four

The team has done a laudable job. This is hard work, made logistically difficult by the lack of shared time, but I see that the team has grown to trust each other, and to evaluate design artifacts on their own merits. In a sense, there may be some value to the chronological distribution, since an artifact has to be evaluated by itself rather than defended in person. As the team worked on the rules, a cogent game design emerged. Several modifications and clarifications were made along the way, and some rather major revisions as design "holes" were found. There was a little bit of confrontation, but it was all very healthy and led to great outcomes in terms of both the design and the learning.

Game rules, three of four

Here are a few one-paragraph highlights:

Yesterday, we were dealing with the nature of warfare among the Middle Mississippians. In particular, we were looking for information on the palisades they built and the reasons they may have raided. At one point, I was in the room with four Computer Science Majors, and each one of them had his nose buried in an academic book—yes, a book—about archaeology, anthropology, and Middle Mississippian culture. I wish I had thought to take a picture. When I told my collaborator, History professor Ronald Morris, about this, I think it brought a tear to his eye.

Game rules, four of four

Mid-morning, we were trying to tackle the problem of the scale of the game world and the distribution of resources. I encouraged the team to use the whiteboard or one of the tables to physically lay out pieces, in such a way that they could be easily moved around. One of the students went almost-immediately to the "prototype" drawer of the file cabinet, pulled out another student's prototype that involved lots of paper chits, and dumped them on the table. This strikes me as significant evidence of learning and team work: the student remembered his teammate's prototype, realized it met our immediate need, knew where it was filed, and brought it to bear to solve the immediate design need. Another student went to the box o' stuff and pulled out a VGA cable, which became the river. In doing this, the team realized that while we had talked about having rivers on the map, we had never discussed the river with respect to villagers' travel. This opened an important new avenue for discussion, both of the Middle Mississippian culture and of game balance.

Sketch of the game world. The two named players, Stan and Barbara, are nascent user personas.

At the very beginning of the rules articulation, one of the students pulled out her laptop and began transcribing everything that was being written on the board. In retrospect, I should have stopped her, encouraging her instead to be completely present with the process at hand. It was a good instinct she had to want to capture the physical writing in a digital archive, but it was premature. In past projects, I have seen students get confused about conflicting information, especially regarding whether things in physical space or digital space are current and up-to-date. I have a strong preference for analog information radiators, which is the main reason I advised the team toward a physical task board over, say, a shared spreadsheet or digital project management software. I mention this here to help me remember, next time, to make sure everyone is fully present in what I consider to be the critical part of the task at hand, since I think the students are still learning to distinguish critical from ancillary.

As I walked to work Tuesday, I was reminded of how useful it was at the VBC to have concept and production art pasted all over the walls. We did this relatively late in the process, but that team identified it as a key element for both helping the atmosphere and facilitating communication between technical production and art production. So, I contacted the primary artist for this semester and asked her to post a few concept art pieces on the wall. I was pleased to see some excellent sketches, annotated with her commentary, right out of the sketchbook. When Ron came and hang out in the studio today for a bit, he had some questions about it, so I encouraged him to just put his questions right up on sticky notes, and they are captured in this picture as well.

Concept art is hung on the wall, with some author and team annotations

Today's last step was for the Wednesday Afternoon Crew to look over the rules, identify missing pieces, and then start striking anything that didn't contribute to the core game design. They ended up patching three holes in the design and cutting one feature that didn't clearly tie to the design objectives. Now, they are off and running with a few related tasks:

• Digitally prototyping the production/consumption rates in a spreadsheet to see if our mental math produces the balance and scarcity of resources desired.
• Physically prototyping the essential game rules so that we can playtest with upper-elementary school children to see if the dynamics we desire arise from the mechanics we defined.
• Craft a one-page design document that can be posted on the wall

Author's note: I realize as I am about to publish this that I never actually finished my post saying what we're doing this semester. I guess I'll have to get to that another time. To make a long story short, we're making a game for fourth-graders about the Middle Mississippians. You probably inferred that from the story above.

Global Game Jam 2013: WikiBeat

I spent last weekend at Global Game Jam. Last year I was at the VBC, and I took several students with me to Columbus, where we had a great time. This year, I brought two students from my game studio to the Indianapolis site, hosted at the Art Institute of Indianapolis. We arrived in plenty of time to chat with some of the other jammers, many of whom were students from the area, including one other Ball State student. Closer to 5pm, we moved over to the computer lab for the keynotes and theme announcement.

The theme—in case you didn't click the link before or haven't heard—was a heartbeat. It was announced with a bit too much melodrama, but I congratulate the organizers for identifying a universal theme. I want to say a bit about "theme" here, though. Last year's theme was the ouroboros, and while many game ideas explicitly included snakes and tails, there was a lot of discussion of metaphorical interpretation. This year, everything I heard had to do with hearts quite literally. This was disappointing. One of the primary reasons I took the team to Columbus last year was to meet up with Ian Schreiber, who is one of the GGJ organizers, and I remember him stopping in toward the end of the jam and saying something like, "There are so many damned snake games!" Clearly, as a member of the theme selection committee, he was hoping for metaphorical interpretation. I wonder what he thinks of all the heart games created this year?

There were a few reasons that I wanted to go to the Indianapolis site this year. It was organized by the Indianapolis IGDA, of which I am a member, although I don't make it to a lot of their events; this made it a good opportunity for me to go see some friends that I only see a few times a year. I have some other friends in the city that I was hoping to meet up with for lunch, but sadly that didn't come together due to a communication breakdown. I also have several alumni in Indianapolis, including folks I worked very closely with in 3:15 Studio and Root Beer Float Studio. Josh Hurst was the only one who came and participated in the jam, but I am glad he did, and we were both eager to work together again. (Here's his blog post about the weekend.)

When Josh and I heard the theme, our minds both went to the same place: hearts and babies.¹ We figured there would be a host of games about hearts and babies, and so we decided to draw inspiration from the diversifiers. We had both read through them before and were intrigued by this one:

The Truth is Out There
Some external real world element, such as weather, time of the day, etc, affects gameplay. It MUST affect gameplay and can’t be only a scenario element.

After the theme announcement, we started brainstorming data sources and gameplay elements. We liked the idea of a word game in which the player has to know what topics are currently in the news or trending, and with this inspiration, we started exploring available data sources. All the jammers were in one crowded room, so Josh and I went to one of the other rooms for our experiment. As a result, we missed some of the discussion and pitches, but we were already on a good course that we knew required some serious investigation.

We were really excited to try to get data out of Google Trends, my thought being that popularity and unpopularity are the oxygen and carbon dioxide of culture. Unfortunately, there's no public API for it, despite there being noise about a release as far back as 2007. Twitter required accounts that we didn't want to tinker with. Turns out, a project of mine had just recently been covered by the AP, and so I was inspired to see if they provide public RSS feeds, which they do. In fact, they're conveniently separated into eleven categories.

Plan A: AP RSS

We talked about doing multiplayer on mobile devices, but that felt both ambitious and limiting—although we both had Android devices, we decided we'd rather have the game be widely playable than customized to one mobile OS. We ended up with a sketch of a hotseat multiplayer design for the Web, with the plan to add simple networked multiplayer if we had time. We chose GWT, in part because I've really wanted to build and ship something with GWT for ages, and also because we knew it would simplify adding App Engine networking should we have time to add that.

At this point I will note, in case it's not already clear, that neither of us are Web developers, and we were victims of second-order ignorance, heading for a brick wall. We didn't know that at the time, of course, so we got to a happy place and headed home/hotel for a good night's rest. We both have infants at home and had just finished stressful weeks at work, so a good night's sleep was certainly in order.

The Pyramids, home of the AII, early in the morning

I got up early the next morning and was back at the AII by 7:30AM. Two bleary-eyed jammers were in the computer lab, and everyone else was asleep. You can't keep a morning-coder down, so I set up shop and got to work. Josh arrived a little later, and by midday, we had all of our RSS parsing logic done and had turned to prototyping the interface. We had been writing everything in Eclipse using TDD, and at this point Josh suggested that we should get the end-to-end prototype running in the browser. I told him I was not worried about it, but a moment later realized he was right, and so we slapped together a simple UI.

Maybe you've heard of the same origin policy. It's a security feature in the browser that prevents scripts from connecting to domains other than the one from which they were loaded. The implication for us was that our Javascript in the browser, loaded from our server, could not open a connection to the AP RSS feeds. It took us hours to realize the problem, and when we did, there was much forehead-slapping. We found some documentation about workarounds, but none seemed to work.

After a bit of investigation, we decided we'd try JSONP, which appears designed specifically to circumvent the same origin problem. It requires a server to support it, and the AP doesn't, so we began to look for similar kinds of data sources. We spent a few hours tinkering with different servers until we landed upon the MediaWiki API supported by Wikipedia. This API is fantastic. Everything you can do through the browser, you can do through the API. Based on the data we could get, we retooled our design so that the player would get points for identifying Wikipedia pages with the most edits in the last thirty days. It's not quite the same as "trending", but certainly, controversial and newsworthy topics see a lot of edits, so it's in the same spirit.

Plan B (or C, or something): Wikipedia Revisions

We set to work retooling the design and decided to abandon hotseat in favor of a local high score table, which was easily implemented with GWT's cookie support. A single player can play to beat their high score, and one can easily share this score with friends to get ad hoc multiplayer, without our having to implement multiplayer as a mechanic. This observation actually emerged from testing, where Josh and I informally challenged each other to beat scores when testing the API access.

Somewhere in the middle of the afternoon, we needed to go get some lunch, so we went down the street to Famous Dave's. I haven't been to a Famous Dave's since visiting my brother in Madison around fifteen years ago. Our waitress was fantastic, and I enjoyed a generous lunch platter along with a Wee Mac. It was so good that I took a picture.

Southside Rib Tips at Famous Dave's

By the end of the day on Saturday, we had and end-to-end prototype working in the browser, but in dire need of some beautification. We called it a night around 12:30am and headed back to the hotel for a few hours' sleep.

Here's an interesting phenomenon: If you have a newborn at home, and you worked all week, and you programmed from 7:30am until 12:30am, and you don't set an alarm, ... well, you can't really guarantee when you will wake up. I woke up a little groggy and looked at the clock, surprised to see that it was 9:30am. We missed the continental breakfast, but at least we were well rested! I grabbed a yogurt and coffee, and Josh ran out for bagels, and we were back at the jam by 10:00am.

Sunday, we officially named the project WikiBeat, and we worked primarily on the high score support and user experience. The best contribution was Josh's idea to turn each guess into a link to the relevant article. It's obvious when you take a step back, but the thought hadn't crossed my mind. We made one blunder with about two hours to go, where we decided to modify how the player's guesses were presented, which meant altering a core data structure that was also part of the presentation layer (yeah, I know). As a result, we discovered later that there is an error where after your first game, your score may or may not be computed correctly. Sorry about that.

Raised by a cup of coffee

After all the submission deadline, there was some time for demos to show results to the other local jammers. You can see all the other games that were created at the GGJ page for the AAI site. I'll point you directly toward the other two that my students created, aMAZEing Embolism and Distance Makes the Heart Grow Angrier. Both of these games were spearheaded by my students, who were able to form teams around them. I like to think the work we've done together on game design and game programming helped. As predicted, all the other games included hearts literally, and when I presented WikiBeat, one of the few questions was, "What does this have to do with the theme?" I briefly explained what I said above, about literal vs. metaphoric interpretation, but since I think Josh and I may have been the only ones who discussed metaphoric interpretation, I suspect this may have been lost on the rest. I suppose that's the flip side of my and Josh's separating ourselves from some of the rest of the pitch process, that our ideas were a bit insular.

I had a great conversation with my students on the way back to Muncie about game design, game development, and jamming. I think it was a good learning experience for them, and I know it was for me. It took me a week to start this post, and it took me most of the morning on a day off to write it. Regardless, I enjoy the process of pulling all the details back into my head and getting them into a coherent narrative. I am reminded of a comment that came up the week prior to GGJ: if game development were easy, people would do it all the time. Turns out, it's really hard, and that's what makes it such a rich intellectual activity.

Thanks to the Art Institute of Indianapolis for hosting and providing subs on Friday night, to Indiana Uploaded for the pizzas on Saturday, and to all the global partners and sponsors. Extra special thanks to Thomas Marshall (Puca Studios) and the Indianapolis IGDA for organizing the event.

Here are the GGJ page for our game and the playable version of the game. Global reported high score so far is David's 301 (in the comments below). Post if you can beat it!

---

¹ On the way home, I talked to my students about the design process. When I told them that Josh and I assumed there would be a lot of "baby" games, they had no idea what I was talking about. For those of you who don't have children (or maybe missed this experience), there is an amazing moment early in the pregnancy where you can hear the little person's heartbeat. It's well before all the sonograms and checkups are possible, and it's an amazing experience. Hearing a heartbeat makes me (and Josh, and probably lots of other fathers) think about that moment.

Fog of War: An obfuscated defect

One of the major technical contributions I made at the end of the Underground Railroad project was to fix the fog of war feature. The designer's intent was that a player would only see the current county and one county away; if the player was at a depot of the Underground Railroad, they could see one farther. The team had already implemented a map and an invisible overlay to handle movement. The overlay had the same topography as the map that the player sees, but it was color coded. When clicking on the map, we look up the corresponding coordinates in the overlay to determine the county, as well as whether the countryside or county seat was selected. The same system is used for the popup hints: look up the mouse position in the overlay, and if its a legal target, inform the player.

The theory behind the fog of war, then, was simple. We added an opaque layer on top of the map, which I will call the fog layer.¹ When the player's position changes, first determine which counties should be shown. Then, look for those counties in the color-coded overlay. Because the invisible overlay is the same shape as the actual map, turn the corresponding pixels transparent in the fog layer. Presto! We've cut out exactly the right shape in the fog layer to see only the counties that should be shown. Because the map was so big, the team came up with some appropriate optimizations, restricting the search for matching colors to those areas on or near the edges of the current camera view.

Starting the game in Jackson County, KY. The player only sees one county away.
(Freedom is North, so we don't let the player go deeper into the South.)

The team created a very domain model of the game counties, assembled using the Builder pattern in a fluent internal DSL. A representative line looks like this:

Add(Make ()
  .WithID (CountyID.DelawareIN)
  .Coordinates (40.211893f, -85.396077f)
  .SetDepot (1)
  .IsOnRiver ()
  .NorthernIndiana ())
.WithColor (35, 50, 0);

This is adding a new county to the registry. In particular, this is Delaware County, whose county seat has particular GPS coordinates, which had one depot, is on a river, and is in the northern half of Indiana. The last part constructs the color key in the county registry. Note that the actual visual color is arbitrary, since the player never sees it: all we're doing here is saying that the pixels colored as the RGB triple <35,50,0> correspond to Delaware County.

Here's where things get interesting. Unity3D has two color classes: Color and Color32. The former represents colors as four-dimensional floating-point vectors in the range [0,1], while the latter uses integer vectors in the range [0,255]. Each class has a directly corresponding constructor:

static function Color (r : float, g : float, b : float, a : float) : Color

static function Color32 (r : byte, g : byte, b : byte, a : byte) : Color32

Also, both classes support implicit conversion to the other. This means that if you have an object of one type but need the other, it will automatically do the sensible conversion—or the most sensible conversion it can.

I remember the students who worked on the domain model had some trouble understanding these different classes very early in the semester, but then I assumed that everything was taken care of. However, in trying to trace down defects in the fog of war feature, I saw some very strange color processing code. As I poked around the code, I noticed that both Color and Color32 were being used in different places. Whenever there was to be an implicit conversion, the same piece of bit-fiddling code appeared. That certainly shouldn't be necessary, since there's no way the conversion API was broken. After some exploration, I tracked it all back to the county registry builder. Here's the original implementation of WithColor:

public Builder WithColor (float r, float g, float b)
{
    Color key = new Color (r, g, b, 0);
    _county.Color = key;
    _builder.map.Add (key, _county);
    return _builder;
}

The astute reader will note that the error can already be identified. Spoilers ahead.

The call to WithColor sends in a triple of integers—35, 50, and 0 in my original example. These int values are silently converted into float, since that's what the method expects. Then, these floating-point values are passed to Color, which also expects float arguments. Remember how Color is defined? It's a four-dimensional vector of floating-point values in the range [0,1]. Yet, the Color class happily accepts the 35, 50, and 0. If you inspect this Color object or ask it to print itself, you find out that it's <35.0,50.0,0>, as you might expect. Then, if you convert it to a Color32, that object is <255,255,0>.

This also makes sense, if you think about it for a while. The Color class only really cares about values in the range [0,1], and it interprets anything above that range as being equivalent to one. So, if you drew the Color that is <35.0,50.0,0.0>, you would get bright yellow, not dull reddish green. If my students ever fully understood this, they certainly didn't articulate it or move to fix it: instead, they developed a kludge to pull the Color values out as floats and pack those back into a Color32 manually.

There are several lessons to this story. Here's what I got out of it:

1. Read the API docs. There's no good reason to send floating-point values like 35.0 to the Color constructor.
2. Make sure your API produces appropriate warnings when being used in such a weird way. There could have been a warning either upon sending the out-of-range values to Color or when they were used in the automatic conversion.
3. Don't use primitive types when they are not semantically appropriate. Color should not actually take floats as arguments when what it really wants is values in the range [0,1]. So, make a class to represent that concept, and use that. Before you complain that this will impact performance, remember that premature optimization is the root of all evil.
4. Refactor. The bizarre color-handling kludge showed up in at least two places, clearly the result of copy-paste coding. Had the original developer stopped and refactored this away, he would have at least made a better abstraction for handling the problem. In the best case, he would have recognized the problem and fixed the root cause.
5. Automatic conversion is awful. You might think you're improving readability of your code, but only if you and the reader share the same mental model and expectations. Better to make it explicit. To me, it's similar to the desire for using static factory methods over explicit constructor calls: good naming can reveal your intention.

By the way, The Underground Railroad in the Ohio River Valley is now open to the public. Enjoy!

---

¹ I was surprised that the students didn't understand "fog of war" as a metaphor. The original opaque image  looked like literal fog, and at first I couldn't understand why they had chosen it. I had used a technical term from game design, thinking it was common vocabulary, and this led to unexpected team behavior. Usually when this happens to me, it's computing jargon, not game design jargon!

Reflecting on the Fall 2012 Game Design Colloquium: Expectations and Reality

Background

In Fall 2012, Ronald Morris and I team-taught an honors colloquium about serious game design. At the start of the semester, I wrote about the achievement-based grading system we intended to use and how it fits into a larger academic-year game production project. To make a long story short, we had a team of students from a variety majors, and each was expected to make an educational non-digital game for the Indiana State Museum. I took the day today to compose my reflection on the course, which is now shared with you in four parts: my expectations when we designed the course; what actually happened when we ran the course; insights gleaned from the final exam; and what I might do differently next time.

This is my third attempt at composing this reflection. The first was a collection of notes and observations that were too raw to share. The second was a narrative that attempted to map expectations, execution, and lessons learned into nice triads. This ended up being a mess, as the relationships are much more rich than simple one-to-one mappings. I mention this because it strikes me as significant to my reflective practice: in taking the time to write carefully, I have been able to understand the semester a bit better and come to peace with some things that had been bothering me.

Expectations

Students would enroll because they were interested in either the topic or generally in immersive learning experiences. By explaining all of the expectations in the course description, including an explicit statement regarding nine hours of attention per week, students would know that they needed to take the course seriously and commit themselves appropriately. Achievement-based grading would motivate students to engage with the material. Given that this was an honors colloquium—and hence only open to students in the Honors College—they would be responsible enough to set a good pace of achievements throughout the semester. The achievements leaderboard was posted in a shared location, which would lead to positive peer pressure for everyone to keep up. Some achievements rewarded students for reading and commenting on each others' essays, and this would lead to interesting discussion, deeper thinking, and better critical analysis.

Since the students have set up their own schedule based on interest and other personal commitments, their independent activity during the first third of the semester would complement the in-class discussion of game design fundamentals. This activity would then strengthen their work in the next third of the semester, which would be devoted to iterative prototyping. By the final third, the students would be focused wholly on producing their games, with all the other achievement-based background work done.

To help students with the Socializer achievement—as well as getting through the canon of genres—I would announce all of the meetings of the Society for Game Design and Development as well as the Game Days at the Muncie Public Library. Students would approach these events with some expected trepidation, but once they realized the friendliness of the community, they would continue to attend these events, both for personal pleasure and to gain exposure to new game design ideas.

At the end of the semester, we would showcase our games to the Indiana State Museum. Some of the games may be usable right away, but all would serve as media for communicating important design ideas. Given their expertise in production, the ISM could very likely produce better looking or more durable game bits. This means that clarity would be a key concern in the students' designs.

Each student would make their own game deliverable, but the designs themselves would not be graded. This could be too subjective as well as dependent upon prior experience and extracurricular digital production skills. Instead, we would give students a mark based on their following a good iterative process. As explained in the course description, we expected each design to go through several iterations, and failure to present a prototype in each iteration would result in grade reduction. Hence, the only source of grades for the student were the achievements and the iterative design process, meaning that students could really choose their grade based on their level of commitment. Because we did not need any other grades, the final exam would be a reflective exercise.

Reality

We have little information about why the students enrolled in the colloquium. Most of the students had little to no background knowledge in game design, and little passion for the topic. One student told me that she enrolled because "design" was outside of her major and experience: she identified it as an area for personal growth during her undergraduate studies. This is a great reason to take a course, and she proved herself capable during the semester. Another confided that he took the course only because of the time it met and to satisfy the Honors College's two-colloquium requirement.

For the first third of the semester, almost no achievements were completed. Ron and I gave the students some class time without us to collectively develop a plan, and this partially worked: they started holding meetings with their classmates to play the games of the games canon. Unfortunately, this happened after the transition from fundamentals to prototyping in the course schedule. Because they were completely unfamiliar with many established genres, they could not draw upon these in their own prototypes.

As we got into the prototyping phase of the semester, we started by formally scheduling students every two weeks or so. The students preferred a more ad hoc approach in which individuals would bring prototypes as they were ready. This caused a conflict with the course description, in which we intended calendar-based iterations with students presenting a new prototype each cycle. The students were right, however, that some prototypes took longer than others, especially in those cases where students completely changed topics. We agreed to let the students follow this process, entrusting them to hold each other accountable, even though it meant our iteration-based grade penalty system would become unenforceable. 

As the semester went on, we witnessed design stagnancy. Many students brought essentially the same prototype after having had a week or two to work on it, and it became clear from their discourse that there was little to no playtesting happening outside the class meetings. A few students were able to recover from this with some not-so-gentle pushing by Ron and me, ending up with good designs but only after having fought against our feedback.

At midsemester, there was still very little progress on the achievements, though with a few exceptions. Ron and I held individual conferences with the students, during which we provided some feedback on their designs and also encouraged them to enact a plan for meeting the achievements. There was a lot of smiling and promising, but very little material evidence that these meetings were worthwhile. Most of the achievements ended up being completed in the last two weeks of the semester, many appearing rushed and certainly not contributing to the students' all-but-finalized designs. Because we had these individual meetings, though, we know that it was procrastination and a failure to plan on the students' part that led to this situation, as opposed to a fundamental flaw with our expectations.

Some of the achievements simply took a long time to complete: students made progress on playing the games in the canon for several weeks before marking the achievement as complete. However, the one that was delayed the longest, which caused me the most frustration, was the Socializer. Only one student completed this achievement before the last two weeks of the semester. In their reflective essays, students acknowledged—after the fact—that it would have been better to go earlier and more often.

Near the end of the semester, we engaged in two rounds of external playtesting. The first was with Motivate Our Minds, and the second, College Mentors for Kids. Our students highly regarded these opportunities, even in those cases where the young playtesters were not quite in the target age group for the game. I would have liked to see more preparation on the part of my students: some had not defined all of the terminology in their games, and others were missing key physical pieces of their games—which, again, betrayed a lack of playtesting.

We had three in-class presentations in the last two weeks of the semester for the Scholar achievement, one each on A Theory of Fun for Game Design, The Art of Game Design: A Book of Lenses, and Homo Ludens. I was surprised by how easily the students slipped into discussion mode, engaging each other and voicing opinions. At the same time, it was disappointing at the end of the semester that some of them were still falling prey to very naive notions of game design and learning. I opted against steering these conversations, instead listening for evidence of who had integrated knowledge from their experience and who had not. I would like to claim that those in the former group made better games as well, but I think my recollection of the conversation is too colored by my post hoc opinion of their games.

The students made an impressive display of their games for the community partners. We invited representatives from the ISM to campus, and each student explained their game's theme and demonstrated gameplay. The ISM staff responded most positively to those that were simple and effectively described. This is predictable but unfortunate for the students, since the games that actually contained the best designs were not necessarily given the attention they deserved. What I mean by "best designs" are those that reflect what is known about games and learning, and not coincidentally, these were mostly the ones that went through the most revisions.

Most of the games do good design elements, and I believe that we can call this a successful outcome. Comparing them to the students' first critical analyses and initial designs, we can see a general transition from novice toward advanced beginner. However, there are also those corresponding design decisions that appear to have been left without critical analysis, but of course, learning how to ask these questions is part of learning the craft. We are currently considering which of the designs to move forward with into digital production in the Spring, and there are several that I think would make great starting points.

Final exam reflection

The final exam consisted of the following questions:

1a: Pick a specific exhibit at a museum you visited this semester, not including the one for which you made a game already. Now consider that the museum approaches you to create a game for use in a summer camp, where students engage with thematic ideas (history, science, etc.) in a day camp format. Describe the process by which you would deliver such a game. Note that we are not asking you to design the game, but to describe the design process for the game.

1b: Explain, in detail and drawing on your personal experience, why you believe the answer to 1a to be true. 

2: What was the relationship between the course structure and your learning this semester? Be sure to specifically address peer evaluation of prototypes and essays as well as the achievement system. You might also consider what elements you would change or add. 

3: How many hours did you spend on this class each week? Did this change during the semester, and why? 

4: Excluding yourself, who did an outstanding job this semester? Consider both their contributions to the class and their product.

The first was designed to get them thinking about what they did this semester and consider what they have learned. Part 1b is more important to the learning process, as this requires metacognition. The answers to these questions were all similarl: the students would do it the same way we did it this semester. While the students were able to summarize all the various steps we had taken, and they appeared to have faith that this was a good way to proceed, I was a little disappointed that there was not more critical discussion of what worked and what didn't work. In particular, only one student nailed the importance of research for serious game design. That is, to make a game that matches a museum theme, the designer needs to have a good understanding of that theme.

The second question was designed to get specific feedback on course structures. Almost every student mentioned that they liked the achievement system but that they would have liked some deadlines in order to help them manage their time more effectively. In particular, they wanted the Scholar achievement moved earlier in the semester to provide more foundation for their work. If we had one more meeting, I would have pointed out to them that this course of action was available to them all semester, and that it was their procrastination that pushed this to the end of the semester. I hope that they recognize this and that it was a good learning experience, but in the future, I would rather prevent the students from having to learn this lesson, so they can focus on course ideas instead. One of the best ideas shared by a student was to have achievements for playing more than one game in a genre, rather than just having them for playing different genres.

The third and fourth questions were Ron's ideas, and I am glad we added them. I assume that their answers were honest, since there was nothing to be gained at this point—remember that this "exam" did not contribute to the students' grades at all. According to their answers, about three students spent the expected nine hours per week on the course, and the rest spent less, some much less. This is interesting, as there was some complaining during the semester about how long it takes to learn and play some of the games in the canon. Several students admitted to spending no time on the course during the first third, as we expected. I think this shows that the level of rigor we expected of them was appropriate, even only some of them really rose to the challenge, and it indicates that students may do better with more scaffolding.

As for the fourth question, it reflected a similar phenomenon to the ISM's visit: the students seemed to value highly their peers who were most vocal, even in cases where—in my expert opinion—they had nothing to say. I take this as an indication that we needed more opportunity for the students to talk about specific design details: this would prevent the generalities and platitudes that students are so willing to share.

Next time

With an ounce of luck and a grant, I should be able to teach a similar course next Fall. With this in mind, here is a summary of what I would consider doing differently next time.

• I need to plan on a class full of partially-interested novices rather than hope for a few with passion and background knowledge. Should such students show up, I'm sure I can capitalize on it, but I need to make sure I'm not quite so surprised when initial critical analyses are all about kids' games.
• The students should do more smaller designs as a way to explore the design space and get used to the designer's mindset. In particular, I should have them make and share one-page design documents on shared themes, a trick I picked up during the semester from a conversation with Lucas Blair at Meaningful Play.
• Keep the achievement system, but add deadlines. Scott Nicholson—who I also met at Meaningful Play—recommended a system in which students have a few dates during the semester by which they must complete a threshold number of achievements in order to earn a certain grade. For example, completing three achievements by each of the dates would earn an A. I will likely do something similar, probably sending Scott a friendly request for more details as the hypothetical future course gets closer.
• Decompose and decouple achievements. Rather than have one that takes weeks and five designer games to complete, make this five different achievements. My colleague Evan Snider has explained to me how he chains together smaller achievements into "quests," which is something I might consider, although I do worry about taking the metaphor too far.
• Only use positive grading, not punitive grading. Our original model had grade penalties based on student infraction of policies, but this implies that students have points to lose. I prefer a model where students cannot lose credit, only gain it.
• Push them harder, individually if possible. The best products, and the best learning, came about when Ron and I contacted students individually to push them to do better. Each rose to the challenge.
• Include a request for feedback as part of the final exam. Since the university moved to digital course evaluations, their usefulness has plummeted. Most students do only perfunctory evaluations and do not provide the kind of useful feedback as when done on paper and in class. The comments from the exam were much more useful to me than the course evaluations, even though they ostensibly asked the same questions.

The Development of "The Underground Railroad in the Ohio River Valley"

I am just finishing my role as Technical Director of an educational game project, The Underground Railroad in the Ohio River Valley. My friend and colleague Ronald Morris received a grant from the Entertainment Software Association to produce a game on this theme, and he and his team worked through Summer on research and design. Early in the project, I offered the services of my CS315/515 Game Programming class to serve as the technical team, knowing that this would be a good immersive project for them.

Preproduction

Some of Ron's students did preliminary research and design before summer, but it was during the summer semester that most of the design happened. I was not heavily involved in this stage: I did a bit of consulting with the design team, primarily on what scope was appropriate for my team to complete in the Fall. At the end of the summer, I was given a set of design notes, which I transcribed and interpreted into a design wiki. The original format was a PowerPoint slideument, which was wholly inappropriate for the task, in part because there was no way to tell by looking at the printed copy that some of the data tables exceeded the size of the page. In retrospect, I wish I had remembered to point the design team toward Stone LeBrande's One Page Design approach—not that they necessarily need one page designs, but it certainly would have gotten them thinking more critically about how to communicate this information effectively. Regardless, by copying the information into a structured design wiki, I was able to build my own mental model of the game, codify nomenclature, and identify missing pieces. As usual, I used Google Sites as a lightweight wiki. The fact that the design was essentially static meant that I didn't have to deal with refactoring or maintaining the wiki: it was primarily for structured presentation to the technical team.

Lead designer Michael Smith demonstrates the paper prototype to community partner Jeannie  Regan-Dinnius of the Indiana Department of Natural Resources Division of Historic Preservation and Archaeology

Jeannie explores the physical prototype

My experience with the first semester of Morgan's Raid production taught me that wrangling 24 undergraduates took all of my time—time that would be better spent helping a smaller group to learn and make progress on the game. The chair of my department is very supportive of this kind of immersive learning work, and he permitted me to cap the enrollment of CS315/515 and institute an application process. This allowed me to communicate my expectations to students before the semester started, so every student who applied knew that we would be working in a studio space, with a client, and with the expectation of nine hours per week dedicated to production. Thanks again to the support of my chair, the team was given the use of our undergraduate research space, RB368.

RB368, before the start of the semester

Team logistics

My team of twelve gathered on the first day of the semester to begin work. I had prepped them over email and as part of the application process, and we used the first week to talk about project management using Scrum and the basics of Unity3D. I had used both of these in my semester at the Virginia B. Ball Center for Creative Inquiry and felt comfortable applying them to this new project. We set up a task board and got to work, planning seven two-week sprints to bring us to a successful completion.

The class was scheduled at 9AM Monday, Wednesday, and Friday, and this was when we held our "daily" stand-up meetings. The fact that many people had 10AM classes very quickly became an impediment, albeit an expected one. Unlike at the VBC, and unlike the fortunate scheduling of the Morgan's Raid Spring Team, these students had non-complementary schedules. As a result, there was very little whole-team collocated work outside of 9AM MWF. After-hours meetings were regularly initiated by the team, but I was rarely able to attend. Each student was on their honor to give nine hours of work per week to the project, but it would have been nicer to have more collocated time.

The technical team hard at work

The lack of collocation was a particular impediment with respect to art production. While the technical team was working for academic credit, the artists and musicians were hourly workers hired through the ESA grant. Ron was responsible for recruiting and managing them, although this caused some communication problems when the artists and musicians did not know to whom they were accountable—the professor paying their checks or the professor giving them work. I do not doubt that this would have been ameliorated had they worked in RB368 along with the technical team in crossfunctional teams, as was their direction; instead, each retreated to his or her own corner of campus. The predictable result was that a lot of time was wasted producing assets that were not usable.

After failing to complete any user stories in their first sprint, the technical team was able to pull itself up by its proverbial bootstraps and get productive. We conducted retrospectives at the end of each sprint to reflect on what was working and what wasn't working. The same issues tended to come up time and again, but I do feel like the team made progress in self-management and accountability during the semester. One of the perennial problems I have in dealing with students—which I suspect is a subset of dealing with humans—is their love of generality. Students will very gladly speak in platitudes and generalities, rather than nailing down specific people for specific problems. As a result, everyone tends to smile and nod and feel that a reflection was useful, but not necessarily hold each other accountable, since there's always a way to wiggle out. I got a copy of Patrick Kua's The Retrospective Handbook during the semester but have not had the chance to read it yet; I hope that I can find some practices within that I can bring to next semester's project (more on that in another post).

Managing and teaching

Managing these projects can be stressful. Each sprint, the team talks about trying to make steady progress, yet almost every sprint, it feels like there is no way they will complete their tasks on time. The team's retrospective notes show that they grow to desire steady progress. However, there is still a strong tendency to fall to the illusion of progress in the first half of the sprint, doing a lot of talking and hand-waving, and then rushing through the second half. As a result, validation was inconsistent, and tasks were rarely done done. Even at the end of the project, a major defect still existed in the build, and the response from the students working on this feature was, essentially, that the defect was enshrined in their implementation and we'd have to live with it. (NB: I fixed it in about half a day's work, but see the section on Expertise below.)

The stress this semester was compounded by my nagging doubt that I was not spending enough time sitting with the students and creating software with them. In the three hours per week that I saw the whole team, my time was spent on project management: clarifying design ideas, coordinating communication, pulling on the artists for sketches, and barely having time to look over shoulders at source code before people ran off to their next meetings. My personal schedule did not permit my coming in to after-hours meetings, in large part because of the welcome addition of my third son. I needed to prioritize family over working extra hours, and I have no regrets about this! Yet, I cannot shake the feeling like there had to be a way to influence the students more strongly toward best practices. I had put together a team programming manual in an attempt to codify collaborative practice, and though this document was distributed before the semester and reviewed in the first week, it went almost completely unheeded. Actually, it was a bit worse than that: the students religiously followed the parts that made life easier and ignored the parts that actually would have made the implementation better. In particular, I borrowed two bits of Robert C. Martin's Clean Code:

• Don't use comments: they rot and indicate a failure of attention to careful design.
• Don't repeat yourself: replace copy-paste code with better abstractions.

I'm sure you can guess which one of those two they followed. Hint: we had a lot of copy-pasted code with no comments to explain it. To be clear, I am not blaming the students for being novices. I think that a major factor here is that they could not see the master working his craft, and so I fear that many did not learn some of the deeper lessons I hoped for them. My principle contrast here is with the Morgan's Raid Spring Team, a team in which I was embedded for about nine hours per week in Spring 2011.

A found several missed opportunities for good software design as I've worked directly with the code this week. One of the most egregious is in the handling of percentage tables. The core game design involves tables of outcomes determined by percent likelihood, in the vein of a tabletop wargame. Code like this can be found throughout the implementation:

Random rand = new Random();
float roll = rand.nextFloat();
if (roll >= 0.0f && roll <= 0.05) {
  // do something
}
else if (roll > 0.05f && roll <= 0.10f) {
  // do something else
}
// etc.

This happens a lot in the code, including the redundant lower-bounds checking. A simple table abstraction, with methods to add entries and compute percentages, would have made this much more readable and maintainable. I'm not sure how to help students perceive these affordances, but it's something I want to try to spend more time on in both CS315/515 and CS222, which is the prerequisite.

There were several times when I was able to push students in the direction of better design, and some of these came up as significant learning outcomes in our semester retrospective. I worked with students to incorporate the Builder pattern in the domain model and frequently assisted with functional decomposition. I also guided the team to use a formal state machine analysis of the game and then use the state design pattern to implement it; this change impacted the entire team for the good. These interactions, in which I guided students toward great ideas of software design, showed up positively in course evaluations and the semester retrospective.

The team considers the task board and the state diagram

The picture above demonstrates the recurrence of a theme that shows up in my scholarship: the importance of dedicated space for these kinds of projects. The picture above shows how the whiteboard served as an information radiator: the tasks were always posted, and critical diagrams such as the state diagram stayed up for weeks. (In fact, the team seemed to really like the term "information radiator," incorporating it into their dialog and writing after my introducing it to them.) The team could have erased and reused the board for other purposes at any time—and they frequently did early in the semester—until this kind of information began being posted. Then, the team recognized the need to keep these models available for easy reference at all times. Consider also the handling of the game's various dialog boxes: they underwent many revisions before the team settled on a standardized way of handling them, and the visual designs occupied a side board for about half the semester.

Dialog box design standards

Working with clients and the community

Ron had a class at the same time as CS315/515, and so he could not attend our Sprint Review meetings. Knowing that we needed his feedback, we held Sprint Review Redux meetings at 10AM, and about four or five team members would stick around for it. It didn't come up until the course evaluations, but the students who stayed really valued this interaction. I had not thought much about the impact of this meeting on the students, primarily thinking of it as something I needed to ensure we were going in the right direction. After reading the evaluations, I realized that this such meetings are critical to the team's morale: these meetings helped them feel like valuable contributors to a bigger purpose rather than programming automata or hired help. I wonder if a different teaching schedule would have led to the team's being more consistent in completing tasks with highest quality, if they could have built more empathy for Ron's and the players' needs?

The technical team conducted two external playtests of the game late in the semester, knowing that the design team had already tested the core mechanics. These were after-hours, and participation was encouraged but optional. I think that, as with the meetings with Ron, these meetings helped the team to understand better the context and impact of their work. The conventional use of such playtesting is to identify problems, but in this case, much of the design was done before my students got their hands on it. I think that the primary benefit here was not in usability, but in helping my students remember what elementary school students are like, and that there were real end users for this project.

Team members Matt Brewer and Daniel Wilson playtesting at Motivate Our Minds

On expertise

The game is scheduled to "ship" on December 31, coinciding with the end of the ESA grant. Long ago, I had promised Ron that I would help out during the break if there was anything left to be done, and that's what I've been doing this week. Some of the artists are still on contract to finish assets, and so I continue to direct their efforts while also integrating their work into the project, fixing defects, and adding features. Working intimately with my students' code has forced me to think about the nature of novices and experts.

So far this week, I have spent twenty hours working on the project. This is more than the amount of attention I asked my students to spend on the project in a whole sprint, based on the number of credit-hours they earned. There is no definitive ratio that says how much more productive an expert is than a novice, and I've encountered claims from five to twenty. If we take the conservative estimate for the sake of discussion, then we can say that I accomplished this week what I could expect one of my students to do in about eleven weeks of work, or roughly 2/3 of a semester. If we further consider the productivity costs of interruptions—students having to manage four or five other courses, jobs, and relationships, whereas I'm working from home and still relying on my wife to wrangle the boys—then I've easily done more than I could expect a single student could do in an entire semester.

Putting it in this perspective, while I am frustrated to see some low quality code in the project, I think it's important to still qualify the learning experience as a success. Some of them have only been programming for two years, and that has been while enrolled as a full-time student! The students certainly learned a lot about working on an interdisciplinary team, with all the joys and pains that come with leadership, accountability, trust, empathy, and professionalism. They got to build a real software system, large enough that they were able to watch it nearly collapse under the weight of their own bad decisions yet still have time to fix it. They saw the need for something better: for formal state-based analysis, for design patterns, for object-oriented and functional decomposition, for having high standards. They got to see how hard it is to actually make a game, and that tools like Unity3D only barely manage the complexity—there are no silver bullets. They worked in C#, which was a new experience for many of them, and they got a taste for how it is like Java and yet got to tinker with delegates and properties. Sounds like a win to me.

Personal conclusions

I took the better part of the day to write this reflection because I know that by writing it, I would be able to better articulate what I learned from this experience. I have divided my conclusions into two sections. First, the ones that we all already know to be true, but it's good to be reminded:

• Given the opportunity to make something significant, students will rise to the challenge.
• Having a dedicated space is critical.
• Collocation is important, especially across disciplines.
• Face-to-face communication is always preferable.
• Many of the outcomes of immersive learning would go unarticulated without making the time for reflection.

Here are some more specific notes that I should keep in mind:

• I should encourage future teams strongly toward one-page design documents, if for no other reason then to get them thinking about how best to communicate their designs.
• I need to notice when I perceive an affordance that my students do not. When this happens, I need to point out not just what action can be taken, but how I recognized it, so that they can learn to see them as well. 
• I need to be conscientious about modeling professional behavior so that my students can learn to think, design, and communicate as professionals. Scheduling and executing code reviews is a good place to start.
• To improve the impact of reflections and retrospectives, I should find a way to encourage specificity and accountability.
• It is good to keep all the team members in regular contact with the community, including clients, playtesters, and other stakeholders.
• It is better if all team members share goals and motivations. More specifically, it's better if everyone is working for credit or for pay, not a mixture of both. This leads to conflicts of priority within the team, particularly with the natural rhythm of the semester.
• While I can lead a team in the production of someone else's design, I prefer leading teams of students through the holistic design process, from inspiration to finished product.

Current status

As of this writing, the game is in public beta and playable for free online. There is some work-in-progress art that comes up when you cross the Ohio River, but that should be replaced by the end of the day. A few team members have volunteered to do some QA over the break, and I believe all the defects have been ironed out.

The game's main title screen

Students' pride in their work: A CSEdWeek 2012 Post

Two weeks ago, I received an invitation from an undergraduate to come to the presentation of his independent study project. Turns out it was part of a series of presentations connected to a colleague's graph theory courses. Some of the presentations were connected with educational mobile applications, which piqued my curiosity enough that I decided to attend. I was one of three or four guests in the classroom, and the students demonstrated some decent technical prototypes. I provided a little bit of feedback from a HCI and design perspectives.

The experience made me think about the good work that my students are doing. The following Friday, I related the story to my CS222 class, who were a day away from the deadline on their six-week team projects. The teams and projects are self-selected, and this semester, I had three teams, each of which created a video game. I told the students, honestly, that their work was more interesting, and I asked whether they would like me to invite the rest of the department to come to their final project presentations. They responded immediately and excitedly that we should, and so I posted the announcement on the department's Facebook page.

Last Monday were the final presentations, and we brought in about eight outsiders—not bad for a class of twelve students! My recollection is that there were three faculty/staff there and around five undergraduates, all of whom had taken CS222 in the past. This is significant since these student-attendees came in with realistic expectations, having gone through the six-week project in a similar format themselves.

The three presentations went well. I had given them a presentation evaluation rubric ahead of time, and this rubric emphasized three categories: the executable release itself, achievement of milestone #2 objectives, and software architecture. Each of the groups covered these categories quite clearly in their allotted fifteen minutes. Note that game design was explicitly not part of the course: I made it clear to the students at the outset that they were welcome to create games in their six-week project, but that I would only formally evaluate them as software artifacts and not for their game design qualities, since that was outside the syllabus. Still, most of the questions from the audience were related to game design, and the teams provided responses that showed a good general understanding of game design; this is good for me, since hopefully I can recruit these students into my game design and game programming courses in the future!

It is well known that many people fear public speaking, and my students are not exceptional in this regard. I think that in most cases, these students would not volunteer to speak in front of a room of students, faculty, and staff; yet, when presented with the opportunity, they jumped at it. This speaks to the power of providing students with motivating contexts for their work. By giving students the freedom to choose a project that they wanted to complete, not only did I get them to commit to the requisite technical and collaborative tasks—they also gained the important social experience of public presentation.

I know I'm two days late in my CSEdWeek pledge to write a related blog post, but I hope that this post helps point in a fruitful direction. I've been using student-directed projects in my teaching for quite a few years with success, but I don't always make the opportunity for public presentation. Seeing how my students learned from this experience, I am going to try harder in future CS222 offerings to open up the final presentations. By putting it here on my blog, you can hold me to it next semester.