Tales of Preproduction: Refining the prototyping procedure

I am teaching the Game Preproduction class for the second time this semester, and this time I am joined by Antonio Sanders as a team-teacher from the School of Art. There are already a lot of interesting things happening as we have a class that is now have Computer Science majors and half Animation majors. We have also extended the class time to a "studio" duration, so we meet twice a week for three hours per meeting instead of the 75 minutes I had with my inaugural group last year.

Given that quick summary of our context, I want to share a significant change that we made to the prototyping process from last year. Last year, the team adjusted the schedule because we didn't dedicate enough ideation time to prototyping, and so this year, we set aside five days for this. Each day, the students are supposed to bring in a prototype that answers a design question. I remember this also being challenging last year, and it wasn't until late in that process that we remembered the seven questions that Lemarchand poses about prototypes in A Playful Production Process. 

In an effort to get the students thinking more critically about their prototypes, we have required them to write short prototype reports that address Lemarchand's seven questions. The last of the questions, which Lemarchand himself typesets in bold to show its importance, is, "What question does this prototype answer?" What the reports help reveal, which was harder to see last year, were cases where the questions themselves were either malformed or unanswerable. That is, students are going into prototyping without a good idea of what prototyping is. Several times, I've seen students show their prototypes, and when I ask what design question they answer, the students have to look it up on their reports. This is pretty strong evidence that the questions were developed post hoc. What's most troubling is that, after having completed four of the planned five rounds, these problems are still rampant.

Early in the process, my teaching partner suggested students think about design questions in the form "Is X Y?" where X is a capability being prototyped and Y is a design goal. For example "Is holding the jump button down to fly giving the player a sense of freedom?" While this heuristic proved helpful, a lot of students struggled with it: in part, I think they didn't understand that it was only a heuristic, and in part because they haven't practiced the analysis skills required to pull a design question out of an inspiration. If I were to use this again, I'd follow the obvious-in-retrospect need to rename those variables, to something like "Does this player action produce this design goal?" (Unfortunately, the discussion of design goals comes up later in the book, so maybe even this idea is too fuzzy for the students.)

Many of the questions that students want to pursue are actually research questions. I mean this in both the colloquial and the academic senses. A question like, "Does adding a sudden sound make the player scared when they see the monster?" is obviously answered in the affirmative: one need only look at games that induce jump-scares to see that this is effective. Questions like, "Do timers increase player stress?" are simple design truisms that are not worth prototyping. In yesterday's class, I tried to explain to the students that if the question is generic then it's a research question, and that design questions are always about specifics. In particular, through science, we approach generic questions through specific experiments that attempt to answer the general question; through design, we answer specific questions through specifics directly.

Reflecting on these problems, it becomes clear that the earlier parts of the semester were not goal-directed enough. Students acknowledged after our in-class brainstorming session that they were not brainstorming game ideas (but that's a topic for another post). When the students did research, many of these were also not goal-directed. Now, in prototyping, we're more easily to see what students are interested in, and we can point out to them that their interests and issues can and should be solved by blue-sky ideation or by research. However, we haven't baked that into these first five weeks. Put another way, we took a waterfall approach to ideation whereas perhaps next year we should try an iterative one.

We're in the process of collecting summaries of all the students' prototypes. I put together a form that uses this template for students to self-describe prototypes that are viable for forming teams around:

This game will be a GENRE/TYPE where the player CORE MECHANISM to GOAL/THEME. 

I'm eager to see if this was a helpful hook for the students. I will have to ask them about it on Tuesday and then see if it's something we can use with next year's cohort.

Notes from the Inaugural Game Preproduction Class: Draft Schedules

 Last week, my teams presented their draft macro charts. This week, they presented their draft schedules. All of the teams made good progress here. I showed them how to use pivot tables to figure out how many hours they had for each priority level; they could use this for task assignment as well, but none of them had yet gotten into the level of detail required. We had some useful conversations about the relative merits of pairing or mobbing on some tasks while parallelizing others.

One of the groups included four formal playtesting sessions, not for validation of any particular feature but for general feedback. The other two groups recognized that this is valuable for scheduling. This led us to a discussion about the kinds of things that take time that were not yet in their schedules. These are the categories we identified:

• Creating a store presence
• Playtesting
• Organization and administration
• Release engineering
• Community building
• Legal / Business / Tax
• Marketing (posters, handouts, YouTube, release party, shirts)
• Convention travel
• Dealing with contractors

I look forward to see how the students incorporate these into their final schedules. As they work on their schedules, their macros, and their vertical slices, I think they are really coming to understand how many different pieces need to come together for this to work.

As we were wrapping up, one of the students commented that, although he is quite fond of his project, he thinks our best bet would be to kill two of them and all work on one. I consider this a good result, as the student is seeing how much work even one of these projects will require.

Notes from the Inaugural Game Preproduction Class: Commencing with a More Aggressive Schedule (and something about 4 C's)

One of my tasks for the day was to take my digital notes for CS390 and turn these into activities for the students to do. Up to this point, I had a reading schedule in mind, but my notes about the kinds of activities I might ask students to do outside or inside class were unrefined. I began working on it this morning, but very quickly I came across something troubling in the schedule: my original plan had us reading portions of the textbook and doing supporting exercises up through the end of March, which only left less than a month of dedicated time for teams to pull together a vertical slice, a game design macro, and a schedule. Yikes.

I took this to class today, and we spent half of the day discussing plans for the future. I suggested that we could keep to the original plan, giving the benefit of building a deep understanding of the ideas in the reading while also having the drawback of less time for hands-on work. I proposed that we could flip this around, cramming about 120 pages of reading into very little time, basically just getting used to where things are in the book, and then devote ourselves to our immediate design problems. I also suggested a middle path. That is the way most of the students wanted to go, and so that's what I just updated on the course plan.

The timing of this turned out to be in our favor. The readings naturally clump into themes that correspond to the three deliverables. Hence, our next three meetings will involved focused discussion on the vertical slice, the game design macro, and the schedule, respectively. 

Looking back, I wonder if we could have followed a more aggressive reading schedule from the beginning. I enjoyed a leisurely pace through some of the chapters and the opportunity to explore things like automatic writing, brainstorming, and mind mapping with my students. Some of these were more useful than others, however, and I've already written about how these things didn't help us converge during the ideation phase. Next time I teach the course, I will have to give this careful consideration.

The expectation that I had for the students today was that each team explore what Lemarchand calls the Four C's: Core loop, Character, Camera, and Control. Honestly, I thought this would be a slow pitch for the student teams, but when we got into the discussion, I wished we had the whole 75 minutes for it. Having to express these four concepts got each of the teams to have important discussions, some of which were clearly not quite resolved at the time of the presentations. There wasn't direct disagreement, but there was definitely uncertainty. Also, there were a lot of good comments from the other team members (and me) about some of the decisions and the way they were articulated. I had a hard time expressing to the student when I was giving recommendations about process and when I was giving advice about design, which makes me wonder if I ought to be distinguishing those more clearly. Maybe I'll ask the students what they think about that.

Notes from the inaugural Game Preproduction class: Divergence followed by Divergence

This is a follow-up to my previous post, in which I focused on particular problems that arose in the prototyping process from the inaugural offering of CS390. Here, I want to take a different perspective on the last few weeks and share some thoughts about the course itself.

The preproduction approach described by Mark Cerny and codified in Richard Lemarchand's book maps well to the Double Diamond model: diverge, converge, diverge, converge. The first diamond is ideation and the second results in the vertical slice. Yesterday, my students got together for our first big convergence meeting... and we got nowhere. What happened?

As laid out in the course plan, we spent January getting started by framing our work with the first several chapters of Lemarchand. February included some reading but mostly a major brainstorming session followed by the development of prototypes: first paper prototypes, then digital, then a 2-week period that was supposed to be intensive, variable-mode prototypes, leading up to the big convergence on February 16th.

Even while we were in the middle of the prototyping, I noticed that we weren't converging. I mentioned it to the class, and I got a general consensus that they understood what I meant, but it wasn't obvious that behavior was changing. The students' work that I observed from their prototyping fell into two categories. One group of students went head-down into one idea and never came out of it. Some got locked into a theme and others into implementation details, but in either case, they didn't explore the design space—they never diverged. The other group of students continued to diverge: each prototype picked up a completely different idea, some not even related to the original brainstorming session. This meant that at our Feb 16 meeting, when I asked them to share what directions they had in mind, some of them were still locked into a single concept while others became enamored of something that we had just seen the previous day. 

It should be clear why that first group is carrying risk: they have not explored the design space. The reason the second group carries similar risk is that they are hooked on an idea. Keep in mind what I wrote about yesterday, that the ostensible prototypes were not actually answering important design questions themselves. While I could be wrong, and more careful research would be required to prove it, I have strong faith in my hypothesis that students were still getting caught up in ideas rather than findings. They had not found the fun in a concept but imagined that a concept must be fun. 

At the end of our 75-minute conversation (which I myself derailed at least once, and which I had to pull back from the students several times), we ended up deciding to do another round of digital protoyping, this time in pairs. In order to help them proceed, I'm requiring them to answer Lemarchand's seven questions. I am hopeful that this will help us move forward, even though we've already passed the 15% of project budget that Lemarchand recommends for ideation.

What makes me hopeful? After our meeting on Thursday, I wrote an announcement to the students trying to give a little more basis for what we ought to be seeing. This prompted a student to reach out to me, to share with me a possible design question for a prototype to answer, and to ask whether it was a good question. It turns out it was not a good question... not yet. It showed that the student was moving in a fruitful direction, though, and so after a lot of typing, I was able to explain why that question was not good yet, but how it could be modified. I took this asynchronous conversation, removed the particulars, and reposted the essence of it for all the students. Some of it felt like hard love, but I held out hope that it was what they needed. This hope was validated later in the evening, when a pair of students reached out to talk about their design question. It turns out that their question, which they explained came after spending 1.5 hours reviewing the messages I had sent and the framing from the reading, was excellent. It was not just good; it was qualitatively better than anything that had been shown or discussed in class up to this point. I hammered that point home in our conversation, and I think they understood me.

My point, as ever, is not just to share stories, but to use the stories to reflect on my experience so that I can improve. Here are a few things that I need to think about doing differently in the future.

• Collectively reduce the brainstorming output. Retaining all ~100 concepts throughout ideation pushed students toward continued divergence. One of the students pointed out that we could have had some kind of elimination strategy to help us see the space reducing. For example, simple voting could identify top contenders, and prototypes would answer the specific design questions.
• Encourage working in pairs or small groups. The way I framed the problem, it sounded like students had to work alone, but I would have accepted group development. Working together would mean more fruitful discussion and more shared ownership: it's not Bob's idea, it's our idea. This also means students with less production skill could pair up with someone who could show them the ropes.
• Spend more time on the problem of question identification, as previously mentioned. This could be done by workshopping questions together such as when returning to the big list of concepts, and it could also be done by requiring students to use Lemarchand's seven questions.

Course Planning: The First Offering of CS390 Game Studio Preproduction

One of the most exciting aspects of my department's new Game Design & Development Concentration is a three-semester capstone experience. Eventually, this sequence of courses will be team taught through Computer Science, Animation, and Music, but since my department is a year ahead in the curriculum approval process, we will be having a CS-focused sequence the first time out.

The first course in the sequence is CS390 Game Studio Preproduction, and I am teaching it for the first time this Spring. I have spent many hours prepping for this course, and this week, I am finally feeling confident in my design decisions.

As the title implies, this is the preproduction experience for the production courses, which will be a sequence of senior-level courses offered for the first time next year. The success of production hinges on good preproduction, and so there's a lot of pressure in CS390 to get things right. I decided to use Richard Lemarchand's A Playful Production Process as the primary text for the class, and we'll be reading through the book and completing exercises from it during the semester. The advice in his book is that 15% of time be spent on ideation, and mapping that to a three semester sequence means that my students will spend the first six weeks of the semester in that phase. Coming out of ideation, I expect the students to be able to form teams around ideas as they move into preproduction proper. At the end of the semester, each team will be expected to have a vertical slice (or at least a beautiful corner), a game design macro, and a production schedule.

The semester will culminate in the student teams' presentations to a Game Proposal Review Board. This board of industry professionals will give feedback to each team and have the power to approve or reject each project. The approved ones will be the ones that move forward into production next year. Students on rejected projects will be reassigned to other teams so that we can produce the highest quality outcome. I already have two board members signed up, including one alumnus, and I am waiting to hear back from one more.

Lemarchand's book gives a good intellectual framework for thinking about preproduction, but I was still left with many course design decisions. One of these decisions was how to grade the students; since I am required to give a grade, I need to make sure it is done in an appropriate manner. I have been thinking about how to get more results out of the efforts I put into grading, and this course provided an opportunity for me to think more about that. In particular, I would like a little more clarity and a little less of the reduction in implicit motivation that comes with grading. As a result, I'm taking a page from my colleague David Largent's handbook and trying a more by-the-books approach to specifications grading. 

I have set up categories in which students can earn "tiers" (grades) based on their work and dedication to the class. The easy two to set up were participation and exercises, the first of which deals with classroom engagement and the latter with individual assigned work. These are all graded on an S/U basis, and satisfactory performance unlocks higher tiers. I will do something similar for the four end-of-semester outputs, but I want to be able to think more about those and also to negotiate the details with my students. For the final category, I added achievements. As in CS222, I was inspired by the idea of rewarding students for doing authentic work that excites them, but I did not use the same formal system as in that class. Instead, I set it up so that the achievements are harder and, essentially, someone either completes one or not. Then, in the final course grade, the achievement acts as a sort of buffer, allowing students to gain higher course grades. A student who does no achievements but aces the rest of the requirements will still get a well-deserved A, while a student who slips up in another category can use an achievement to still demonstrate excellence and get a well-but-differently-deserved A.

The class is actually below the usual enrollment threshold, but because it is part of a new program, the dean has approved it to go forward regardless. I know all the students who are signed up since they have all come through other courses I have taught, and so they all know that we're in for some serious business and hard fun. A couple of the students are also taking my immersive learning CS490 class, which means they are actually taking more than the normally allowed number of games-related courses since they are sitting in a transitional curriculum—good for them!

As usual, I have put my course plans publicly online, and you are welcome to read them. I am looking forward to getting to work with these students and see what they can make.