Tuesday, April 30, 2019

Happy Accident Studio and Canning Heroes: Reflecting on the Spring 2019 Immersive Learning Project

It was a great semester in my Spring Game Studio class. I had more applicants than ever before, thanks in part to recruiting assistance from the Office of Immersive Learning. It made it harder to select a team, but it also demonstrated the demand for this kind of experience. I ended up accepting six Computer Science majors, one Music Media Production major, two Animation majors, one Journalism Graphics major, and one English major with a Computer Science minor.

This class worked with Minnetrista in the second semester of a two-semester sequence. They inherited a paper prototype that was selected by me and our primary contact at Minnetrista, George Buss, the Director of Experience and Education. The specific prototype was inspired by Cooking Mama and involved players preparing and cooking foods related to Minnestrista's herb garden and Farmer's Market. The Spring class was able to take this project in connection with the renovation of the historic Oakhurst Home, which is where the original recipes of the Ball Blue Book were developed.

The end result of this work is Canning Heroes, which is a cooperative game for 1-4 players. It is designed to be played on a large touch screen, and our partners at Minnetrista are currently working on the logistics of installation in the Oakhurst home. The source code is available on GitHub, and the executable can be downloaded from itch.io. This is my first time releasing anything on itch.io, and I was pleased with how easy it was to post the game and add student collaborators to the project.

The students dubbed themselves "Happy Accident Studio," which is an homage to Internet-culture icon Bob Ross, who recorded many of his famous painting shows in Muncie right on Minnetrista's campus. I am very pleased with the work done by Happy Accident Studio. Like the best of my immersive learning teams, they were able to learn quickly how to work together and hit stride about ten weeks into the semester. This allowed us to make the necessary revisions in the tail end of the semester to really make the game shine. I think Canning Heroes is the most polished product to come out of my immersive learning classes, and I am hopeful that it will be installed at Minnetrista for public consumption.

Happy Accident Studio was a good team by any metric. They had their flaws as any team will, but overall, they really came to understand what multidisciplinary collaboration means and why it is necessary for this kind of risky and creative work. They struggled with accountability, as student teams often do. They had great eureka moments, particularly around playtesting and code quality. It turns out that except for two seniors, the rest of the team was juniors. This means they will be around next year, and it makes me wonder if we can make some hay while the sun shines.

This was the second immersive learning team I have mentored who have used Unreal Engine 4 for their projects, the previous one being last Spring's Fairy Trails by Guy Falls Down Studio. There was one team member from Spring 2018 who was able to join again this semester, and it was good to have another voice with some experience in the room. He and two others had done some work with UE4 before, and they were enough to seed the learning experience for everyone else. Once again, I was generally happy with how quickly students were able to learn the tools of the engine, particularly Blueprint programming. The team got up and running quickly, and the mistakes they make—such as poor functional decomposition or bad naming practices—are general code quality problems that dog all novices. That is, there was nothing lost by using Blueprint, and much to be gained. Also, whereas Fairy Trails was written entirely in UMG with an enormous amount of effort going toward platform support, Canning Heroes makes better use of the UE4 platform, even though it's entirely "flat" using Paper2D.

Reflections on Student Assessment

Two stories from this semester have me thinking about whether I want to change my student evaluation strategy in future immersive learning projects. This team experienced many of the common successes and failures during the semester, but these two stories stand out in my memory.

First, there was an artist who wanted to create tutorial animations for the game. Her instinct was to create them in After Effects so that then we could import them into the engine. I suggested that, instead, she learn to create the animations in-engine. The tutorial was already using assets that we had in the game, and if the animation were done using keyframes and tweening in-engine, then anyone could tweak them as necessary, for example if the assets or timing needed to be changed (both of which happened, of course). By contrast, if she had created the animations using her familiar tools, any change would require going back to After Effects and repeating the render and import process, decreasing the number of people who could make the change while increasing the bus factor. To this student's credit, she jumped right in, and within a few hours was as adept with creating in-engine animations of 2D assets as anyone else who had been using UE4 for months. In our final retrospective meeting, she brought this up as a great learning outcome of the course: that there are times to use familiar tools and times to learn new tools, and not to be afraid of either.

The second story is not quite so positive. Our music major was de facto audio lead for the team, being in charge of both original compositions and sound effects. Several times during the semester, I encouraged her to do integration work, following the value of interdisciplinary collaboration and multidisciplinary pairing that is explicit in the methodology. However, even at the end of the semester, her approach was to upload files to the server and then step aside: she could not complete the loop of creating a new sound, putting it in the game, and evaluating it. A related story is that throughout development, many team members desired to randomize the sound effects. Again, I encouraged the team generally to look into UE4 support for this, but it was never pursued. It turns out that just after the game was complete, I watched an archived UE4 livestream that prompted me to look more into the audio system myself, and I was crestfallen to see that what the team wanted could be done with a single node of sound cue. I am certain that, had this student shown the ambition of the artist from the first story, we would have a much better aural experience in the game today.

Reflecting on these stories made me wonder whether there was some kind of more formalized incentive that I could use to nudge the students toward manifesting the attitudes and behaviors I desire for them. Several years ago I looked into KPIs as an industry practice that I could adopt in the studio to maintain project authenticity, but my investigations at the time led me to conclude that the timeframe of KPIs doesn't match conventional course structure. Regardless of my pedagogic choices, I am still locked in a three credit-hour class that meets three times a week for fifteen weeks, which doesn't seem to have the breadth required for Google-style KPIs. However, I have also recently been tinkering with specifications grading, particularly toward the goal that grading criteria become unambiguous. At least in theory, removing ambiguity from the grading scheme should help students plan their actions prudently.

The alternative idea that crossed my mind, which I share here in part as an excuse to explore it further, is paths for team members. "Paths" at least is the word that came to mind, informed I suspect by my anticipation of the new Pathfinder Adventure Card Game Core Set, but alternatives might be "roles" or "quests." Keep in mind that I manage the team using a philosophical model of team ownership inspired by agile practices. That means everyone owns everything: anyone can work on art, sound, story, programming, UI, and so on. Occasionally I have teams where one aspect of the game requires a lead who approves work in that area, but that's always in response to dysfunctional team members. Most of the time, I can get students on board with the idea of total team ownership and its concomitant responsibilities.

Paths might offer a concrete way to inform students about expected behaviors, potentially annotating them with incentives (that is, grades). We might start with something like Path of the Novice, which would include incentives for reading the team methodology document, getting accounts set up on Slack and whatnot, putting something on version control. From there, maybe one could branch out into:

  • Path of the Artist: create an asset, put it in version control, and integrate it into the game.
  • Path of Code: conduct a code review of someone else's work; have your work subject to code review.
  • Path of Quality: run playtesting sessions for the game, documenting the results
  • Path of the Robot: automate the game build using continuous integration 
Obviously, these are just off the top of my head. Perhaps students could choose a path each two-week iteration and provide evidence at the end about whether they met their goals. Perhaps it really needs levels of achievement. The clear problem that I can see is that it might constrain people in a way that total team ownership is designed not to: it may trick a team member into not contributing what the team needs at some critical point because it's "not their job" (i.e. not on their path). Maybe, then, the result of this reflection is that all I really need is a simpler rule: everyone needs to be able to integrate and test their work. Unfortunately, that really devolves into the existing rule, "Follow the methodology," which is missed by a subset of team members. 

This whole discussion of how to manage future projects has to be framed by Price's Law. I learned a year or two ago that there's a general rule that half the work is done by a square root of the team. It seems that this has strong implications for a group like my immersive learning class. Which disciplines are represented under the square root limit can strongly influence the direction the project can go. Yet, at the end of the semester, I still have to assign grades, and they should be done as fairly as possible. It's not the goal, then, to ensure that everyone produces equal outcomes, but rather than their work is fairly assessed.

What's Next

I've been doing the two-course sequence of immersive learning game development projects since Fall 2012, but I'm taking a little detour in 2019-2020. I received approval from my department to offer my game design class as a special topics class for CS majors and minors in the Fall, so I'm going to try that instead of teaching it as an Honors Colloquium. What I would like to do, as I wrote about two weeks ago, is try to transform my game design course into a regular CS course, which would make it much more sustainable. Fall's plan is set, then, but Spring is still up in the air. As I mentioned above, I would like to take advantage of the fact that I will have more students than ever who have experience with game design and development, but it will be a little while before I can determine what shape that will take.

2 comments:

  1. "You can lead a horse to water but you cannot make it drink." I think your suggestions about ways to better "motivate" the students have merit but as you mentioned flaws can be found. My question is - do we really want to eliminate this type of behavior or do we want to leverage it as a teaching experience. We have all seen/heard/experienced working on teams (professionally) in which the workload or effort is not distributed appropriately, is this an instance where we can leverage this directly at those students that may have "something more going on" to highlight the importance of this interaction?

    Congrats on a successful semester!

    ReplyDelete
    Replies
    1. Good question, I'm definitely on the "make learning experiences" side. I struggle sometimes with how to call students' attention to such things in a way that maximizes fruitful learning and minimizes social pain. Really, the easiest case may have been the semester where I "fired" a student, and then the rest of us could talk honestly about what we had seen, heard, and thought---and what to do about it---but without worrying about hurting that student's feelings, because they were already gone.

      Ideally, the team methodology leads to team accountability, but in practice, that's really hard for anybody, and I think it's especially hard for students. The more savvy students recognize when the team is failing to hold itself accountable. I'm not exactly sure what the less savvy ones perceive.

      Delete