This was the second half of a full-year immersive learning experience. I wrote about last Fall's honors colloquium on game design back in December, and that course also prompted my essay on sustainability in games. Two students from Fall's course continued with me into the Spring game production class, and we were joined by nine other undergraduates.
Gameplay Overview
Spirits at Prairie Creek Park is a geolocative game played at Prairie Creek Park, and it is designed for families or small groups. If you open the game while you're not at the park, you get a Google Maps link for directions. If you are at the park, you are given the option to start the game. |
| Title Screen while at t he Park |
 |
| Map Screen while at the Playground |
 |
| Exploring the Playground |
After the timer counts to zero, the players choose what they observed.
 |
| The Playground Checklist |
 |
| Discovering "Scamp" at the Playground |
What Went Well
We received significant encouragement from our community partner. Jason was always available for feedback, and he was eager to share his feedback and ideas. I think he could really see how a unique game like this could engage the kids at camp, and the families they brought to the park, in new ways. Working with a local community partner certainly has some benefits over working with those further out, such as my colleagues in Indianapolis. However, I think it was his enthusiasm that mattered more than his proximity: he treated us like he had a horse in the game, a real vested interest in our success. It helps too that the park and reservoir really are beautiful: my team was inspired by our handful of trips to explore and test.
The team made great use of our technology stack. The game is implemented in web components using Polymer. None of the students had used this before, and many had never even touched Javascript or Web development at all. However, most of them got over the hurdle fairly quickly, learning to deal with a stack that included Polymer, command-line git, npm, HTML, Javascript, CSS, Bower, GitHub, and Travis-CI. We also used Slack and Google Drive for communication and collaboration. I had to get my hands into the code quite a bit, trying to bring along a student wingman whenever I could, and I conducted a lot of the pre-merge reviews, since I knew the most about the stack. I built two versions of our debugging tools as well, which helped the team to internally test features without having to drive out to the reservoir. (Incidentally, the second version of the debugging tools are shipped with the product: click the menu in the upper-right and you can get to the developer options. This lets you play the game from anywhere by simulation your location, for example, which is how I generated those screenshots above.)
It is about a twenty-minute drive from campus to the reservoir, and so the team realized early that we should build a locally-playable version while we work out the kinks in the technology and explore the game design. For more than half the semester, the team worked on a version of the game that was playable on Ball State campus. It was given the placeholder name Spirits at Ball State, which ended up evolving into the final project's name. There were three sites easily accessible from outside the Robert Bell building where we met. We built and tested several paper prototypes that could be used locally as well, as we were sketching in the software architecture. We spent much more time on this version than I had wanted, but this is evidence that we made the right choice since we could fail faster.
Many of the design decisions were driven by a team member with a keen eye for analysis. For example, the number of options presented at each location, the amount of time provided for the senses, and the overall vocabulary of the application were all based on lightweight prototyping with representative players. Two team members took several sketches and ideas to a local library branch to get feedback from kids in an afterschool program there, and this led us to two of the most important design decisions of the game. First, before this visit, we had the player only discovering one spirit at the end of the experience, but we realized that this bit of motivating feedback was better done after each site. The second important change is related: we realized the kids really liked naming the spirits, and they thought of them as working together in teams, and so we added the ability to name the spirits, and the "Heroic Vee" presentation in the end screen.
 |
| The Team at the Butler Undergraduate Research Conference |
I think reaching out to the community to show their work was more important for this team than any other since the actual number of people who can play the game is very slim. In past projects, we could share a link that anyone could play, but this geolocative game requires you to be at Prairie Creek Park. We made a trip to Indianapolis to present at Butler Undergraduate Research Conference, and while students enjoyed the trip, we had terrible placement: our poster was significantly off the beaten path compared to everyone else, and so they had very little engagement from attendees. However, I also scheduled a visit for the team to speak with my friend Cathy Hamaker at The Children's Museum, and this made the whole trip worthwhile. They were able to draw connections between the game design and exhibit design process, hear some wonderful stories about the struggles of good family interaction design, and also see amazing new exhibits while hearing the stories of how they came about.
 |
| The Team and Friends at The Children's Museum |
 |
| The final story map: More on that... someday? |
What went less well
Students love the idea of naming their team, but I stop them from doing this too early in the semester. I remember reading advice years ago that suggested you don't have a team until you accomplish something together, and since then, I have held off on bringing up team naming until the students successfully completes a sprint. By that heuristic, this group of students never actually became a team—and that might not actually be far from the truth. Back in March, I wrote about how the team was struggling to find itself and hit stride, but I don't have much evidence that this intervention had much impact in our worst problem areas. We did not complete any of our six sprints; there were always tasks left incomplete. I believe this is a first for me, to have a group that did not complete any sprints.
We had many discussions in our retrospectives about how to overcome this, including enforcing cross-disciplinary pairs for a week, holding each other accountable to test the game outside rather than through simulators, and posting sketches and works-in-progress to the walls. Some of these things never happened, even when we had individuals tasked with being accountable to them. When I brought it up wearing my Scrum Master hat, the best I got were lame excuses. I'll tell you, dear reader, it was strange, this deep sense of inertia among these students, unlike any I have ever seen. We had a methodology document that laid out particular kinds of ground rules, including rules for conflict resolution. In post-increment essays, students would write about their experiences in ways that clearly violated the methodology, and I would point this out in my feedback... and nothing would change. I read back through students' essays in determining final grades, and for some I saw this all semester: they would submit an essay, I would provide feedback, and nothing would change, and we would repeat this for the next iteration, all the way to the final exam.
I think it would be obvious to anyone observing the studio that we had two cliques. There were four students who served the role of artists, and then there was everybody else. The artists sat together and talked while they worked, but only ever with each other, while everyone else learned to cycle partners and help each other. Our methodology statement embraces focus as a key value and has specific rules about off-topic conversations in the studio, and one of my greatest failures of the semester was not calling people out on this. I was annoyed by the incessant jibber-jabber, but I did not point out the violation early enough, and then I got myself into a position where I talked myself out of bringing it up because I hadn't brought it up before. There is not a clean line between a short off-topic discussion and an hour's rambling, but I know I should have held up my Scrum Master responsibilities here. I do not know if other students saw this as my failure or not, or if they even saw it as their own failure for not speaking up, but nobody ever did, and so we did not have a good environment for focus.
The team suffered from serious asset pipeline problems. From the beginning of the semester, I was telling the team that everyone had to be able to be able to evaluate and integrate their own work, which means being able to run the game locally and push changes to git. There was a split almost exactly along clique lines, and even after multiple discussions, it was clear that most of the artists could not run the game or integrate their work into it. This means that they were unable to evaluate their work in context, which is, of course, ludicrous, but it's like the point above: there was a split, a miscommunication here that still puzzles me. More than once, we had artwork uploaded to Google Drive or Slack at the 11th hour, or even slightly—or grossly—after the deadline, that then someone else would have to integrate. Slack and GitHub were great for coordination among team members who were responsible about keeping up with them, but other students simply did not keep up—and their essays reveal that they did not think it was even their responsibility to do so.
Finally, as for the game itself, I think what we have is an interesting and novel design that encourages conversation about the environment and outdoorsmanship, being technology-mediated without being technology-focused like most games are. However, what we shipped is really just a first working prototype in many ways. There are many opportunities to improve the play experience without too much extra effort, such as replacing the digital timer with an animated one, animating the introduction of the spirits with some simple sound effects for more "pop," improving the overall look-and-feel of the application, and introducing more variety in the discoverable spirits. There is also the overarching narrative issue: why are there spirits at the park? In the final screen, what exactly are they protecting the park from? From an education point-of-view, we don't have any solid evidence or anecdotes to indicate that the game produces learning outcomes in the students since we ran out of time for any on-site testing with families. To be clear, I believe what we have is good, but we have only scratched the surface.
 |
| At the Reservoir |
An Interlude about Llamas
With just a few weeks left in the semester, a subset of the team took it upon themselves to name the themselves. In a flurry of creativity, one of the artists whipped up an 8-bit flame llama, and a few of the students fell in love with it. It quickly became a mascot, at least to this subset, and so we dubbed ourselves Flame Llama Studio. A recurring theme in the studio among the programmers was that they would forget to update bower when dependencies changed, and so we coupled the flame llama with a reminder/slogan for a cool T-shirt idea.
We needed something that expressed the name of the studio as well, so the same artist made a studio emblem.
Conclusions
There's this lingering question I have had since near the end of the semester: how did we get into a position where four artists produced a total of ten shipped art assets in fifteen weeks? I was excited to have so many talented artists coming into a team: how did they produce so little? The best answer I have is that the artists—or at least most of them—never really saw themselves as members of the team, but rather as contract artists. If all they had to do was make art, then their role was to wait and be told what art to make, then to make it before the deadline. Indeed, looking only at the outcomes, this is what they did: they produced only the things they were specifically told to do. By contrast, if they understood that they were members of an interdisciplinary cross-functional project team, then they would have seen that they were expected to do much more than just produce art. There were a few points during the semester where some of the art clique were working on art that was not currently needed, while there was important work to be done for the current sprint, and in these cases I pointed out that we didn't need them to spend time on things outside of our current goals. Somehow they took this as, "Don't draw anything or show initiative," instead of, "We need all hands on deck to solve our immediate needs." Again, if a student's vision of themselves was as only being responsible for art, then the former sounds like what you might have heard, instead of what was actually intended. I am still unsure of the root cause of this misunderstanding, which bothers me, because without a good root cause, I'm not sure what actions I can take in the future to prevent this problem. (In truth, this starts to push into the realm of what I suspect vs. what I can blog about.)
I am reminded of a phrase we used a lot in my household when I was young: Missing the Big Picture. Here are two quick examples. One of the things we talked about doing was making coloring pages for kids that would have the game's URL on them. The last week of classes, a few team members said they could work on this. What they did was take spirit drawings and remove the color from them, leaving line drawings, ... and then they stopped. Of course this is a necessary step to making coloring pages, but it misses the big picture: you don't make coloring pages by just removing color from PNG or JPG images. A similar story: a subset of students worked on a poster for our campus showcase event, and just before they finished they asked for my feedback, and I saw they had designed it in Photoshop. These two short stories paint a picture of students who focus on doing what they know, rather than asking whether it is the right thing to be doing.
We can take "Missing the Big Picture" a bit more scientifically and apply the lens of the Dreyfus Model of Skill Acquisition. Being a novice or advanced beginner in this model means that you don't want to see the big picture and that you cling to the rules without considering the context. Of course, second-order ignorance is a defining characteristic of being an advanced beginner: I don't think the decolorization or the inappropriate use of Photoshop were malicious, but they were certainly ignorant. Perhaps then this is a call to action for me as a mentor to find ways to improve my awareness of the activity of the studio so that I can intervene more quickly to find the Zone of Proximal Development: find that critical difference between what students can do by themselves vs. what they can do with my assistance, and then help them advance from there. I think this is especially hard on a team where there is generally low knowledge about all the steps required, although this is a situation I have had the past few years in the Spring Game Studio. I am excited that in the Fall, I will be able to teach Game Programming before doing another Game Studio course in Spring 2018, since I think this kind of sequencing can help raise the bar of student understanding. Then again, maybe it's a false hope, since last year's excellent work didn't have this benefit, and it wasn't clear that the two students from last Fall's game design course became well-informed leaders of our design process. Maybe it's just more puzzles, and it still just has more to do with having the right people regardless of their backgrounds. It's a conundrum that despite years of doing this kind of work, I'm still not sure how to approach. My next step will be to take a closer look at Google's Re:Work, to see if I can find any evidence of local trends that would align with or conflict with some of their findings.
 |
| Official team photo with Amazing Backlit Action and Wall Phantoms |
During this semester, I listened to some Robert Martin and Allen Holub presentations while painting, and these got me thinking more about how agile principles can be deployed with students. I was particularly struck by something Holub said during his "Death of Agile" talk: "This is a process for adults." To use the popular vernacular, some of my students have serious issues with adulting. I have written several papers on the use of agile techniques in undergraduate teams and I have read many more, but I do not remember this particular perspective brought up before. Another point Holub makes in his talk is that the fixed durations of Scrum's Sprints are decidedly anti-Agile. These two pieces have been turning around in my mind. Does Scrum work with students because it is not agile? Given a team that cannot complete a sprint, would they do better without the artifice of a sprint, or would they do worse? I had a similar thought after reading Kent Beck's excellent Extreme Programming Explained 2nd edition: can a team of novices go full-XP, or would their lack of experience cripple them? I will point out that these are all empirical research questions, but I don't have the resources to conduct the experiments or case studies necessary to investigate them, so I'm left in the land of craft rather than science—not necessarily bad, but also not provably good.
I know that most of the team had a great learning experience. I have observations of their activity, their essays reflecting on experiences, and knowledge represented in the artifacts themselves. Truly, in addition to the normal learning I expect, I saw some great transformative moments for students—but not for all. In determining students' final grades, I went back and reviewed their essays through the semester leading up to the final, which consisted of two reflective essays. Some did not actually show any signs of having learned much of anything, neither about the interdisciplinary goals of the course nor about how to learn from reflective practice. I re-read the feedback that I gave them as well, and it was sad to see how some of them apparently did not act upon any of it.
Several members of the team were sophomores and juniors, and this means that I can continue to work with these students during their time at Ball State. Several have signed up for my Game Programming course in the Fall, and I look forward to getting them into leadership positions where they can draw upon their experience in the studio to foster success. Next year, I will also be continuing my own explorations into geolocative game design and interactive narrative—a topic I expect to blog about soon, so I'll link that in when it's ready.
The number of people who will ever play Spirits at Prairie Creek is pretty small: just getting the word out about any digital product is a challenge given how much stuff is available online. For us, we need players to not only know about it, but to also be at specific place to play. We have Google Analytics tracking plays, and I'm hopeful that Camp Prairie Creek will help spread the word about the game. In the meantime, we have an interesting game design, although it's not clear if I can turn this experience into a conventional scholarly article. I think there's round here for a great empirical evaluation, but I lack the resources to conduct that myself; maybe someone will be interested in doing this as an honors thesis.
And finally...
An easter egg for those who read this far, a late-semester alternative rendition of the team's slogan:
Thanks for reading! As always, please feel free to share your feedback in the comments. If you try the game, let me know what you think!
No comments:
Post a Comment