The Story of Collaboration Station

I had an amazing team this past Spring—Space Monkey Studio—and we built an original educational game called Collaboration Station. The rest of the post is my reflection on the semester's activity.

Title Screen

This past academic year, I was fortunate to receive internal immersive learning funding to undertake a project in educational game design and development. My friends at The Children's Museum of Indianapolis served as community partners, and we agreed that it would be interesting to theme games around the International Space Station. In particular, we were interested in how the science of the ISS can be expressed in games that are accessible to kids.

I followed the same format as the last two years, where in the Fall, I taught an Honors Colloquium on educational game design, and in the Spring, I led a game production studio course. The colloquium introduced the students to fundamentals of game design and learning science, and each student created prototypes of original educational games. Teaching the colloquium gave me the opportunity to think about these game designs as well, and so I was able to use some of my own creative processes as a case study: that is, I was engaged in the same authentic learning and design activities as the students, and so I could share some of the tricks and techniques that help me. I had originally planned to share my iterative designs online through this blog, but other work took priority over that; I would still like to experiment in public design at some point in the future, but it wasn't in the cards this time around.

Memory Game: The first minigame

It's worth mentioning that this colloquium was one of the best I have led. It helps that I get a little better at it each time, of course. This group quickly caught on to the rhythm of activities and presentations. Many of the final projects ended up looking a lot like the example games I had students play, but this always happens. For most of the students, it is their first and their last encounter with game design, and so one should not be surprised if the results are relatively simple. The biggest challenge for me in that class is balancing the desire to introduce a broad range of game genres vs spending time on deeper discussions of analysis and design.

During the colloquium, I assembled a few of my ideas with some of the students' ideas to develop the core concept that would become Collaboration Station. In fact, that name was used by one of my students for a completely different project, but it turned out to be a good fit for what we were doing. Here's the introduction section from the game concept document:

ISS Mission is a local-network cooperative asymmetric digital game in which each player takes the role of an astronaut on the International Space Station. Players are faced with authentic ISS missions and have to delegate responsibilities among themselves. If the players are successful, they unlock more challenging missions. If the players fail, public support for science drops and the communists win.

The document cites Space Cadets and Puzzle Pirates as inspirations; not mentioned in the list, but also a critical inspiration, was SpaceTeam.

Sliding Tile Puzzle Game: Gets harder the more you play

I recruited a team of ten undergraduates to participate in the six credit-hour Spring Game Studio course. The team comprised six Computer Science major (one dual major with History), one English major, two Art majors (Animation), and one Music major (Music Media Production). The tech side of the team was relatively young in the major, but I knew that the networking side of the application was going to require some heavy lifting. I decided to do some tech experiments during Winter Break with the goal of having the basic networking back-end completed. I started by using WiFiP2P, which the documentation certainly makes sound appealing, and we began production with this as our target network layer. However, after seemingly endless problems, we switched to Bluetooth, which was much more consistent and reliable. In fact, one of the biggest learning moments for me was reverse-engineering how SpaceTeam is able to determine what local devices are running their service: the short answer is very clever use of Bluetooth device renaming.

I'm getting ahead of ourselves. "Networking is hard" was a theme during the semester, but I want to go back to early January and the first time the team got together. Of the ten, four of them had taken my colloquium in the Fall, which I believe is the most cross-over from any group. I really liked these four: they were reliable, intelligent, and funny, and so I was very glad that they applied.

Sequence-Matching Game: The one with the best sound effects

The past two years, I have tried to lead the studio students through a rapid design process. This led to several problems, which I probably mentioned in my lengthy post-mortems. The two biggest problems are these: that the students' designs are necessarily amateurish, and that students become disenfranchised when their designs are not chosen to move forward. This year, I decided to tackle this problem by taking on the  lead designer role myself. I came in with my "ISS Mission" idea, which had already been vetted and discussed in the colloquium as well as shared with the partners at The Children's Museum. Although this took some creative control from the students, it also meant that we could accelerate our production schedule, which, after the previous two years' experience, was worth the sacrifice.

Our schedule for the fifteen-week semester, then, was to have one week of orientation followed by seven two-week sprints. The orientation week served to help the team understand some of the fundamentals of what we were trying to accomplish, both in terms of educational game design and development methodology. I had been using Buffalo as one of my ice-breakers, and it's a great example because it's fun—it feels like a "normal" party game—but it's also a product of a research lab and has been shown to reduce prejudice in players. However, half the team had already seen Buffalo in the Fall and so they knew the twist, and in looking for something else, a friend recommended Two Rooms and a Boom. I used a print-and-play version of the game, and it was perfect for the job: it got everyone talking to each other, learning their names, working together as well as you can in a social deduction game, and laughing. I will definitely use this again.

Tile Rotation Game: The only one I am good at

For production, I fell back on my old standards: the principles of Crystal Clear combined with elements of Scrum, adapted for use in an academic studio context. I have a paper in press about the academic studio, and how I use it to balance the needs of production with the traditional values of academia; that paper will be published soon in Transactions on Computing Education. One of the primarily elements I use to frame our work is essential questions, taken from McTighe and Wiggins' Understanding by Design framework. I have been using EQs in all of my courses the last few years, and I find them to be a powerful centering and focusing technique. For this Spring Studio's work, I chose these four questions:

• How do multidisciplinary teams coordinate activity to create original software products?
• How does our immersive learning context—creating an original educational video game in collaboration with The Children's Museum—affect our software development practices?
• Why and how do visual elements, audio elements, source code, and writing interact in the process of game development?
• Why and how does the cooperative game principle impact us?

At the end of each iteration, the students wrote reflective essays in which they tied their experiences back to these questions. I asked them to write about specific instances rather than generalities, although some of them needed multiple reminders of this. As I am sure I've mentioned before, I have noticed that my students are much more comfortable writing in unsupported generalizations rather than taking ownership of their writing and addressing the challenges of specifics. However, most of them, when pressed, come to realize that the learning comes from the specifics.

Regarding that last EQ, I have been thinking a lot about the cooperative game principle lately, and at some point I need to make time to write out my thoughts about the strange loops between it, game design, and education. It was certainly the one least selected by students for writing about, but it helped me to focus my own perspective. I had thought about writing my own EQ-centered essays at the end of each sprint as well, but between sprints I also write sprint retrospective reports and curate the product backlog, and these two things took up all of the time I could allocate to this work.

Hold each other accountable for writing unit tests

As I was working on my TOCE article last summer, I was forced to return to a challenging conversation I had with an academic a few years ago. I was talking about my studio-oriented teaching, and I compared it to Lave and Wenger's communities of practice. He asked, "Centered on what?" The question was jarring to me, and it encouraged me to re-evaluate what the studio means from a CoP perspective. I realized that the best game development studio experiences I have had at Ball State have been those that have centered on my practice. That is—trying not to sound megalomaniacal—students did best when they could see how I work, when I could explain myself and guide them, and when I was actively contributing to the project. Based on this finding, I made sure that I was not just a mentor to the project this year, but also a bona fide team member. I think the students appreciated working with me in this capacity, although it's impossible for them to compare this experience to other teams' lived experience. My decision to be an active team member did lead to some internal conflicts between when I should be leading by example and when I should be letting them fail gracefully. At the meso level, decisions such as "When should I pass the keyboard?" in pair programming could become challenging, as I knew I could dump out working code faster than the students, but at some point, I needed to intentionally slow down production by switching to the navigator's role.

A reminder to change partners when pair programming

I read some advice a long time ago that said that you should not start by naming the team, because when you first start, you are not yet a team. This advice has served me well. After our first sprint, the team spent some time figuring out who we really were. The team centered on Space Monkey Studio, and one of our artists put together a snazzy team logo.

Team Logo (non-Kosmo version)

Shortly thereafter, one of the team members brought in a stuffed monkey in a space suit—certainly the coolest contribution to a team since someone brought in Computer Engineer Barbie for the Morgan's Raid project. The monkey was named "Kosmo," and like Computer Engineer Barbie, he was perched on a side of the whiteboard where anyone could contribute captions.

Kosmo gets a name

The rhythm of the semester was quite good. The first sprint went well, and it was designed to be a relatively easy win. The second sprint was where, in my feedback, I pushed back on the students a bit for being lax in their commitments. We had a good rapport, and they took this as the formative feedback it was intended, and the rest of the sprints went very well. Even Sprint six, which you can see from the burndown chart left some work incomplete, was a managed failure: the team recognized what went wrong, and we very quickly righted ourselves.

Seven Burndown Charts

Collaboration Station was developed using PlayN, an excellent open source library for game development that allows for cross-compilation to Android, iOS, and Java desktop. From the beginning, we agreed to focus on Android, since we only had one semester and we wanted to keep our scope limited. However, the game relies on Android devices with Bluetooth, which introduces to problems for automated testing: you cannot simulate Bluetooth within the Android emulator, and deploying to physical devices is slow. I developed a clever workaround where the network layer is abstracted so that you can run multiple instances of the game on the desktop and they will communicate over a local socket, whereas on Android, it will use Bluetooth. The reason it is a socket rather than another IPC solution is that the original networking layer used WiFiP2P, which is based on traditional socket communication, so this was a relatively easy abstraction to build in. I would like to add iOS support in the future, and that's something for which I am currently seeking funding.

I am a firm believer in the power of food to bring people together. Talking about this with the team, one member pointed out that eating together specifically encourages safety: if we are sharing food together, then we must be with people who can be trusted. This team ate like no other, and while many people did their fair share to bring treats, one in particular went above and beyond. The picture below shows our snack table near the start of the semester; by the end of the semester, it was a bounty of sweet and sustaining food—and some of us even knew about the secret stash of emergency Girl Scout Cookies in the cabinet. I brought down my Keurig from the office, which a few team members enjoyed. Others were tea drinkers, and so they brought in the Bunn on the left for heating up water. Our main-treat-contributor kept the team in constant supply of K-cups, teabags, and filtered water, in addition to the rest of the comestibles.

The snack table, with inspirational poster above

Early in the semester, the team identified several learning objectives for the game, drawing these from state standards for 4th through 8th grades. Our intention was to use these objectives to derive the minigames and narrative content. However, early playtesting revealed something unexpected: the players by and large did not know anything about the space program at all. They did not know what astronauts did besides "float around," nor did they know that the International Space Station was a real thing. Inspired by this finding, the team revised our vision away from the content-oriented state standards and more toward the broader goals, that the ISS is real and what astronauts do—still emphasizing the cooperative nature of space expeditions. One of the concrete actions this prompted in the team was to replace the hand-drawn images of our introduction screens with actual photographs of astronauts and the ISS, to drive home the idea that although the game is fictional, its setting is not.

A scene from our introduction, only slightly altered from the original photograph

The game evolved and grew during production. We started with a single mini-game, the memory game. In keeping with the agile spirit of regular delivery, we built a very loose narrative around the original version, where the player had to clean up the experiment area of the ISS. The images were drawn from experiments on the ISS. Over the course of the following twelve weeks, the story, all of the art, and almost all of the code within this minigame would be replaced, based on the team's learning to work together and to interpret player feedback. We added a sliding tile puzzle next. About 2/3 of the way into the semester, we hit stride, and we added the two final minigames: a tile rotation puzzle and a sequence-matching game. This gave us two minigames themed around the science of the ISS and two around the maintenance of the ISS; in the main narrative, then, the players must do both in order to succeed across the three scenarios that comprise the expedition. Other features got smoothed out as well during production, aided by the transition from WiFiP2P to Bluetooth mentioned earlier. Early builds allowed the player to specify their own name, but this was dropped in favor of the current approach, which maps players to countries.

Original Welcome Screen (Game Version 0.1)

I missed not having someone dedicated to social media as I did last year; we have not built up a following around Collaboration Station, so even though we launched a few days ago, we still have very few installations. However, this team did have someone who had a natural talent for local outreach and event planning. I find such work to be stressful, but students to whom I have delegated such work have often dropped the ball. This particular team member really shined here: we had hands-down the best presentations at the Ball State Student Symposium and the Butler URC, and our launch party at the Charles W. Brown Planetarium was a thing of beauty. I find myself wondering how I will recruit someone to do this kind of work in the future. Past attempts to recruit someone with a marketing, PR, and outreach focus has never worked out well, but twice now I have been lucky to have such people end up on the teams regardless of my efforts. I suppose luck is one of my skills.

The Space Monkey Studio from BSU is giving a great demonstration @thebutlerurc pic.twitter.com/HGXsZxTQAp

— Butler URC (@TheButlerURC) April 10, 2015

In conclusion, I am proud of Space Monkey Studio and the work we did together, and I am also happy with the structural changes I made this semester. Taking the role of lead designer allowed us to start production earlier and removed the possibility of students being disenfranchised that their own designs were not chosen over their peers'. Using short, two-week sprints helped us keep a regular rhythm during the semester. Having a team member dedicated to social events and outreach took a lot of that pressure off of me. My being an active team member meant that we could take on a game with significant technical challenges despite the relative inexperience of the undergraduates.

Thanks for reading, or at least for flipping through the pictures! Check out the game on the Play Store, check out the project Web site, and feel free to leave a note in the comments.

What We Learned in CS222, Spring 2015 Edition

In the Fall, I had some nagging doubts about the balance between individual learning and collective learning of fundamental Clean Code concepts in CS222, and so for the Spring, I made a significant change to the first few weeks of the course. I kept the overall rhythm: three weeks of warm-up, a two-week project, and a nine-week project delivered in three iterations. In those first three weeks, I added weekly assignments designed to help students keep pace with the reading and to assess their learning of the same.

I recognize that the ideas presented in the book can be rather challenging, and many students deceive themselves into thinking they understand it when, in fact, they cannot apply the concepts in programming. I do not blame them for this, but rather, I see it as one of the big challenges of CS222 to level the playing field; for example, a student who comes in with relatively little understanding of what a subroutine does still needs to emerge with a stronger understanding of functional decomposition, that a method should do just one thing without side-effects. Hence, I decided to use a mastery learning approach where the three assignments are graded on an S/U basis—either you showed that you understood the assignment or not—and you can turn in one assignment for evaluation or re-evaluation each week of the semester.

I knew that this would increase my grading load, but I was also excited at the prospect of incorporating mastery learning into the course. This model should, in theory, allow me to address my concerns that some students were "slipping through" the course by having their partners do most of the heavy lifting on the nine-week project. However, in order for the assessments to be valid, I needed differentiation: if everyone was evaluating the same code, then it would be too easy for the unscrupulous or the panicked to copy a solution from a classmate. Hence, I reused an approach that I have used in the past, where I ask them to evaluate code that they themselves had written in the past. I also made available the option that they could evaluate open source project code instead of their own, although this ran into the standard problems (which I guess I had forgotten about) that these students don't know what open source means nor how to differentiate a real project from throwaway examples on the Web; the only result of this was that some students wasted weeks writing about low-quality, non-open-source tutorial code they found on a random search.

The last piece of the puzzle was how to ensure that all students actually earned their satisfactory grade on the three assignments by the end of the semester. I decided to make this overt and clear: if you don't complete all three assignments, you cannot earn higher than a D in the course. These assignments, after all, represent that you personally understand the required course content, in particular the use of good names, clear methods that do one thing, readable source code, and the single responsibility principle.

These good intentions led to some stressful times for both me and the students in the Spring. The most challenging impediment was the initial low quality of the code that students brought with them from their previous semester. Most students had relatively little trouble fixing names once they understood the standard conventions (methods should be verb phrases, for example), but beyond that, there were much deeper problems. Their past projects were rife with side-effects, long methods, mixing of model and view, forced garbage comments (such as "// end of for loop"), and no evidence of object-oriented decomposition: everything was static and public. Add to this the fact that the students by and large did not care about the code—neither now nor in the past—and you get rather a difficult slog through the assignments. The third assignment was essentially to refactor the code to follow SRP, but in most cases this required touching literally every line of code. Kudos to those students who got through it, and I know that there were some students who got through the assignments with a much richer understanding of course content as well as much improved metacognitive and code-reading skills. However, this came at a cost of unreasonable time spent by both them and me: there must be a way to achieve these learning and assessment goals with less pain.

At the end of the semester, I dropped the D-cap grading policy and instead instituted a quantum deduction for each assignment not yet completed, and I combined this with an offer for students who were "stuck" on the assignments to send me alternative evidence that they learned the material. Only one student took this approach, but what he did was brilliant: he restated the assignments, pointed out what intellectual work they required, and explained how he had shown his ability to do this within the final project. Right there, we have an example of what I mean: he showed me that he understood specific elements by clearly and proficiently writing about them.

One idea for fixing this part of the semester is to reuse some past game studio project code and give this to CS222 students for evaluation. Unlike their own past code, these projects were ostensibly written to follow Clean Code from the beginning, and so there should be fewer fundamental problems. I am not sure of a fair way to break up a project across thirty sophomores, and doing it randomly could end up with a struggling student inheriting some very bad code. However, this would have the advantage that they would be dealing with real project code that was written by students only one or two years their senior.

During the rest of the semester, I reused by achievement-based grading system as before, where earning achievements unlocks higher grades. I took away the "quest" system for simplicity, but I found myself missing the concept of leveled achievements. I am not yet sure if I will bring that back in the Fall. One piece I have spent some time thinking about is how the achievement system could use more peer evaluation. Reading about digital badges, I know that a common successful practice is to allow anyone to sign a badge, but that the perceived quality of the badge is related to the signer's authority and expertise. I would like to do something similar, allowing the students to grant achievements to each other, which would provide incentives for them to look at and think about each others' work. This could also serve as a high-pass filter, where students only show me their work if it has been vetted by a peer. I spend an unfortunate amount of time writing feedback of the form, "This method is not a verb phrase," and if I can turn that pain into a peer learning opportunity, all the better for it.

In the Fall, I am scheduled to teach two sections of CS222, which I have not done before. I need to think a bit this summer about what that means. Often I come to class with rough lesson plans but end up reacting more to what students are asking about or struggling with. With only one section, it doesn't matter if I shuffle the order of topics based on this interaction, but with two sections, this sounds like a recipe for a headache. I wonder if I should take a more studio-oriented approach, spending more time in class helping students with activities rather than modeling and moderating discussions. I have tinkered with producing videos to model behavior (a story for another blog post), but I find this to be very stale: so much of my in-class modeling is really an interactive performance, feeding off of the verbal and physical feedback of the crowd. If I drop a new keyboard shortcut in a live session, for example, I can see the perplexity and astonishment of the crowd and then talk about not just what I did, but how I did it. If I record a video, it's much more automatic and looks like magic, without students' being able to raise their specific questions at just the right time. On the other hand, the video does make it easier for students to rewind and hear me explain some theoretical ideas more than once instead of relying on their notes or, more likely, ephemeral memory.

I don't regret the changes I made this semester, but they certainly had some unintended consequences. I felt like I had some difficulty "reading" the class, perhaps because it was 8AM and so had more spotty attendance than usual. However, I felt that once we hit stride, I really got to know the teams, and I am proud of their final projects. As usual, there were ambitious projects that had to be scaled back, but really, all of them ended up successful.

At the end of the semester, the students rated these items as being the most important to them:

• Test-Driven Development [11 votes]
• Don't Repeat Yourself [11 votes]
• Working on a team [10 votes]
• Single-Responsibility Principle [10 votes]

Looking at the rest of the 112 items that they listed, there are specific practices identified which make up these top four, such as naming methods with verbs, respecting teammates' perspectives, and model-view separation. As the students came up to place their votes with stickers, a student asked if he could put a sticker on me, since he thought I was the most important part of the class. This was a nice compliment, and I don't think it had ever come up before in teaching this course. As another student heard this, he said that he wanted to put a sticker on my mug for the same reason, but he figured the mug was too sacred an artifact.

I will be revisiting CS222 this summer as I prepare for Fall, but for now, it's time to change gears. There are still many stories from the Spring semester that I would like to reflect on and share here. One of my early-summer tasks is to prioritize the rest of my summer tasks, and I need to be conscientious about making time for reflective writing. In the meantime, however, it's time for some breakfast. Thanks for reading, and as always, feel free to leave a note in the comments.

Getting practice at programming

Around the time that grades are posted at the end of the semester, I tend to get a few emails from my CS222 students, asking what they can do to get more practice with programming. The questions often come from students who got through the first two programming courses without really understanding some core concepts, and these deficiencies start rearing their heads during the nine-week project of CS222. This is a great awakening for the students, and I applaud them for the desire to invest personal time between semesters to improve their skills.

Also, I recently came across—though now cannot figure out where—the idea that we each have a limited number of keystrokes in us before we die. (The concept seems to be tied to keysleft.com, although I only found that today in trying to figure out who I saw writing about it.) From this perspective, all those emails I've sent to students over the years seem like a bad investment, not because it didn't help that one student, but because it could only have helped that one. This is related to the this semester's lack-of-blogging guilt, which I'm sure I will be able to write about in a few days.

Without further ado, let's get to advice for students (roughly sophomores) who want to get some practice.

First, if your main problem in CS222 was with Test-Driven Development, then the next step is easy: read Test-Driven Development: By Example. Then, read it again. The book is deceptively simple, but it contains the seeds for cognitive transformation. Then, the next time you want to write any kind of code, hold yourself accountable to TDD.

If you feel like your Java skills are just not up to snuff, spend some time with The Java Tutorials. The great thing about that site is that you can easily pick the area you want to learn about: the basic language, essential parts of the standard API, building a simple user interface in Swing, and so on. Another advantage of these official tutorials from Oracle is that they are precise and consistent in their use of language and terminology, and this can help you build your technical communication skills. Yes, you can do a random Web search for any of these, but Oracle's collection is well edited and presented.

A few sites have emerged in the last few years dedicated to helping programmers hone their skills with canned problems. Project Euler is one of the most famous, and I know students and alumni who really enjoy the challenges and the community. They make great training exercises if you are interested in programming competitions, such as the ones at the annual CCSC Midwest Conference or the ACM International Collegiate Programming Competition. Personally, I find them a bit too much like homework, but your mileage may vary.

A different take on this kind of exercise is the Code Kata. The idea behind this is that you get to be a better programmer through intentional exercises, exercises that both help you remember what you know and push you to extend your skills.

A talented alumnus recommended cyber-dojo.org, which includes a wide selection of exercises and a community of reviewers. A slightly more aggressive model comes up at codefights.com, which I have not tried but heard recommended by another alumnus.

There are myriad open source projects on the Web. Sometimes it can be difficult to see how to get involved, and I see that many people mistakenly think that "get involved" necessarily means writing production code. This article at SmartBear paints a more accurate picture of how these communities tend to work and how you can contribute, regardless of your programming skill level.

One of the best ways to motivate yourself is to build a thing you care about. Whatever your interest is, you can write programs for it. Games? Figure skating? Home security? Soccer? Whatever it is, think of what you can do at a very small scale, and build it. Keeping the scale small is important, because actually completing a project is much more difficult than just tinkering around with it. I do a lot of work with game programming, and many students come to Computer Science because of an interest in video games. Many of these same students fall into the trap of thinking that the first game they program should be their grand vision for a new strategic multiplayer role-playing 4x roguelike. Don't do it. Start by recreating the classics, like Asteroids, Space Invaders. Keep the scope small, so that you can get the learning that only comes from finishing.

No matter what you do, do it with reflective practice. Think about what you are doing and why you are doing it. Write about it and talk to people about it. Find a community where you can share your findings and ask questions. This will help you develop your metacognitive skills. If you find that you want to learn more about how to learn, I recommend Pragmatic Thinking and Learning.

I hope this list has been helpful. If you have more favorite links, books, and tricks, feel free to leave them in the comments.