Wednesday, May 11, 2016

Parting thoughts about Studio 368

As I finish up my obligations from the Spring semester, I want to take some time to reflect on my Spring Game Studio course--the group that would come to call themselves Studio 368 and produce the original educational game, Traveler's Notebook: Monster Tales. I have written some about this during the last few months, so here's a collection of relevant links:

There are a few topics that I have not written about, so I will take the opportunity here to share them.
The sign on our door
By the way, that little monster on the top of the sign there is Debbie--full name, Little Debbie Snackcakes. One of our team members sketched her onto our "Keep Calm and Find Monsters" poster (which sadly I don't seem to have), and she became our mascot.

Multidisciplinarity

This semester had the best mix of majors I have ever had in a studio course, better even than the mix at the VBC. The team included: four Computer Science majors; one Telecommunications major with an emphasis in audio production, who also had a CS minor; one Philosophy major with a CS minor; one Art major specializing in animation; and three English majors specializing in Creative Writing. There were some tasks that people fell into pretty naturally: the CS majors got right into the programming, the artist into illustration and visual design, the English majors into writing. Our audio was all designed and integrated by our Telecommunications major, and one of the English majors also contributed significantly to the visual design, drawing upon her minor in Professional Writing (which includes a significant media design element).

Those are some of the common discipline-oriented production tasks involved in game design and development, but there are also important interdisciplinary aspects. Here, the team really came together and took initiative. Some of the students joined me in writing a scholarly article about our work, which is currently under review. Several contributed to the visual design of the game over multiple iterations. When it came time to conduct playtesting sessions, our Philosophy major showed her humanistic and analytical prowess in designing, executing, digesting, and disseminating our evaluation protocol. About halfway through the semester, we became aware of fundamental design problems, and this resulted in a beautiful, rapidly-developed paper prototype for a new design that was assembled by a CS major and our Philosophy major. This particular artifact served as an important intermediary work product, serving to align the team's disparate visions about where the project was heading.

Working on the paper prototype

Technology

While I have been using GitHub and Travis for projects since last summer, this was the first time I worked with more than one or two other people using this technology. It was a little spotty at first, in part because so many of us were a little green, but once we agreed on how to coordinate feature branching, we hit stride. I think we still could have been a little more formal about how to handle pull requests and who is responsible for doing them. I wonder if this is a place where I should be more central in the process, teaching by example how to conduct a review of a pull request, and slowly doling out the responsibility to others on the team as they learn. I am uncertain, however, as to whether this would slow the team beneficially or artificially--if they need to make those mistakes with git in order to understand how to proceed.

A particularly tech-savvy team member knew a lot more about GitHub than I did, and he pointed out that we could set up an organization page as well as a project page. Given that our project is open source, this gives us free hosting for a web presence and the game itself, which is pretty slick. Our custom domain name simply redirects to our github.io page, with Google Domains' subdomain forwarding taking you to the info page or directly to the game. Thanks, GitHub!

We adopted Slack at the recommendation of one of our technically-minded team members, and this proved to be a great boon to the team. Over the past several years, I have struggled to find a team communication mechanism that fits the right balance of convenience for a maximal number of users. Usually, I have fallen back on Google Groups, which is essentially a mailing list; the problem is that kids these days, they don't conscientiously check their email. Slack hit us right where we needed it, with multiple entrypoints and notification settings for different users. If I were to use it again--which I expect I will--I will do a little more research on best practices for the use of channels. We used one #development channel, where programmers talked to each other and we had Travis sending continuous integration messages. This was much too noisy. The team had a lot of fun with Slack, which also meant that sometimes it was hard to separate the signal from the noise, so something like a new pull  request might get lost. This is probably something that regular users have bot-based solutions for, and one of the team members even started writing such a bot during finals week; next time, I need to come in a little better prepared.

All together, we built prototypes in three different programming platforms. In our first push for a playable prototype, the team decided to use Java with JavaFX because it was familiar to them, even knowing that it would not likely be our target platform. While they worked on this, I worked on proof-of-concept systems in both PlayN and Polymer. After that first iteration was passed, we looked at all three platforms and centralized on PlayN due to the ability to write in Java and use familiar tooling while also being able to publish to HTML5 via the GWT compiler. 

Team T-Shirts were an important part of our technology stack

Integration

We never quite hit a good cross-functional integration rhythm this semester as has happened in past semesters. Art integration was an awkward manual process, with files being uploaded to Google Drive, downloaded by a programmer, probably renamed to match conventions, and then added to the project. By contrast, in Bone Wars, we had the artists integrated directly into our repository, so they could freely update files themselves. It's worth noting, though, that we had two artists that semester and for twice the credit hours, so we had more breathing room: our one art major this semester really went beyond the call of duty in producing the amount of original work that he did.

We once again used a design log, and this came up frequently in students' reflections about what facilitated team work. I think we could have been even more deliberate in using it to explore design decisions: a few times, some ad hoc contribution would end up in the code and then become seen as normal, even though it broke the game's theme. It is also possible that some team members never did reference the design log, as very late in the game, there were still some team members using the game's nomenclature incorrectly, which invariably leads to confusion. Of course, this was a highly-experimental course, and we came in with inspiration but almost no design at all. It is really amazing that this team was able to accomplish what they did in just one semester and three credit hours. It helped that I had an extra course release for research this semester, as it meant I also had one less thing to worry about and could focus on helping Studio 368 succeed.

Story integration was also awkward. I initiated the design of a JSON-based story specification format, so that a single JSON file could describe the world. (This actually came out of my Polymer-based prototype.) One of the team members undertook the challenge of creating a graphical level editor in JavaFX that could read and write this format, which--in theory--would make it easy for those focused on writing to integrate with the game. Unfortunately, that never really happened: the writers never adopted the editor as a primary tool, instead collaborating on Google Docs. This led to some conflicts, where their creative writing was not always aligned with the formal needs of the project. It was the first time since my 2012 VBC seminar that we had to write our own editor, but this one was not integrated well enough into the rest of the stack to maximize its utility. In all of our iterations, I don't think the story team ever saw and debugged their work in the game until after the iteration was formally over. Looking forward, if I were to continue this project or undertake something similar, I would want to more formally include the editor in sprint planning and ensure that it supports both rapid integration to the game. 

Inspirational messages from Debbie

Acknowledgments

This project was supported by an Immersive Learning Micro-Grant from the College of Sciences and Humanities, to whom I am extremely grateful. We would not be able to make it a real studio without the generous support of my home department, Computer Science. One of the reasons I had such a great team was because of trusted friends on campus who sent me their best and brightest, and I owe them gratitude as well. Special thanks to Jennifer Grouling and Joyce Huff from the English department at Ball State, both of whom volunteered their time to share their scholarship with my team; Cathy Hamaker at The Children's Museum, for sharing her stories and time with the team; and to Rebecca Parker and Stuart Cotton at Muncie Public Library's Connection Corner, for being willing to work with us on this venture.

And of course, thanks to those students who took a risk in signing up for the game production studio. You did great work, and I am proud of you.

It's a long story.

Thursday, May 5, 2016

What we learned in CS222, Spring 2016 edition

It's that time of year again, where I like to share a short retrospective on CS222. You may recall that I had thought about making some significant revisions to the course, particularly with respect to the trio of assignment, project, and achievement. In particular, I thought about incorporating student portfolios, but as I read more about this approach, I saw strong encouragements that students should be able to pick what goes into their portfolio. With the short, holiday-packed winter break, I didn't have time to really dig into the research on portfolio-based grading and integrate it into CS222. Summer is a more relaxed time for making this kind of structural change.

As always, I have several notes sketched out, in various places, about what went well and what didn't go well. I keep coming back to the idea of a more structured approach to project planning, particularly with respect to user stories. So many students do this badly, even after repeated feedback, and then have difficulty with their project for lack of concrete requirements. I really don't want to muddy the iterative approach by having additional items due in the middle of it, so maybe I need to structure the days leading up to the first iteration more carefully... but that might mean dropping or shortening the two-week project, which tends to be an important learning experience and turning point for students. Clearly, there are a lot of unknowns, and I need to spend some time thinking about it.

Without further ado, here's the student-generated list of what they learned in CS222 this semester, with the number of votes:
Clean code*24
TDD*16
SRP6
DRY7
Descriptive naming2
It's OK to start over4
CRC cards2
SMART2
User stories8
Mode-view separation5
Team development*12
Pair programming 4
Red-Green-Refactor1
git9
enum
dependencies2
implementing libraries2
object vs data structure
code for the readers2
parsers
commenting best practices1
definition of OO9
bots
functional programming1
polymorphism
encapsulation2
inheritance
abstraction
stack overflow is KING
scrum3
agile
android
gradle
test coverage
test coverage exploitation
multithreading2
builder desing pattern1
observer design pattern
Drefus model of skill acquisition
personas
game & mobile development
user-centered design2
dependencies on List
Scala (presentation)
quantum computing
AI efficacy
prototyping3
Acceptance testing2
critical components
intellectual property5
8am is troublesome ("sucks")3
classroom is awkward
walk on the left
BNL- not as good as in the past
IntelliJ is more featured than Eclipse
IntelliJ is better than Elipse2
Gradle is fragile
Github is complex2
Github is powerful
github has a steep learning curve
IntelliJ has a steep learning curve3
Pitching1
Finishing / shipping a project2
Time management3
Slack is great
JavaFX3
FXML
Don't generate code you couldn't write1
Team communication is key5
Horizontal & vertical formatting
Egyptian vs K&R braces
DocumentBuilder (XML)
Package naming conventions
Unit testing4
Constant naming
Magic numbers
Essay reflections
Idea -> reality2
Creativity3
Don't procrastinate2
Critical thinking5
NPR is phasing out RSS
Making an app
Design thinking framework
Jar files
Managing scope1
Burndown chart
Technical debt
Mocking
Debugging
Timers are a pain
Compiler warnings1
Don't update mid-iteration
Network interfacing
UI design3
HBox & VBox
Scene Builder
Android Studio
Dedication2
Pizza makes apps
Branching1
Forking
Adjusting sleep schedule
Open Source Licensing1
Dr. G dislikes textbook authors
Donut sticks -> no real wheat
RMS is eccentric
There is no silver bullet1
Studio 368 only has 4 CS majors
Caffeine is good3
Dr. G likes jazz
Ragtime is racist
They won't kick you out of RB1091
How to take constructive criticism1
How to give constructive criticism
Better to have compiler warnings than nothing at all
Working with unfamiliar APIs
Don't program alone3
Controllers1
Inner classes
Take breaks when coding
Review code in short segments
Commit frequently
Don't push broken code
Working in iterations
Sometimes you have to break code to fix it

The three marked with asterisks were the top three on which students could write their final reflective essays.

There are no real surprises here. It was an 8AM class, so I would have been surprised if this had not come up on the list. There are usually a few personal things about me on there; I know someone said "Dr. G. likes tea," which I did not actually write up. The Ragtime reference is from a discussion we had in class the morning after I had seen the musical for the first time, and---through what I remember to be a far-reaching discussion---brought the racist themes of both Ragtime-the-art and Ragtime-the-production into comparison with diversity in software development.

Git, GitHub, and IntelliJ came up frequently, although not with very strong weight. These are the tools that I myself only started using last summer, and sometimes they still trip me up. As I have mentioned before, I do think the switch to GitHub is worth the effort since it's such a popular tool: if we're going to deal with a real version control system, we may as well use one that also has cultural purchase and can land students internships and jobs.

I get more and more students taking on Android projects. Whether this is related to the similarity between Android Studio and IntelliJ, I cannot say, but it does complicate evaluation. I really don't know gradle besides the very basics, and I have no reason to learn it with my current projects, where Maven is working fine for me. I suppose it's OK, though, for students to see that I don't know everything, and I have to also poke around on Google and Stack Overflow when things go wrong. With all the different versions of IDEs and Android libraries, it tends to take longer to clone and build a project than it does to evaluate it.

That's it for CS222 this semester. I have around thirty essays to grade before I can really call it done, but those are pretty easy: they are student reflections, a final stab at tying their real, authentic experiences to the big essential questions of the course.

Someday, I should do something with all this "what we learned" data...

Wednesday, May 4, 2016

A Project Retrospective and a Final Exam for the 2016 Spring Studio

I announced the release of Traveler's Notebook: Monster Tales a few days ago. This was the semester project of Studio 368, a multidisciplinary undergraduate team that I mentored during the Spring semester. After each of our four iterations, we held iteration retrospective meetings, during which they contributed answers to these four questions:

  • What did we do well that if we don't write down, we might forget?
  • What did you learn?
  • What should we do differently?
  • What still puzzles us?

I am fond of this format, and I use it regularly with my student teams. (In fact, I've written an empirical evaluation of this approach, should you care to read it.)

Studio 368 finished production on April 27, and on our final meeting of May 2, I led the team in a project retrospective. First, we collaborated on drawing a timeline of significant events from the semester, including production schedules, guest visits, important decisions, external playtesting, and dissemination opportunities. Then, I challenged them to think about what they had learned--their answers to the second reflection question above--and to annotate these on the timeline with sticky notes. Unlike a normal retrospective meeting, where people work individually and then find clusters, this time I encouraged them to post items as soon as they thought of them, and to talk to each other about refining the articulations where appropriate. I circulated some colored dots, which could be used to affirm or agree with items others posted.

The result looked like this:

That's too small to read, but it gives you the overall shape. Below, I have transcribed each note, attempting to maintain the orthography while prudently fixing spelling errors. If an item had dots affixed to it, I mark this in parentheses.
  • Don't be afraid to step out of your bubble and try something NEW (2)
  • Narrative-driven games are really interesting and redefine regular games (2)
  • FOOD is important (1)
  • game design for academic research is DOUBLE hard
  • GAME DESIGN is HARD (5)
  • Slack is AWESOME (1)
  • Articulating how we record qualitative data is difficult
  • Getting qualitative data is important to understanding the project (& how it meets its goals)
  • Github and feature branching is amazing (1)
  • How to integrate my artwork into a working platform / game (1)
  • KIDS LOVE VIOLENCE (1)
  • The importance of physical space (2)
  • The usefulness of the React library
  • How to build a game efficiently in an interdisciplinary team (1)
  • How to communicate with a team consisting of members of various disciplines (1)
  • How a culture's "monster" represents a real fear or threat to that culture (5)
  • Monsters are windows into culture (1)
  • Everything we make will be remade. Care about it but don't get attached. (2)
  • Pair programming accelerates production while minimizing errors (2)
  • How to adapt to a new set of programming libraries in a timely manner (2)
  • Implementing creative limitations makes for richer encounters (character limit, cultural implementation, multiple encounter reactions, etc) (1)
  • How to condense my writing to successfully meet character limits (1)
  • Design log is law... Design Lawg
  • Establishing visual metaphors as game mechanics (3)
  • In game design, the best idea wins! (1)
  • A good paper prototype takes time, but adds great clarity to the project (3)
  • Lack of specific roles in Agile/Scrum
  • It's okay to call a sprint a failure (5)
  • Jumping in on what others have been working on is difficult, yet a great learning experience (that should be had)
  • Don't throw up (1)
  • Going to events with the team really helps us bond (3)
  • I'm interested in game academia (1)
  • What we are making is something refreshing but innovative (3)
  • Google analytics: how to implement it
  • Sound design enhances the gaming experience (2)
  • Kids will find the limits of whatever you're developing (2)
  • We learned a lot (1)
I think this is an excellent list, full of beautiful learning outcomes for an interdisciplinary undergraduate course. The students seemed to enjoy the activity as well, engaging in the process with both quiet thoughtfulness and friendly laughter. 

One of the intentions behind this activity was to get them ready for their final exam. I have tried several different approaches to giving final exams for this kind of immersive learning class. This semester, I went back to our essential questions (on the course description) as well as the learning outcomes from my project proposal. I wanted to crack these open and expose the various ways that students can start to tease apart the lessons from the context. I am including the final exam verbatim below, which includes both prompts and a justification for itself:

What is the final exam?


Most of our work this semester was bound up in the making of Traveler’s Notebook: Monster Tales. Our team has built a shared understanding that is necessarily bound up in the context of our collaboration: the people, the problems, the places, the donuts, etc. The main point of the final exam is to help lift our thoughts out of the particulars of this context, to improve our ability to draw upon our understanding as we move on to other endeavors.


Choose two categories from those given below and respond to one prompt in each. If you think of a different prompt, or even a new category, let me know, and we can negotiate how to move forward. If you spent at least an hour at the Immersive Learning Showcase talking to guests about our project, then you only need to choose a prompt from one category, although you are always welcome to write more.


Transformative Games
  • Consider one or more of these essential questions:
    • How do games encourage or discourage the development of literacy?
    • What is the role of player culture in transformative game design?
    • What is the role of theory in research and development projects like ours?
What do you know about it now that you didn’t before the semester started? (Note that you don’t need to answer the question per se: the point of an essential question is to guide inquiry because it doesn’t have a closed-form answer.)


Expectations and Reality
  • What was your biggest surprise of the semester? Delve into it in an essay, considering: why was it a surprise (that is, what knowledge did you have or not have coming a priori)? When did you realize you were surprised? What did you learn from the experience?
  • Compare and contrast the final product to how you initially thought the game would turn out. What accounts for the differences?
  • If you could improve upon the game in one way (including game design, asset production, and technology platform), what would you improve, and why?


Reflective Practice
  • What was your biggest mistake of the semester? How did it come about, what did you do about it, and what do you think you learned from it?
  • Reflect on what you have learned this semester that is related to your involvement in the game studio, and choose one outcome that is most important to you. Write a reflection about the context of this outcome: Who was involved? In what places did it happen? Over what period of time? Was the experience mediated by technology writ large---software, designed spaces, furniture? What were the sights, the sounds, the smells, the tastes, and the tactile experiences involved? How did these factors contribute to this outcome?


Legacy
  • Write an orientation document for future game studio students, something that could be given to them at the start of the semester to help them succeed in their work.


Discipline

  • Produce an artifact in accordance with your academic focus that represents something significant you have learned this semester.
One of my students responded to this by sharing, “That moment when @paul.gestwicki makes the most reasonable final exam I've ever taken :raised_hands::clap::pray:"” I am not sure that reasonable is the first adjective most people use to describe my approach to teaching, but I am glad this student thought so.

I may write up a longer project retrospective of my own now that the semester is winding down, but for now, I wanted to share the list and thoughts about the final exam. As always, feel free to share your thoughts and reactions in the comments section.