Saturday, April 30, 2016

The Launch of Traveler's Notebook: Monster Tales

I am pleased to announce the release of my Spring Studio team's game, Traveler's Notebook: Monster Tales. It is a two-player, narrative-rich game in which you encounter monsters from various world cultures. The game grew out of my consideration of learning through games like Tales of the Arabian Nights, which I wrote about back in January, and which led to the team's identification of their goals and early design work. I plan to write a more thorough retrospective of the semester once it's all behind us. For now, check out the game at travelersnotebookgame.com!


If you're in the vicinity of Ball State University, members of the team will be at the Immersive Learning Showcase on Monday, 4-6PM in Cardinal Hall at the Student Center. They would be glad to give you a demo and talk about their experience.

Wednesday, April 20, 2016

Metacognitive learning through a design thinking framework

I have written a few times about the design thinking model that I use, lightly adapted from George Kembel's:


Brian McNely and I wrote a paper about how we observed a team falling into a failure mode of looping like this:


That experience inspired me to start using this model as an exercise in CS222. Around the beginning of the third three-week iteration on the final project, I take a day where I begin by introducing this model. Then, I ask the teams to annotate their paths through the phases, starting with their project pitch, and annotating major milestones such as the ends of iterations. This is a great metacognitive exercise that gets students thinking about how they have been proceeding. Inevitably, I get a student who reports that they did it "wrong" by starting in somewhere other than empathy or by jumping between phases, but then I explain that we're not using this in a prescriptive way, but as a tool to help us think about our processes.

In this semester's CS222 class, we did this exercise last week Friday, which was the first week of the final iteration. This is the only photograph I have of the event, but rest assured that this is representative of the kinds of diagrams the students generate.


I usually tell the story about the study that McNely and I did, pointing out that sometimes we software development types get stuck in "ideate-build-test" loops without stepping back and remembering for whom we are building this thing. The arcs in students diagrams tend to have a similar shape, often skipping over the empathy step entirely (which is understandable since it's not really connected to our course learning outcomes).

On Friday, though, I saw something I had never seen before. The last team to present their models showed that they had a strong arc like this:


That is, they observed that they identified the real problems within a specific demographic, but they never actually decided which of those problems they would solve, and how they would do so. Instead, they went right from identifying the problems to programming. Having looked at their code in two formal evaluations and some information evaluations, I think this is astute: it perfectly describes the failure mode that they fell into. The team is pulling itself together in this last iteration, and I hope that this exercise is formative to that process.

As I was talking to my class about this model, I had a moment of inspiration where I realized there may be disciplinary differences in how one gets trapped in subcycles. This is only a hypothesis at this point, but I think I've seen this before in my multidisciplinary classes:


I see programmers fall into the trap where they think of something, build it, think of something else, build it, and so on, without ever meaningfully evaluating it or really returning to the user's needs. The humanists--such as English and History majors--tend to think about the problems that exist and think of solutions, then think of other problems, then think of solutions, and so on, without actually building anything to validate their ideas. I wonder if teaching this model early would help students to frame their processes in a useful way and help them catch themselves unproductive spiraling.

Wednesday, March 9, 2016

Leaving story maps behind

I wrote on the first of this year about how I was excited to try using story maps as a management aid for my Spring Studio team, and in the middle of February I followed up with some of my findings from one iteration's use. The team just finished their second iteration, and after talking about it with them during our end-of-iteration review and retrospective meeting, we are going to drop the story maps and go to Scrum.

The problem with the story map—as we reified it—is that it presumes that the reader already has knowledge of the fundamental processes that are required to complete a story. For example, we had an activity on the story map called "Narrate my friend's story" with a story underneath it called "Read conclusion and result." (Story, Conclusion, and Result are all key terms from our game design and are defined in a design log.) When I look at that story, I can imagine the steps required to complete it: sketch the UI, make sure I have actual conclusions and results, make sure it fits with the overall game UX flow, rough it in code with placeholder assets, start replacing placeholders with real assets, test the integration. The problem is that I am not the one doing these steps: rather, it is a team of people who never did this before. What happened in the first two iterations with the story map is that they would look at "Read conclusion results" and, in the discussion, I would explain that these steps were required. These steps weren't tracked explicitly, though, and so we ran immediately into cognitive overload, and the team moved forward with the illusion that they knew how to proceed. Note that this is basically the same phenomenon as in a traditional class, where you can tell the class something and they will nod but not take notes, believing that they understand; of course these students then fail to follow the proper steps when it comes time to execute the task independently.

I am hopeful that switching to Scrum will make it easier for the team to track its progress on smaller tasks. It will mean more attention devoted to planning meetings, as we break down stories into individually-measurable tasks. One reason I was hoping for a leaner approach using story maps is that this is only a three credit-hour course, unlike the six credit-hour studios of the past few years, and so this will eat even more time away from production. However, if we're not producing the right thing, then it doesn't matter how quickly we do it.

Saturday, March 5, 2016

Rewriting departmental P&T documents to follow Scholarship Assessed

At the end of February, I gave a presentation at Ball State's Engaging Community series on the topic of publishing community-engaged scholarship. I was glad to be invited, although I don't generally describe myself as a "community-engaged scholar." I see community engagement as a necessary component of the authentic game studio experience: if my teams were not engaged with the community, we would not be doing authentic work. In any case, I took the opportunity to talk about the variety of research questions my colleagues and I have investigated, and the corresponding variety of venues where this work has been published. I have posted the slides, if you want to take a look, though as usual they may not make sense without the stories surrounding them.

Before me in the session were two respected colleagues, Chadwick Menning and Mellisa Holzman, who organize the Elemental sexual assault protection program. One of their main talking points was how to contextualize community-engaged scholarship for the promotion and tenure process. Given that a program such as Elemental is not conventional scholarship, some within the university perceived it as being solely a teaching experience (because it arose from a VBC seminar) or as service (because it involves helping people, I guess). Mellisa's sage advice was to contextualize and justify the work within your vita—a tactical maneuver that is facilitated by conceiving of the vita as a personal expression rather than a bureaucratic formula.

Inspired by their story, I began my talk with some extemporaneous remarks about my own P&T adventures. I told the audience that I didn't really have any trouble defending my work as scholarship because I had rewritten my department's P&T document to make it easier. Back in 2008, I chaired my department's P&T committee, and I had just recently read both Scholarship Assessed and Scholarship Reconsidered. With a favorable committee and authority from the department, I was the primary author of the changes that moved us from a laundry list of recognized activities toward a definition of what we mean by "scholarship". Here is the relevant section:
All works of scholarship will be judged according to the following standards (adapted from Glassick, et al., Scholarship Assessed: Evaluation of the Professoriate, John Wiley & Sons and the Carnegie Foundation for the Advancement of Teaching, 1997, p36). It is the responsibility of the faculty member to clearly demonstrate in writing that the standards listed below are satisfied by the body of work presented to document his or her bid for promotion or tenure.
  1. Clear goals. The scholar clearly articulates the basic purposes of the project, defines objectives that are realistic and achievable, and identifies the most important questions in the field.
  2. Adequate preparation. The scholar shows an understanding of existing scholarship in the field, brings the necessary skills to the project, and brings together the resources necessary to move the project forward.
  3. Appropriate methods. The scholar uses methods appropriate to the stated goals, applies effectively the methods selected, and appropriately modifies procedures in response to changing circumstances.
  4. Significant results. The scholar achieves the stated goals. The results of the project add consequentially to the field and open additional areas for further exploration.
  5. Effective presentation. The scholar uses a suitable style and effective organization to present the results of the project, uses appropriate forums for communicating these results to their intended audiences, and presents his or her message with clarity and integrity.
  6. Reflective critique. The scholar critically evaluates her or his own work, brings an appropriate breadth of evidence to her or his critique, and uses this evaluation to improve the quality of future work.
I should note that this section is followed almost immediately by a more conventional section describing what categories of evidence are recognized. However, whereas this was previously an exhaustive list, it is now a list of items that simply don't require much extra contextualization.
The following categories of evidence of scholarship are recognized by the department. However, support materials not fitting clearly into any of these categories may still be submitted. All evidence of scholarship should be documented according to the standards in [the section above].
  • Publishing refereed articles in journals or conference proceedings;
  • Publishing book articles, research monographs, or textbooks;
  • Writing reviews for national journals;
  • Presenting papers at professional meetings;
  • Conducting seminars, institutes, or workshops on computer science topics;
  • Evidence of success in developing, experimenting with, and implementing new teaching procedures and techniques;
  • Receiving grants or fellowships;
  • Adoption outside the department of software or electronically distributed materials that you have developed;
  • Serving as a professional consultant on matters of computer science to organizations or individuals outside the department.
There was no trouble with my department's approving these changes, and these policies have been in place ever since. When I have been considered for tenure, promotion, graduate faculty status, or merit pay, I simply provide a paragraph or two that establishes any unconventional work as scholarship given the six criteria.

This change did not instigate a visible cultural shift within my department: my colleagues are generally engaged in conventional disciplinary research. However, as far as I can tell, no one has pushed back against these changes either. Although I may be something of a black sheep in terms of how I conduct my research program, I don't sense anyone questioning the scholarly nature of it. I am sure it helps that, in addition to my more progressive work, I regularly produce traditional forms of scholarly output as well.

Although I am happy with this departmental policy, I also believe it does not go far enough in embracing Boyer's philosophy. Our document is patterned after a college document that uses a traditional three-category taxonomy of professorial activity: teaching, research, and service. That is, it misses the point that Boyer made, that everything a professor does should be scholarly—that the way of the scholar is scholarship. Worse yet, the college document reuses the word "scholarship" in place of the more traditional "research," which only further confuses the matter. Boyer's four scholarships simply don't fit into a model that separates scholarship from teaching and service. While I would like to see my department's document align with Boyer's model more explicitly, it's not a battle I choose to fight. Regardless, it was an epiphany for me when, as a young scholar, when I realized I could stop thinking about teaching, research, and service, and instead think about doing all things as a scholar.

(Erin Moore helped organize the university's session on Publishing Community-Engaged Scholarship, and she told me that she has received a few requests for more information about my department's P&T policies. I hope this post helps provide a bit more context than I could in some off-the-cuff remarks. As always, comments are welcome.)

Monday, February 15, 2016

What else is on the wall?

In my last post, I described some of the fun stuff on the wall in the studio. There was one piece I didn't describe, because I did not know its story until I asked the team today:
I had assumed it was some programmer art that was put up as a counterbalance to the excellent drawings by our resident art major. In fact, this monster was drawn by one of the kids who playtested our prototype last week Wednesday. What an excellent piece of community-connected inspiration! 

Friday, February 12, 2016

What's on the wall?

My post from earlier today was, pretty clearly, my attempt to jot down thoughts about this semester's studio project before they got away from me. I found myself with an unexpected half-hour or so, which was just right for writing.

But I know why you all really come here: the pictures! So, what's on the wall in the studio today?


On the East wall, we have a great collection of posters and sketches. I encouraged students to bring in some wall decorations early in the semester, and one jumped right on that, bringing the Teemo and Smite posters, and later, the calendar. Once we got our theme, someone else brought in, "Keep Calm and Find Monsters." Today, we got the addition of some thematic Darkest Dungeon images and several original drawings for in-game monsters.


On the West wall is the result of our post-planning UI design workshop. A few of us were standing with markers, a few sitting besides, and everyone was listening in. Out of these emerged a few core ideas that we're going to run with, including: using a 4:3 aspect ratio (give or take); using the left quarter of the screen to show persistent status such as scores and player names, and to give access to options such as restarting the game; showing a world map in the main play area, with a "card" metaphor for encounters.

Spring 2016 Studio: Story Maps vs Scrum Board

I mentioned in my last post that I decided to experiment with story mapping as an alternative to Scrum-style task tracking boards. My team had their first release on Wednesday, which led into an excellent playtesting session at Connection Corner. However, the team only was able to make the release due to the herculean effort of a few technical team members, who worked through the day on Tuesday to add missing critical features to the prototype.

It's worth dwelling on that phenomenon itself for a moment. This is part of a three credit-hour course, which means--according to federal standards--that we should expect nine effort-hours per week from the student during the fifteen weeks of the semester. The team has scheduled meetings for six hours per week, which I set up in order to maximize face-to-face time for this multidisciplinary team. At least one student put in about nine hours on Tuesday alone to help us meet this deadline. He didn't have to do this, in the sense that there would be no punitive grade assigned, and the shame at not having met our goal would have been shared by the entire team. Worse, we would not have been able to conduct our playtesting session, which would mean we would still be grappling with fundamental uncertainties today. On one hand, we need to congratulate the students who put in significant extra effort to help the team meat its goals (and indeed we gave a round of applause on Wednesday); with the other hand, however, we need to admonish ourselves for putting us into a situation where so much work was required before an important deadline. How did we get to that point? It seems there are only two options: either it was unclear to team members that we were headed toward missing the goal, or they did not care that we were going to miss the goal. The behavior of this subset of students who put in extra hours leads me to conclude the former.

On Monday, many stories remained on the spine of the story map, all or nearly all with student names on them as responsible parties. On Wednesday--the day of the deadline--I wrote a note on the board in the morning to ensure that the story map was up to date for our review & retrospective meeting, and even then, the story map was untouched.

Every student team goes through some growing pains where they realize that they need to be more careful about communication and planning. The retrospective meeting on Wednesday confirmed that the students were aware of the need to coordinate more prudently and tactically. I am left wondering, however, how much of this is related to the story map? I could look at the story map and tell that we were not making progress, as notes were not moving, but I think they were blind to this. I suppose this iteration will demonstrate to us whether they will hold themselves more accountable to the practice or not. During the retrospective, a student contributed what I think will be a valuable action item: immediately after the stand-up meeting, gather around the story map to review where we are. Today was a planning meeting for the next iteration, and so it will take until Monday to get any sense of whether this helps or not.

Another item that gives me some unease is the tracking of stories (in a story map) vs tasks (with a conventional Scrum board). User stories are necessarily brief, but to address their need requires a decomposition of the story into discrete tasks. Using Scrum, this was done explicitly during the Sprint Planning meeting. For example, given a story like "As a player, I want to create my character" might break down into a design and an implement task, to make it clear that we should figure out a good user experience before programming it. However, in the last iteration, we saw several UI components that were thrown together by programmers--surely with good intention, but just as surely without any background knowledge in usability and game interface design principles! As above, there's too much ambiguity to drive a particular conclusion: did the students not know that they should step back and work with the team to design a good UI, or were they just working quick-and-dirty on a two-week prototype? The truth may be somewhere in the middle, and I suppose I can hope that if it is purely ignorance that this two-week prototype was a learning experience about the importance of critical design!

This leads me to one last point that I want to record here, so I don't forget. This two-week prototype was "slapped together" intentionally. The team agreed on several methodological rules governing source code, but they also agreed that these rules would not hold for the two-week prototype: this could be done "quick and dirty." Late in the iteration, I started looking more carefully at the code being committed to the repository, and I was struck by how one developer's "dirty" might be another developer's "abominable." When I said dirty, I meant we could be lax about things like having rigorous unit tests, or allowing several parameters to constructors rather than introducing a builder. Others clearly had a different idea! This strikes me as a fascinating study for software engineering research: how do different developers approach a project when told they can do it "dirty" vs. "clean", and what are the contributing factors in their decisions, and what are the inherent properties of the code they produce? Maybe someone has done this already, but if not, it sounds fascinating---and it clearly resonates with the kinds of discussions I have daily in CS222.