Rewriting departmental P&T documents to follow Scholarship Assessed

At the end of February, I gave a presentation at Ball State's Engaging Community series on the topic of publishing community-engaged scholarship. I was glad to be invited, although I don't generally describe myself as a "community-engaged scholar." I see community engagement as a necessary component of the authentic game studio experience: if my teams were not engaged with the community, we would not be doing authentic work. In any case, I took the opportunity to talk about the variety of research questions my colleagues and I have investigated, and the corresponding variety of venues where this work has been published. I have posted the slides, if you want to take a look, though as usual they may not make sense without the stories surrounding them.

Before me in the session were two respected colleagues, Chadwick Menning and Mellisa Holzman, who organize the Elemental sexual assault protection program. One of their main talking points was how to contextualize community-engaged scholarship for the promotion and tenure process. Given that a program such as Elemental is not conventional scholarship, some within the university perceived it as being solely a teaching experience (because it arose from a VBC seminar) or as service (because it involves helping people, I guess). Mellisa's sage advice was to contextualize and justify the work within your vita—a tactical maneuver that is facilitated by conceiving of the vita as a personal expression rather than a bureaucratic formula.

Inspired by their story, I began my talk with some extemporaneous remarks about my own P&T adventures. I told the audience that I didn't really have any trouble defending my work as scholarship because I had rewritten my department's P&T document to make it easier. Back in 2008, I chaired my department's P&T committee, and I had just recently read both Scholarship Assessed and Scholarship Reconsidered. With a favorable committee and authority from the department, I was the primary author of the changes that moved us from a laundry list of recognized activities toward a definition of what we mean by "scholarship". Here is the relevant section:

All works of scholarship will be judged according to the following standards (adapted from Glassick, et al., Scholarship Assessed: Evaluation of the Professoriate, John Wiley & Sons and the Carnegie Foundation for the Advancement of Teaching, 1997, p36). It is the responsibility of the faculty member to clearly demonstrate in writing that the standards listed below are satisfied by the body of work presented to document his or her bid for promotion or tenure.

1. Clear goals. The scholar clearly articulates the basic purposes of the project, defines objectives that are realistic and achievable, and identifies the most important questions in the field.
2. Adequate preparation. The scholar shows an understanding of existing scholarship in the field, brings the necessary skills to the project, and brings together the resources necessary to move the project forward.
3. Appropriate methods. The scholar uses methods appropriate to the stated goals, applies effectively the methods selected, and appropriately modifies procedures in response to changing circumstances.
4. Significant results. The scholar achieves the stated goals. The results of the project add consequentially to the field and open additional areas for further exploration.
5. Effective presentation. The scholar uses a suitable style and effective organization to present the results of the project, uses appropriate forums for communicating these results to their intended audiences, and presents his or her message with clarity and integrity.
6. Reflective critique. The scholar critically evaluates her or his own work, brings an appropriate breadth of evidence to her or his critique, and uses this evaluation to improve the quality of future work.

I should note that this section is followed almost immediately by a more conventional section describing what categories of evidence are recognized. However, whereas this was previously an exhaustive list, it is now a list of items that simply don't require much extra contextualization.

The following categories of evidence of scholarship are recognized by the department. However, support materials not fitting clearly into any of these categories may still be submitted. All evidence of scholarship should be documented according to the standards in [the section above].

• Publishing refereed articles in journals or conference proceedings;
• Publishing book articles, research monographs, or textbooks;
• Writing reviews for national journals;
• Presenting papers at professional meetings;
• Conducting seminars, institutes, or workshops on computer science topics;
• Evidence of success in developing, experimenting with, and implementing new teaching procedures and techniques;
• Receiving grants or fellowships;
• Adoption outside the department of software or electronically distributed materials that you have developed;
• Serving as a professional consultant on matters of computer science to organizations or individuals outside the department.

There was no trouble with my department's approving these changes, and these policies have been in place ever since. When I have been considered for tenure, promotion, graduate faculty status, or merit pay, I simply provide a paragraph or two that establishes any unconventional work as scholarship given the six criteria.

This change did not instigate a visible cultural shift within my department: my colleagues are generally engaged in conventional disciplinary research. However, as far as I can tell, no one has pushed back against these changes either. Although I may be something of a black sheep in terms of how I conduct my research program, I don't sense anyone questioning the scholarly nature of it. I am sure it helps that, in addition to my more progressive work, I regularly produce traditional forms of scholarly output as well.

Although I am happy with this departmental policy, I also believe it does not go far enough in embracing Boyer's philosophy. Our document is patterned after a college document that uses a traditional three-category taxonomy of professorial activity: teaching, research, and service. That is, it misses the point that Boyer made, that everything a professor does should be scholarly—that the way of the scholar is scholarship. Worse yet, the college document reuses the word "scholarship" in place of the more traditional "research," which only further confuses the matter. Boyer's four scholarships simply don't fit into a model that separates scholarship from teaching and service. While I would like to see my department's document align with Boyer's model more explicitly, it's not a battle I choose to fight. Regardless, it was an epiphany for me when, as a young scholar, when I realized I could stop thinking about teaching, research, and service, and instead think about doing all things as a scholar.

(Erin Moore helped organize the university's session on Publishing Community-Engaged Scholarship, and she told me that she has received a few requests for more information about my department's P&T policies. I hope this post helps provide a bit more context than I could in some off-the-cuff remarks. As always, comments are welcome.)

What else is on the wall?

In my last post, I described some of the fun stuff on the wall in the studio. There was one piece I didn't describe, because I did not know its story until I asked the team today:

I had assumed it was some programmer art that was put up as a counterbalance to the excellent drawings by our resident art major. In fact, this monster was drawn by one of the kids who playtested our prototype last week Wednesday. What an excellent piece of community-connected inspiration! 

What's on the wall?

My post from earlier today was, pretty clearly, my attempt to jot down thoughts about this semester's studio project before they got away from me. I found myself with an unexpected half-hour or so, which was just right for writing.

But I know why you all really come here: the pictures! So, what's on the wall in the studio today?

On the East wall, we have a great collection of posters and sketches. I encouraged students to bring in some wall decorations early in the semester, and one jumped right on that, bringing the Teemo and Smite posters, and later, the calendar. Once we got our theme, someone else brought in, "Keep Calm and Find Monsters." Today, we got the addition of some thematic Darkest Dungeon images and several original drawings for in-game monsters.

On the West wall is the result of our post-planning UI design workshop. A few of us were standing with markers, a few sitting besides, and everyone was listening in. Out of these emerged a few core ideas that we're going to run with, including: using a 4:3 aspect ratio (give or take); using the left quarter of the screen to show persistent status such as scores and player names, and to give access to options such as restarting the game; showing a world map in the main play area, with a "card" metaphor for encounters.

Spring 2016 Studio: Story Maps vs Scrum Board

I mentioned in my last post that I decided to experiment with story mapping as an alternative to Scrum-style task tracking boards. My team had their first release on Wednesday, which led into an excellent playtesting session at Connection Corner. However, the team only was able to make the release due to the herculean effort of a few technical team members, who worked through the day on Tuesday to add missing critical features to the prototype.

It's worth dwelling on that phenomenon itself for a moment. This is part of a three credit-hour course, which means--according to federal standards--that we should expect nine effort-hours per week from the student during the fifteen weeks of the semester. The team has scheduled meetings for six hours per week, which I set up in order to maximize face-to-face time for this multidisciplinary team. At least one student put in about nine hours on Tuesday alone to help us meet this deadline. He didn't have to do this, in the sense that there would be no punitive grade assigned, and the shame at not having met our goal would have been shared by the entire team. Worse, we would not have been able to conduct our playtesting session, which would mean we would still be grappling with fundamental uncertainties today. On one hand, we need to congratulate the students who put in significant extra effort to help the team meat its goals (and indeed we gave a round of applause on Wednesday); with the other hand, however, we need to admonish ourselves for putting us into a situation where so much work was required before an important deadline. How did we get to that point? It seems there are only two options: either it was unclear to team members that we were headed toward missing the goal, or they did not care that we were going to miss the goal. The behavior of this subset of students who put in extra hours leads me to conclude the former.

On Monday, many stories remained on the spine of the story map, all or nearly all with student names on them as responsible parties. On Wednesday--the day of the deadline--I wrote a note on the board in the morning to ensure that the story map was up to date for our review & retrospective meeting, and even then, the story map was untouched.

Every student team goes through some growing pains where they realize that they need to be more careful about communication and planning. The retrospective meeting on Wednesday confirmed that the students were aware of the need to coordinate more prudently and tactically. I am left wondering, however, how much of this is related to the story map? I could look at the story map and tell that we were not making progress, as notes were not moving, but I think they were blind to this. I suppose this iteration will demonstrate to us whether they will hold themselves more accountable to the practice or not. During the retrospective, a student contributed what I think will be a valuable action item: immediately after the stand-up meeting, gather around the story map to review where we are. Today was a planning meeting for the next iteration, and so it will take until Monday to get any sense of whether this helps or not.

Another item that gives me some unease is the tracking of stories (in a story map) vs tasks (with a conventional Scrum board). User stories are necessarily brief, but to address their need requires a decomposition of the story into discrete tasks. Using Scrum, this was done explicitly during the Sprint Planning meeting. For example, given a story like "As a player, I want to create my character" might break down into a design and an implement task, to make it clear that we should figure out a good user experience before programming it. However, in the last iteration, we saw several UI components that were thrown together by programmers--surely with good intention, but just as surely without any background knowledge in usability and game interface design principles! As above, there's too much ambiguity to drive a particular conclusion: did the students not know that they should step back and work with the team to design a good UI, or were they just working quick-and-dirty on a two-week prototype? The truth may be somewhere in the middle, and I suppose I can hope that if it is purely ignorance that this two-week prototype was a learning experience about the importance of critical design!

This leads me to one last point that I want to record here, so I don't forget. This two-week prototype was "slapped together" intentionally. The team agreed on several methodological rules governing source code, but they also agreed that these rules would not hold for the two-week prototype: this could be done "quick and dirty." Late in the iteration, I started looking more carefully at the code being committed to the repository, and I was struck by how one developer's "dirty" might be another developer's "abominable." When I said dirty, I meant we could be lax about things like having rigorous unit tests, or allowing several parameters to constructors rather than introducing a builder. Others clearly had a different idea! This strikes me as a fascinating study for software engineering research: how do different developers approach a project when told they can do it "dirty" vs. "clean", and what are the contributing factors in their decisions, and what are the inherent properties of the code they produce? Maybe someone has done this already, but if not, it sounds fascinating---and it clearly resonates with the kinds of discussions I have daily in CS222.

Spring 2016 Studio: Goals and Early Design

This semester's Spring Game Studio is off to a good start. We are exploring narrative-rich games in collaboration with Connection Corner in Muncie. The first week was primarily an orientation, during which we played Buffalo, Tales of the Arabian Nights, and Two Rooms and a Boom, and I introduced the team to fundamentals of serious game design, including MDA analysis. The team also met Rebecca Parker, our contact at Connection Corner. From these discussions, the team was able to agree on the following primary goals for our work this semester:

• Create a game that promotes cultural literacy
• Incorporate narrative as the driving factor in gameplay
• Include a technology component (digital or hybrid game design)

Articulating and building consensus around the goals went much easier than I expected. I asked them to reflect on the conversations of the first week and to write out all the various goals and constraints we could adopt, and to prioritize these. In a team meeting, I asked them to read out their highest-priority goals, and I put these on sticky notes and posted them to the board. Then, I left the room and asked them to sort the sticky notes. I came back after a few minutes, and they had actually completed the exercise quite well, identifying high, medium, and low priorities, and starring the three aforementioned goals as being the most important. 

Result of the team's deliberations on design goals

Our next step was to settle on fundamental game design ideas. I introduced the team to Tim Ryan's format for concept documents, which I like for several reasons. I invited the team to consider contributing in any one of the following ways before our next meeting:

• Identify game design elements that support our shared goals
• Write a game concept document
• Identify and justify a context for exploring cultural literacy, such that it could be taken as the theme of our game
• Create a poster that articulates clearly our three main goals for the project this semester
• Bring donuts

Sadly, no one brought donuts. I dug up two concept documents from my cybersecurity education game project (which, perhaps, I still have not blogged about) and posted those as examples. I then wrote up two concept documents to frame my own thoughts about the project. Only one other student created a concept document by the imposed deadline. A lot of the discussion centered on the ideas I had brought. I felt a little self-conscious about it, as I did not want to force my designs on them, but one student even said explicitly, "I think we should just go with Dr. G's design" (which, from the conversation, I inferred to mean "because he has already done the most thinking about this and has the most experience.")

We were then able to have a focused discussion about the game's theme. Several were proposed, but again the team quickly came to consensus: we will be using monsters from different cultures as a theme for the game. We had talked about how Tales of the Arabian Nights is successful in part because it is a big game about one culture, and that it may be too ambitious to expect similar results from a shorter game about world cultures. Theming around monsters allows to ask interesting questions, such as "How does culture inform what is considered monstrous?" Our background work and discussion leads us to believe that this theme will be attractive to our target audience, which is roughly 10-12 year-olds—a challenging group to engage with narrative-heavy games, and all the more reason to pick a theme that should excite them!

The team was once again challenged to approach the game design via concept documents, and I wrote up two myself—variations on the original two, based explicitly on monsters, and incorporating some further thoughts about the design space. This time, no one else brought any concept documents, but one student explained that one of my designs already captured his ideas. This is reasonable, since my own designs, and much of our discourse, has been inspired by Tales of the Arabian Nights.

I have been using Scrum for years now, but several forces have led me to seek out alternatives. After watching Allen Holub's DevWeek Keynote on #NoEstimates, I was inspired to try story mapping. Yesterday, with our concept laid out, we tried creating a story map, but it felt rather awkward. I think the main problem was that we still did not have the game design ideas well articulated: the concept document was necessarily brief on details. I told the students that I would start a game design log to help express the game design reflected in the selected concept document. As I re-read Daniel Cook's essay on game design logs, I was reminded that the game is the primary artifact, and that all design documentation is secondary—and so one should aim for building a minimal working prototype as quickly as possible. This inspired me to take on some creative and managerial challenges today: I spent about three hours this morning laying out the core game design in a design log, and later today, I am going to revise our draft story map into one that is aimed at an aggressive schedule of building a minimum viable product. 

First attempt at a story map

Incidentally, I didn't realize until after the meeting how very confusing this was to the non-CS people in the room. The CS majors and minors have all had CS222 with me, and so they are intimately familiar with the concept of "user stories," but I fear I did not frame this well enough for the others. One of them pointed out his confusion after the fact: how could one not be confused when we're making a story-based game and also talking about the stories of users?

This post started as a simple post to Facebook, something along the lines of "I am grateful for the creative freedom I have in my job, as I spent the last three hours working on a game design specification." I did not post that, however, because that's only part of the story. Sometimes I regret not blogging more of the mundane details about my teaching experiences, since the overall result is emergent from these moment-to-moment decisions. Hence, this post, which I hope you enjoyed reading, and maybe you have something to contribute in the comments, but if nothing else, it's something that Future Me can read to remember how we got to wherever we end up.

Design questions inspired by playing Witcher 3

Last night, I finished playing Witcher 3. I enjoyed it, and I think it offers quite a bit to think about. This morning, I want to share some of the design questions that it made me consider. Many of the formal elements of Witcher 3 are staples for its genre, but as my previous post intimates, I've been thinking more about the concept of genre lately and how it's not well applied in games. My wife described Witcher 3 as "my kind of game," which is traditionally true: I love a good CRPG fantasy simulator. However, I am also approaching this as a father, playing a game whose story is fundamentally about fatherhood—aware that the game that takes an enormous amount of time and attention that could otherwise be spent on actual fatherhood.

Without further ado, then, here are some design questions that came to mind while playing Witcher 3.

What if time-sensitive tasks were actually time-sensitive? It's conventional design in CRPGs to tell the player about some important and imminent event, but then give them a functionally infinite amount of time to proceed with it. There's a monster killing everything in the swamp, your daughter is being chased by the Wild Hunt, there's a guy waiting for you at dusk by the docks... but you can faff about for days and there's no in-game impact. The reason for this is really that the rest of the systems are not tuned to time being an important part of the system.

What if crafting took time? As long as I have a diagram, any talented smith can do any number of complex operations instantaneously. That is convenient because the game preferences incremental numeric increases over story. What if that was inverted?

What if gear were not limited by level? Geralt is a mutated fighting machine, a master of swords and violence, yet more than once, I found a club that he couldn't figure out how to swing at enemies until he gained levels.

What if herb-collecting were not a drudgery? For example, what if you could mark patches of different herbs and then, like fast-travel, do a fast-collect operation? Or what if you could hire a local kid to go do it for you for a gold coin or two? Making it a manual process is like eliminating fast-travel: it just fritters away player time without contributing to other goals.

What if you felt heroic from the start? Geralt was a veteran witcher before the series even started, yet common wolves posed a serious threat at the start of the game. The idea that players and monsters have levels, and player levels escalate, is a trope of CRPG design: what if we got rid of it?

What if you had a reasonably limited inventory? I was carrying a veritable armory for most of the game. Lax realism of inventory is a prerequisite to avoid frustration when the game privileges stats and stuff-collection over story. What if that wasn't the case?

For a game with such an impressive and thoughtful story, the design actually privileges loot-hunting stat-boosting over story every time. I suspect that if you start breaking these constraints, you would very quickly fall out of the conventional Fantasy CRPG space—but really, I am tired of that game. Couldn't we have an even better Witcher experience if they allowed the story to define the gameplay experience, rather than inhibit the story by tropes of Fantasy CRPG design?

I heard a designer from Bethesda talk about the Elder Scrolls series at a conference a few years ago, and the question came up about how little design or story innovation there was across the series: each one is fundamentally the same, a child-of-prophecy story. The answer was economic: these games have enormous budgets, and there's always a new generation of 14-year-olds for whom [Daggerfall/Morrowind/Oblivion/Skyrim] is their first Fantasy CRPG epic, and so they keep the formula the same. (It helps that nostalgic older gamers will pay for the new installment and memories of that first time as well.) Witcher probably falls into this morass as well: to craft such a beautiful and engaging world takes millions of dollars, and they don't want to risk alienating an audience.

I believe that there is plenty of room for innovation in game design, but after playing Witcher 3 and Dragon Age: Inquisition in the last nine months—and never having finished Skyrim out of boredom a few years ago—I am tired of the tropes. Although I enjoyed playing Witcher 3, I feel like its story was made the worse for the conflict between the narrative and the formal systems.

Characterizing a genre

The Spring semester starts in about a week, and with it, my most recent immersive learning project. Ball State's immersive learning program has allowed me to offer several Spring Game Development Studio experiences, but this Spring the project is a little different. The following video describes my motivation for the project:

In the video, I describe Tales of the Arabian Nights and Agents of SMERSH as "narrative-rich games," but the term is not rigorously defined—and I'm not sure it's the best choice of terms either. It was what I came up with last summer, when I originally gave that presentation at THATCamp Indiana 2015. As I began working on a formal course plan for the Spring, I realized I needed to more rigirously define this genre. This will help us to focus our efforts as well as communicate our findings.

I have identified the following necessary characteristics:

1. Take place in a believable fictional setting. The world of Tales of the Arabian Nights may be unfamiliar to the player, but everything in it is representative of the source material, even if there are contradictions. Agents of SMERSH is similar with its cold war secret agent theme.
2. Use narrative as the primary feedback mechanism. Raph Koster wrote about how narrative is a feedback mechanism in game design. In Tales of the Arabian Nights, player's reaction choice has two forms of feedback: first, the narrative that describes the outcome, and then any changes to the player's game state (story or destiny points, location, status, wealth level, and treasures). Tales gives the narrative first, and the state change is secondary to it.
3. Have measurable goals. That is, there is a winning condition: this is a game after all and not just cooperative storytelling. 
4. Incorporate endogenously meaningful ambiguous decision-making. Along with the previous point, I am borrowing from Keith Burgun's philosophy of games, and his taxonomy of interactive forms in particular. The player in these games must make decisions that are important but without total knowledge, and these decisions have an effect on the game world. To me, one of the most bizarre properties of Tales is how little agency a player really has: decisions are made on very little information, but these decisions are critical to the player's success. This leads to the next characteristic:
5. Reward players for decision-making that reflects cultural understanding. The knowledge that a player brings to bear on decisions is not just knowledge of in-game systems. If you are playing Tales and you face an Angry Ifrit, and you don't have the Combat skill, and you choose to Fight, it is probably not going to end well. This prediction is not based on my having memorized the Book of Tales, nor knowing how many hit dice an Ifrit has. Rather, I am drawing upon my understanding of Anger, Spirits, Fighting, and probably more. I would like to add that I don't mean the game should set up "right" and "wrong" answers to problems, and there should be unexpected twists, but by and large, the story remains logically consistent—or as logically consistent as you can be when you are allowed to respond to a sudden squall with the "Drink" or "Enter" option. 

There are some characteristics which I have not yet determined to be necessary, beneficial, or merely accidental. Principle among these is whether the game needs to involve multiple players. My inspiration for this project comes from multiplayer games, but perhaps the goals of the project could also be met in a single-player experience. This is an important consideration since building digital prototypes of single-player experiences is generally much easier than supporting multiple players. A few potentially important characteristics follow from the adoption of a multiplayer requirement:

1. Take place in an explicit, represented virtual space. In both Tales and SMERSH, the fact that the game takes place in space is critical to the gameplay. Originally, I thought this was a necessary characteristic, but consider the Choose Your Own Adventure style of single-player experience. These take place in an implicit viritual space, expressed only in text, yet a player may feel immersed in the environment without maps or tokens. If there are multiple players, an explicit virtual space makes it easier to see (and believe?) that each character is on their own journey, in a way that is unnecessary for single-player experiences. Note that text adventure games have an explicit virtual space, although it may be obfuscated through the user-interface, and we see in classical MUDs that this space was critical for maintaining a sense of space to players.
2. Players orate narration to each other. Having players read narrative to each other is, in part, a game design trick to keep players involved while it is not their turn, but I think it has a deeper significance here. Oral storytelling is an ancient tradition, and my understanding of the literature (although it's outside my area) is that hearing stories leads to different comprehension than reading stories. Having players read the tales to each other feels practomimetic, subtly turning the players into bards and elders, changing the social and power dynamics. However, I also know of games of Tales of the Arabian Nights being ruined by having low-literacy players—players with poor vocabulary skills who read the narrative poorly or incorrectly, impeding the listeners' understanding of the story.

Taken together, I think we can agree that "narrative-rich game" is not a very expressive moniker for this genre. I am open to suggestions for a new name, and perhaps I will give this as a challenge to the team in the Spring. In any case, these are the characteristics I have identified so far, to set some boundaries around the kind of games we will be investigating in the Spring. I am looking forward to this project, although I admit I am not entirely sure how this will all play out. There are a lot of unknowns, but that's always the case with a research project. Regardless of what we find and how it turns out, I am sure the team and I will have a great learning experience, if nothing else.