Monday, February 14, 2022

An exercise in assessment of scholarship: Another look at the Year of Fam Jams

I'm sure I mentioned it before, but reading Boyer's Scholarship Reconsidered was formative in my development as a young scholar. It helped me abandon the uninspiring taxonomy of teaching, research, and service and instead to think about scholarly ways of being. Boyer's taxonomy includes Scholarship of Teaching, Discovery, Application, and Integration. It makes me sad to see how frequently these terms are misunderstood at my own institution, where people use them as neologisms for teaching, research, and service rather than really engaging with what Bloom was saying.

Glassick's Scholarship Assessed deals with the significant problem of determining whether or not an activity aligns with that scholarly way of being. This model can be used regardless of domain and regardless of what type of scholarship is being assessed. I have never encountered any opposition to Glassick's model except, if my memory is right, by one old crank who insisted that peer-reviewed publications were the only form of scholarship. I have met plenty of people who neither thought about them nor engaged with them, but I don't remember ever seeing someone try to refute them.

A ridiculous series of events leads me to an interesting writing and thinking exercise. I agree with Paul Graham's recent essay, which asserts that "no one who hasn't written about a topic has fully formed ideas about it." Regardless of the series of events, then, it's good to take the time to write. Indeed, I originally made this blog post last night, but after reflecting on it, I've returned to it this morning to tighten it up.

The topic for consideration is whether my 2020–2021 FamJam project represents scholarship. To address this, I will take Glassick's criteria as a given. One could frame this primarily as Scholarship of Integration, although it has elements of Teaching, Discovery, and Application as well.

Clear Goals

The project started with a few goals. One was to build original video games with my family through the application of best practices, contemporary scholarship, and appropriate tools for game design and development. This serves dual subgoals, which are that the family would learn from the experience and we would make something to bring joy to others. Combined with this goal was the desire to investigate the extent to which what I had learned about mentoring undergraduate game design and development teams would be applicable to my own family. That is, what elements or ceremonies from my scholarship of teaching would apply in this context, and how? Another goal from the beginning of the project was to use this experience with my family to inspire other people to try something similar themselves.

As the project continued and our process matured, I thought more about how I could communicate our findings to the right audience. That is, having effective dissemination was a goal from early in the project.

Adequate Preparation

I've been studying game design and development for over fifteen years and have written multiple peer-reviewed and published papers on the topic. I have mentored many student projects, and I myself have made many games and participated in jams, alone and in teams. Before starting my project, I also spoke extensively with a colleague who has guided families in Fam-Jams before, and we stayed in touch intermittently throughout. Maintaining my YouTube channel has also helped me gained adequate skills in video production, which would prove useful for meeting the dissemination goals.

Appropriate Methods

My family ended up doing 13 consecutive monthly Fam Jams, starting in March 2020 and ending in March 2021. Each month, we carved out a day to work on an original video game project. We assigned a creative director, worked on our project, and finished it before the day was out. We incorporated various reflective practices inspired by agile software development, engaging in semi-structured retrospectives after each project.

Doing this work was a kind of action research, getting into the field and participating in the work. As action research, the goal here was to understand particulars. Elements of the project may also be viewed in an autoethnographic light. Part of what makes this method appropriate is that we were making lemonade with the lemons we were given: a global pandemic had emerged, and lockdowns and distancing requirements were imposed, and so my family was the unit that I had to work with. 

We chose to release all of our projects publicly on the Web and to make all the code available under a Free Software license. This way, interested parties could learn from these artifacts, use them in their own work, and also, if desired, seek evidence of my claims made about them.

Significant Results

Over the course of 13 monthly jams, we learned more about working together. Some of these findings were significant in the small in that they helped us improve and learn. I mention this because in educational action research and Scholarship of Teaching, significance is often in-the-small: if the process worked with the intended class, then it is significant. Many of my writings and presentations about the Fam Jam experience mention particular areas where improvement was seen, including the quality of the source code, the formal structure of the music, the elegance of the gameplay, and the visual aesthetics.

Taking a wider view, the project gains further significance. I developed a tested structure for one-day Fam Jams. Through iterative reflection on what we learned, I was able to develop a list of our learning outcomes, which can then be used as potential learning outcomes for others.

As far as I know, mine is the only work of this kind. It has served as a talking point for several of my formal and informal presentations about game design and development.

I have talked to my students about this experience and shown them how I approach it. In this way, one of the significant outcomes of this work is that it models a scholarly way. Students are often inspired to make games because of a love of playing games. There are countless articles and videos explaining how to make games, but what I provide in my work is a framework for how to make games thoughtfully and within a scholarly method. The projects themselves have also served as reference points in my teaching, to show students what is possible in a one-day timebox.

A final point here that could be argued is that the projects themselves were significant because they brought joy to people outside my family. One of our goals was to spread some joy during a dreary season, and we succeeded. Discussing whether this is significant is a philosophical question that could sidetrack us, and my defense of this point is rooted in the transcendentals.

Effective Presentation

There are many layers to the question of effective presentation. I started writing blog posts about my experience from the beginning so that I could come to a clearer understanding. My initial hope was that interested people would search the Web and end up here. 

There is a lot of game design and development content on YouTube, but I did not originally see a way to talk about my FamJam experience on my own YouTube channel. Hooking into GodotCon would increase the chances that I could inspire people and get my findings into the hands of those looking for it. Those seeking information about running a game jam with their family would watch a video and might read a blog post, but unless they are a scholar themselves, they would be very unlikely to find or read a journal article. 

When I saw the call for presentations for an online version of GodotCon, I realized this was an opportunity to reach a much wider audience. GodotCon 2021 was on Jan 23, 2021, and the whole event was streamed on YouTubeThis excerpt includes only my presentation. The conference featured an online chat system whereby the online attendees could interact with the presenters. I engaged with the chat there after my presentation, which was another form of effective presentation.

As mentioned earlier, the software itself is also presented publicly, both the source code and the executables. Anyone who reads my reflections or is inspired by my work can see the actual fruits of our labor via GitHub.

Considering all this, I would argue that this project is potentially one of the most effectively presented works I have ever done. It hits multiple modes and gets information directly where it would be sought.

Reflective Critique

Critical self-evaluation is an essential element of this work: moving through the process toward my goals was a process of regular reflective critique. I have written numerous pieces about the FamJam project, all of which feature at least a modicum of reflection, while some are purely reflective. To illustrate the point, I've gathered links here to all my blog posts during the FamJam project.

  1. Fam Jam #1: Joe Johnson Gets Captured
  2. Fam Jam #2: The Rocket and UFO Game
  3. Fam Jam #4: Get the Food (and a few words about #3)
  4. Find the Ornaments: Thoughts about the December 2020 Fam Jam
  5. Reflecting on almost a year of FamJams: What the family learned
The work also produced these additional blog posts about technical issues that came up while jamming.

My family has continued to do game jams together, and we have experimented with varying the format to reach different learning outcomes. I have contemplated running workshops locally to promote this kind of family event, but the ongoing pandemic and other responsibilities limit such opportunities. I think it would be fascinating to do an ethnographic study of a Fam Jam, and such work would easily find publication in the field of serious games. I am not currently pursuing this line of research, however, since the return to in-person teaching has turned my attention toward studying phenomena around my university students rather than the local community.


That's it. I think this makes a strong case for the 2020-2021 FamJam project being an example of scholarship. I'm happy to hear any refutations you have to the points made above, as I'm sure it would be an opportunity to sharpen my own understanding of Glassick's framework.

No comments:

Post a Comment