Quiz Games and Learning Taxonomies

A few days ago, I posted about the SOLO Taxonomy and my desire to incorporate its ideas into educational game design. Yesterday's serious game design class provided just such an opportunity.

This week was my students' first opportunity to present game concepts. Over the coming weeks, they will be presenting additional concepts as well as multiple iterations of prototypes, as I described in my course planning post. This was their first real opportunity to show something original, and most of them chose to make games based around the International Space Station, a topic of particular interest to the class' partner, The Children's Museum of Indianapolis. The designs varied in genre, scope, and clarity, as to be expected from a first pass. One particular design was essentially a series of questions, where each question was worth a point. That is, this design was essentially a quiz, with no interesting or relevant decisions being made by the player.

I told the students how, in general, I don't have much interest in quiz-style games. Coincidentally, they had just read The Education Arcade whitepaper, “Moving Learning Games Forward,” and I was able to draw on some of the ideas in that whitepaper to justify my indifference. Looking at quiz-style games from the perspective of SOLO, they clearly fall into either the unistructural or multistructural levels of the taxonomy. That is, either the player is recalling a single idea (identify or name) or sets of ideas (list, enumerate). SOLO makes it clear that these games exist at the lower level of learning activity. By contrast, a game that requires analysis or application of domain knowledge would place the learning activity at the relational level, and a game that requires invention or theory-building would be at the extended abstract level. These levels inherently reflect what Burgun calls endogenously meaningful ambiguous decision-making, which I find to be a useful heuristic for evaluating whether or not a game concept will be worth prototyping.

It seems to me that identifying quiz games as low-level learning is just as easy with Bloom's Taxonomy of the Cognitive Domain: quiz games are usually situated at the Remember or possibly the Understand level, the very lowest levels of the taxonomy. The applicability of SOLO vs. Bloom seems to get murkier once when adds meaningful choices to the game. This gives me a direction in which to consider these taxonomies as I move forward with this semester's serious game design efforts.

Bloom's Taxonomy vs. SOLO for serious game design

I think the first time I came across Bloom's Taxonomy for the Cognitive Domain was in my first two years as a faculty member, when I was just dipping my toes into the science of learning. I have written about Bloom's taxonomy before, including my preferred presentation of it:

I have found this presentation useful in teaching and practicing game design. Many games have a learning structure that follows the lower three levels. First, you are given some command that you must remember, such as "press right to move right" or "press B to jump." Then, you are given some context in which to understand the effect these commands have on the world: run to the right and the screen begins to scroll, then continue running and fall into a pit, forcing you to start again. Now, you have the context to apply what you understood, combining running and jumping to get over the pit.

I contend that most games don't really go beyond that. I hesitate to say that recognizing the pit as an obstacle constitutes analysis or that combining running and jumping is any meaningful synthesis. Most games do not teach the player to learn how to evaluate the game against the rest of their mental models either. The modern phenomenon of in-game "crafting" is similarly contained in the lower half of the taxonomy: remember that oil can be combined with flames to create an explosion, understand that this explosion hurts enemies, and you can apply this to defeat the enemy du jour. My contention is that games are designed experiences, and that players are remembering, understanding, and applying the constraints designed for them. Even Minecraft, with all its cultural importance, mostly has kids understanding and applying the rules of the world to build interesting things. Certainly, a few users recognize that the pieces they have been given can be used to synthesize something new, such as building circuits out of redstone, but my observation is that these are a minority—and the people who are following the tutorials to build copies, they are back to remembering, understanding, and applying someone else's design.

This use of Bloom's taxonomy is useful for game design thought experiments and for discussion, but it wasn't until GLS 2013 that I found out that many teacher educators are teaching Bloom's taxonomy as dogma, not as a useful sounding board. A poster session was presenting an alternative taxonomy, one that used the same labels but put analysis and synthesis closer to the bottom, using these to describe the kinds of tinkering users do with digital technology. (Clearly, they are using different interpretations of these labels than I am, but that's intellectual freedom for you.) This alternative was based on anecdotes and observations, much like Bloom's original, but this leads to a problem: Bloom's taxonomy is presented as a predictive, scientific model, but as far as I can tell, it is not empirical. In fact, cognitive science tells us that the human brain does not actually follow the steps presented in either of these taxonomies.

Reading Hattie and Yates' Visible Learning and the Science of How We Learn, which was recommended on Grant Wiggins' informative and inspiring blog, I was reminded of the fact that Bloom's Taxonomy does not represent a modern understanding of learning. The book introduced a different model, one that was first defined in 1982 but that I had never encountered before. It is Biggs and Collis' SOLO Taxonomy, where "SOLO" stands for Structure of the Observed Learning Outcome. It is summarized in this image, which is hosted on Biggs' site:

Hattie and Yates conveniently summarize the taxonomy—one idea, many ideas, relate the ideas, extend the ideas—and point out that the first two deal with surface knowing while the latter two deal with deeper knowing. The figure points out that each level of the taxonomy is associated with key observations that can be aligned with assessments. For example, if a student can list key elements of a domain but cannot apply, justify, or criticize them, you could conclude they are at the multistructured ("many ideas") level of SOLO. It strikes me that this has the potential to be powerful in my teaching, and I look forward to incorporating it.

So, how can SOLO contribute to an understanding of game design? It seems we run into the same limitations that hinder game-based learning, primarily those of transfer. Notice that the extend abstract level of SOLO explicitly refers to generalization to a new domain. It's true that I can learn how to jump over pits or destroy goblins with flaming oil, but this knowledge is locked away in the affordances of the game. This perspective is taken from Linderoth's work, particularly "Why gamers don't learn more," which applies the ecological theory of development to explain why learning from games does not transfer.

If nothing else, the SOLO Taxonomy can provide both a target for serious games and guidance toward assessments. Given a content area and the desire to create a game to teach it, I can target a specific level within SOLO. For example, if I only want players to emerge with surface-level knowledge, I might target the multistructural level, but if I wanted players to be able to connect the content to something else they know, I would need to target extended abstract. Then, I can reference the key words from the corresponding level of the taxonomy, and use these to define an assessment of whether or not the game worked. In fact, it strikes me that one could also take key words from the adjacent levels, and use this to detect extremes. As my game design course is wrapping up the preliminaries and moving into game concepts, I will try to create an opportunity to try this.

Bloom's Taxonomy of the Cognitive Domain

Two nights ago, I gave a presentation at the 2011 IGDA Indianapolis Charity Toy Drive, a great event that I hope becomes an annual tradition. My presentation was entitled Fun, Learning, Games, and Responsible Design (in under 20 minutes). I gave an overview about the relationship between games and learning, and my core message was a rallying cry, asking designers to seriously consider what players learn by playing their games. My perspectives on this are heavily influenced by Raph Koster's work, and as I told the crowd, every designer should read Theory of Fun for Game Design.

I generally don't post slides from my talks because the slides don't stand alone: you could not reconstruct the message by only looking at the slides. I agree with Martin Fowler, that "slideuments" are flawed by definition. However, there was one slide from my talk that really seemed to resonate with the crowd. I based much of my presentation on applications of Bloom's Taxonomy of the Cognitive Domain toward gameplay experiences. Wikipedia has a nice, simple diagram that presents the taxonomy, and I have used this in presentations and handouts before---but this was always when using OpenOffice.org or LibreOffice directly. For the IGDA presentation, I needed a PDF, and something about the conversion to PDF left the diagram quite jaggy. To solve my problem, I made my own variant on the public domain image within LibreOffice, and it looks a little something like this:

This variation is CC-BY in case you want to insert it directly into your own presentations, papers, posters, etc. An ODG version is available as well. Full license terms below.

Bloom's Taxonomy of the Cognitive Domain by Paul Gestwicki is licensed under a Creative Commons Attribution 3.0 Unported License.

Based on a work at en.wikipedia.org.

Bloom meets Design Thinking

I have been working on a book chapter the explores many of the themes on this blog, particularly those relating to the Morgan's Raid project. Here is a sketch that did not make it into the chapter: an exercise in comparing Bloom's Taxonomy of the Cognitive Domain to a design thinking process.

I think the idea is compelling, but I have not had the time yet to fully flesh it out. My initial analysis revealed what one might expect: most of the design thinking steps explicitly involve cognitive activity at the highest level of the cognitive domain. Maybe there's something more there, and maybe there isn't. In any case, it was fun to draw.