Quiz Games and Learning Taxonomies

A few days ago, I posted about the SOLO Taxonomy and my desire to incorporate its ideas into educational game design. Yesterday's serious game design class provided just such an opportunity.

This week was my students' first opportunity to present game concepts. Over the coming weeks, they will be presenting additional concepts as well as multiple iterations of prototypes, as I described in my course planning post. This was their first real opportunity to show something original, and most of them chose to make games based around the International Space Station, a topic of particular interest to the class' partner, The Children's Museum of Indianapolis. The designs varied in genre, scope, and clarity, as to be expected from a first pass. One particular design was essentially a series of questions, where each question was worth a point. That is, this design was essentially a quiz, with no interesting or relevant decisions being made by the player.

I told the students how, in general, I don't have much interest in quiz-style games. Coincidentally, they had just read The Education Arcade whitepaper, “Moving Learning Games Forward,” and I was able to draw on some of the ideas in that whitepaper to justify my indifference. Looking at quiz-style games from the perspective of SOLO, they clearly fall into either the unistructural or multistructural levels of the taxonomy. That is, either the player is recalling a single idea (identify or name) or sets of ideas (list, enumerate). SOLO makes it clear that these games exist at the lower level of learning activity. By contrast, a game that requires analysis or application of domain knowledge would place the learning activity at the relational level, and a game that requires invention or theory-building would be at the extended abstract level. These levels inherently reflect what Burgun calls endogenously meaningful ambiguous decision-making, which I find to be a useful heuristic for evaluating whether or not a game concept will be worth prototyping.

It seems to me that identifying quiz games as low-level learning is just as easy with Bloom's Taxonomy of the Cognitive Domain: quiz games are usually situated at the Remember or possibly the Understand level, the very lowest levels of the taxonomy. The applicability of SOLO vs. Bloom seems to get murkier once when adds meaningful choices to the game. This gives me a direction in which to consider these taxonomies as I move forward with this semester's serious game design efforts.

Bloom's Taxonomy vs. SOLO for serious game design

I think the first time I came across Bloom's Taxonomy for the Cognitive Domain was in my first two years as a faculty member, when I was just dipping my toes into the science of learning. I have written about Bloom's taxonomy before, including my preferred presentation of it:

I have found this presentation useful in teaching and practicing game design. Many games have a learning structure that follows the lower three levels. First, you are given some command that you must remember, such as "press right to move right" or "press B to jump." Then, you are given some context in which to understand the effect these commands have on the world: run to the right and the screen begins to scroll, then continue running and fall into a pit, forcing you to start again. Now, you have the context to apply what you understood, combining running and jumping to get over the pit.

I contend that most games don't really go beyond that. I hesitate to say that recognizing the pit as an obstacle constitutes analysis or that combining running and jumping is any meaningful synthesis. Most games do not teach the player to learn how to evaluate the game against the rest of their mental models either. The modern phenomenon of in-game "crafting" is similarly contained in the lower half of the taxonomy: remember that oil can be combined with flames to create an explosion, understand that this explosion hurts enemies, and you can apply this to defeat the enemy du jour. My contention is that games are designed experiences, and that players are remembering, understanding, and applying the constraints designed for them. Even Minecraft, with all its cultural importance, mostly has kids understanding and applying the rules of the world to build interesting things. Certainly, a few users recognize that the pieces they have been given can be used to synthesize something new, such as building circuits out of redstone, but my observation is that these are a minority—and the people who are following the tutorials to build copies, they are back to remembering, understanding, and applying someone else's design.

This use of Bloom's taxonomy is useful for game design thought experiments and for discussion, but it wasn't until GLS 2013 that I found out that many teacher educators are teaching Bloom's taxonomy as dogma, not as a useful sounding board. A poster session was presenting an alternative taxonomy, one that used the same labels but put analysis and synthesis closer to the bottom, using these to describe the kinds of tinkering users do with digital technology. (Clearly, they are using different interpretations of these labels than I am, but that's intellectual freedom for you.) This alternative was based on anecdotes and observations, much like Bloom's original, but this leads to a problem: Bloom's taxonomy is presented as a predictive, scientific model, but as far as I can tell, it is not empirical. In fact, cognitive science tells us that the human brain does not actually follow the steps presented in either of these taxonomies.

Reading Hattie and Yates' Visible Learning and the Science of How We Learn, which was recommended on Grant Wiggins' informative and inspiring blog, I was reminded of the fact that Bloom's Taxonomy does not represent a modern understanding of learning. The book introduced a different model, one that was first defined in 1982 but that I had never encountered before. It is Biggs and Collis' SOLO Taxonomy, where "SOLO" stands for Structure of the Observed Learning Outcome. It is summarized in this image, which is hosted on Biggs' site:

Hattie and Yates conveniently summarize the taxonomy—one idea, many ideas, relate the ideas, extend the ideas—and point out that the first two deal with surface knowing while the latter two deal with deeper knowing. The figure points out that each level of the taxonomy is associated with key observations that can be aligned with assessments. For example, if a student can list key elements of a domain but cannot apply, justify, or criticize them, you could conclude they are at the multistructured ("many ideas") level of SOLO. It strikes me that this has the potential to be powerful in my teaching, and I look forward to incorporating it.

So, how can SOLO contribute to an understanding of game design? It seems we run into the same limitations that hinder game-based learning, primarily those of transfer. Notice that the extend abstract level of SOLO explicitly refers to generalization to a new domain. It's true that I can learn how to jump over pits or destroy goblins with flaming oil, but this knowledge is locked away in the affordances of the game. This perspective is taken from Linderoth's work, particularly "Why gamers don't learn more," which applies the ecological theory of development to explain why learning from games does not transfer.

If nothing else, the SOLO Taxonomy can provide both a target for serious games and guidance toward assessments. Given a content area and the desire to create a game to teach it, I can target a specific level within SOLO. For example, if I only want players to emerge with surface-level knowledge, I might target the multistructural level, but if I wanted players to be able to connect the content to something else they know, I would need to target extended abstract. Then, I can reference the key words from the corresponding level of the taxonomy, and use these to define an assessment of whether or not the game worked. In fact, it strikes me that one could also take key words from the adjacent levels, and use this to detect extremes. As my game design course is wrapping up the preliminaries and moving into game concepts, I will try to create an opportunity to try this.