Quiz Games and Learning Taxonomies

A few days ago, I posted about the SOLO Taxonomy and my desire to incorporate its ideas into educational game design. Yesterday's serious game design class provided just such an opportunity.

This week was my students' first opportunity to present game concepts. Over the coming weeks, they will be presenting additional concepts as well as multiple iterations of prototypes, as I described in my course planning post. This was their first real opportunity to show something original, and most of them chose to make games based around the International Space Station, a topic of particular interest to the class' partner, The Children's Museum of Indianapolis. The designs varied in genre, scope, and clarity, as to be expected from a first pass. One particular design was essentially a series of questions, where each question was worth a point. That is, this design was essentially a quiz, with no interesting or relevant decisions being made by the player.

I told the students how, in general, I don't have much interest in quiz-style games. Coincidentally, they had just read The Education Arcade whitepaper, “Moving Learning Games Forward,” and I was able to draw on some of the ideas in that whitepaper to justify my indifference. Looking at quiz-style games from the perspective of SOLO, they clearly fall into either the unistructural or multistructural levels of the taxonomy. That is, either the player is recalling a single idea (identify or name) or sets of ideas (list, enumerate). SOLO makes it clear that these games exist at the lower level of learning activity. By contrast, a game that requires analysis or application of domain knowledge would place the learning activity at the relational level, and a game that requires invention or theory-building would be at the extended abstract level. These levels inherently reflect what Burgun calls endogenously meaningful ambiguous decision-making, which I find to be a useful heuristic for evaluating whether or not a game concept will be worth prototyping.

It seems to me that identifying quiz games as low-level learning is just as easy with Bloom's Taxonomy of the Cognitive Domain: quiz games are usually situated at the Remember or possibly the Understand level, the very lowest levels of the taxonomy. The applicability of SOLO vs. Bloom seems to get murkier once when adds meaningful choices to the game. This gives me a direction in which to consider these taxonomies as I move forward with this semester's serious game design efforts.

Configuring Java development on Linux

I run Linux Mint on my workstation and my laptop, and so my students see me using it in the classroom and in my office. A few students have dipped their toes into GNU/Linux this semester, but configuring a useful and robust Java development environment can be challenging for someone new to the Linux ecosystem. I am writing this blog post to document how I set up my environment, and why I do it this way. I will assume a Debian-based Linux distribution, although I have used this approach on RPM-based distros as well.

The first step is to get rid of the default OpenJDK installation. I sympathize with the goals of the OpenJDK community and of free software in general, but at the same time, I am content to run Oracle's JDK for development purposes. Instructions for doing this depend on your distribution, and a quick search will likely show something like this:

sudo apt-get purge openjdk*
sudo apt-get purge icedtea-* openjdk-*

It probably doesn't matter if you keep the OpenJDK installed, but since we're not using it, we may as well clean it out.

In the bad old days, I would install all my Java libraries into /usr/local, and then I would curse myself over getting the permissions right. These days, I just put them in an apps folder off of my home directory—that is, at /home/pvg/apps. If you're very new to *nix, not that the tilde (~) is used to represent your home directory, so I also refer to this directory as ~/apps.

Next step is to download the JDK from Oracle. I will assume we're using Java SE 7, release 40, just for the sake of discussion. The approach is the same for any release, and in fact, allows for parallel installation of different releases. I extract the JDK into ~/apps/jdk1.7.0_40 using standard archiving tools. Then, set up a symbolic link in ~/apps to point to this particular release of the JDK. From the ~/apps directory, it looks like this:

ln -s jdk1.7.0_40 jdk1.7.0

Now, I can reference ~/apps/jdk1.7.0 and it goes immediately to release 40. If I want to update my Java version, I simply drop it into ~/apps and update the symbolic link—no other changes are required.

Making this JDK accessible from the command line just requires a few tweaks to the .bashrc file. This is a hidden file in your home directory; this means you won't see it with a standard ls command, but you can if you do ls -a. Since I use emacs, I can open the file with emacs .bashrc from my home directory; vim or another editor would work similarly, except not quite as well because they are not emacs. What we'll do is add two export commands, one to specify JAVA_HOME and another to tweak the path. I put them at the bottom of my .bashrc, along with a comment reminding myself what I was doing. The result looks like this:

# Use the JDK in ~/apps
export JAVA_HOME=/home/pvg/apps/jdk1.7.0
export PATH=$JAVA_HOME/bin:$PATH


You will need to open a new console or source the .bashrc file to see the change. Now, if you type the command java -version, you should see that you're running JDK 1.7.0 Release 40 (in my example). Note that you can use the which command to determine which executable is being run, so which java should return /home/pvg/apps/jdk1.7.0/bin/java.

My installation strategy for Maven, Ant, and Eclipse is similar. This gives me fine-grained control over which version I have installed, and I can hop between versions by simply tweaking my path. For example, in my ~/apps directory I have apache-maven-3.1.0, and then in ~/.bashrc I have added the following lines:

export MAVEN_HOME=/home/pvg/apps/apache-maven-3.1.0
export PATH=$MAVEN_HOME/bin:$PATH

I keep a link to Eclipse on my desktop for ease of access. There is a trick to making this work in KDE; I don't know if this is applicable to other desktop environments. In my Eclipse configuration file (~/apps/eclipse/eclipse.ini) I have added a link to the JavaVM that should be used to start Eclipse. The two lines look like this:

-vm
/home/pvg/apps/jdk1.7.0/bin/java

These go before the -vmargs flag that is probably already there.

I hope this is useful information for my students and others who are experimenting with configuring GNU/Linux systems for Java development. If you have other tips or approaches, feel free to leave them in the comments.

Bloom's Taxonomy vs. SOLO for serious game design

I think the first time I came across Bloom's Taxonomy for the Cognitive Domain was in my first two years as a faculty member, when I was just dipping my toes into the science of learning. I have written about Bloom's taxonomy before, including my preferred presentation of it:

I have found this presentation useful in teaching and practicing game design. Many games have a learning structure that follows the lower three levels. First, you are given some command that you must remember, such as "press right to move right" or "press B to jump." Then, you are given some context in which to understand the effect these commands have on the world: run to the right and the screen begins to scroll, then continue running and fall into a pit, forcing you to start again. Now, you have the context to apply what you understood, combining running and jumping to get over the pit.

I contend that most games don't really go beyond that. I hesitate to say that recognizing the pit as an obstacle constitutes analysis or that combining running and jumping is any meaningful synthesis. Most games do not teach the player to learn how to evaluate the game against the rest of their mental models either. The modern phenomenon of in-game "crafting" is similarly contained in the lower half of the taxonomy: remember that oil can be combined with flames to create an explosion, understand that this explosion hurts enemies, and you can apply this to defeat the enemy du jour. My contention is that games are designed experiences, and that players are remembering, understanding, and applying the constraints designed for them. Even Minecraft, with all its cultural importance, mostly has kids understanding and applying the rules of the world to build interesting things. Certainly, a few users recognize that the pieces they have been given can be used to synthesize something new, such as building circuits out of redstone, but my observation is that these are a minority—and the people who are following the tutorials to build copies, they are back to remembering, understanding, and applying someone else's design.

This use of Bloom's taxonomy is useful for game design thought experiments and for discussion, but it wasn't until GLS 2013 that I found out that many teacher educators are teaching Bloom's taxonomy as dogma, not as a useful sounding board. A poster session was presenting an alternative taxonomy, one that used the same labels but put analysis and synthesis closer to the bottom, using these to describe the kinds of tinkering users do with digital technology. (Clearly, they are using different interpretations of these labels than I am, but that's intellectual freedom for you.) This alternative was based on anecdotes and observations, much like Bloom's original, but this leads to a problem: Bloom's taxonomy is presented as a predictive, scientific model, but as far as I can tell, it is not empirical. In fact, cognitive science tells us that the human brain does not actually follow the steps presented in either of these taxonomies.

Reading Hattie and Yates' Visible Learning and the Science of How We Learn, which was recommended on Grant Wiggins' informative and inspiring blog, I was reminded of the fact that Bloom's Taxonomy does not represent a modern understanding of learning. The book introduced a different model, one that was first defined in 1982 but that I had never encountered before. It is Biggs and Collis' SOLO Taxonomy, where "SOLO" stands for Structure of the Observed Learning Outcome. It is summarized in this image, which is hosted on Biggs' site:

Hattie and Yates conveniently summarize the taxonomy—one idea, many ideas, relate the ideas, extend the ideas—and point out that the first two deal with surface knowing while the latter two deal with deeper knowing. The figure points out that each level of the taxonomy is associated with key observations that can be aligned with assessments. For example, if a student can list key elements of a domain but cannot apply, justify, or criticize them, you could conclude they are at the multistructured ("many ideas") level of SOLO. It strikes me that this has the potential to be powerful in my teaching, and I look forward to incorporating it.

So, how can SOLO contribute to an understanding of game design? It seems we run into the same limitations that hinder game-based learning, primarily those of transfer. Notice that the extend abstract level of SOLO explicitly refers to generalization to a new domain. It's true that I can learn how to jump over pits or destroy goblins with flaming oil, but this knowledge is locked away in the affordances of the game. This perspective is taken from Linderoth's work, particularly "Why gamers don't learn more," which applies the ecological theory of development to explain why learning from games does not transfer.

If nothing else, the SOLO Taxonomy can provide both a target for serious games and guidance toward assessments. Given a content area and the desire to create a game to teach it, I can target a specific level within SOLO. For example, if I only want players to emerge with surface-level knowledge, I might target the multistructural level, but if I wanted players to be able to connect the content to something else they know, I would need to target extended abstract. Then, I can reference the key words from the corresponding level of the taxonomy, and use these to define an assessment of whether or not the game worked. In fact, it strikes me that one could also take key words from the adjacent levels, and use this to detect extremes. As my game design course is wrapping up the preliminaries and moving into game concepts, I will try to create an opportunity to try this.

Learning from Failure—as a game mechanism

One of my favorite insights from the scholarship of teaching and learning is that, essentially, all learning is learning from failure. Every time a person encounters a mismatch between a mental model and reality, it is an opportunity to learn. I have been working for several years to incorporate this idea into my teaching, notably in the expansion of the CS222 project from two to three increments, specifically because it gives teams more chances for safe failure. Indeed, one of my favorite descriptions of university is that it is a "safe fail" environment, where students can fail in order to learn, without the economic cost it would take outside of academia.

It was only recently that I thought about what this phenomenon means in terms of game design, and in role-playing games in particular. Character advancement is a common component of RPGs, giving the player the opportunity to increase the skills and capabilities of his character. Such advancement is generally limited through in-game resource management, such as accumulating threshold values of experience points or by earning sufficient skill points to expend on new abilities. Dungeons and Dragons set the precedent whereby experience points are earned by overcoming obstacles, most often through combat but (with a good DM) also by other means. This has become the de facto standard and can be seen in all manner of modern games, including computer RPGs and RPG-inspired boardgames: success earns points that are used to gain skill.

Recently, I came across The Zorcerer of Zo (ZoZ), a role-playing game by Chad Underkoffler published by Atomic Sock Monkey. In fact, I have had a copy for almost a year, having bought it in a Family-Friendly RPG Bundle of Holding, but it wasn't until a few weeks ago that I read it. The game is based on Baum's Oz series, and since my two older boys and I are currently on the tenth book, we are quite familiar with the setting.

ZoZ uses the "good parts" variant of Underkoffler's Prose Descriptive Qualities (PDQ) system, the full version of which is described in a free document from Atomic Sock Monkey. A critical aspect of the system is that it eschews conventional attributes, skills, and inventory for qualities such as "world-traveler," "small," or "afraid of cats." Each quality is ranked, the range starting at Poor [-2], Average [0], and Good [+2], with each rank giving an adjustment to the 2d6 used for all conflicts. Again, for full detail, check that free PDF. It was a lot of fun to create ZoZ characters with my sons using this approach, since most of the time was spent describing the character's background and interests. One is a talking mouse from an island of merchants, who is in fact a small world-traveler who is afraid of cats. The other is a rockman warrior from a far-away island, who, since he does not need to breathe, simply walked through the ocean to Zo after hearing that it was a nice place to visit. There are no classes or races, and the setting lends itself to this kind of creative storytelling: it doesn't matter that there were no rock men in the Oz books, as long as my son wanted to make one, it was easy to create.

It is the character advancement system of ZoZ that got me thinking about the backwards nature of the conventional RPG systems. When a character encounters a conflict with a chance of failure, the player rolls 2d6 against a target number. If the roll meets or exceeds the target, the character succeeds. If not, the character fails and gains a Learning Point, and a player can improve his character by later spending these points. Consider how this matches what we know about how the brain works: when everything goes well, we actually learn very little, but when we make mistakes and reflect on them, our skills and knowledge improve.

It should not be missed that earning a Learning Point when missing a roll takes a lot of the sting away from failure. Although you did not get the result you wanted, you still get something: the opportunity to learn. This is a true yet countercultural statement. Conventional wisdom is that failure is bad, and this is a dangerous meme that is hard to overcome. Nowhere is it so endemic and unquestioned as in formal schooling environments, the very places whose mission it is (or should be) to instill a love of learning—and a love of learning necessitates tolerance for, if not embracing of, failure.

A player is still rewarded for success, of course, but it is done through the narrative. The use of narrative as a feedback mechanism is cleverly addressed in Koster's essay, "Narrative is not a game mechanic," which I recommend to anyone interested in weaving games and authored narratives together. In ZoZ, players can earn Hero Points for their characters by taking especially brave or noble actions. Hero Points are not used for character advancement, but rather to shift the story in the players' favor such as by getting hints, trading in favors, or getting a one-time boost to a roll—a sort of Oz Karma.

I have been impressed with Zorcerer of Zo and enjoyed playing it with my family. It has inspired me to consider how I might incorporate learning from failure as a game mechanism in my own designs, and more generally, how I might take more ideas from my scholarship and apply them to my game designs. This semester, I will be engaging in an experiment in public game design in concert with my honors colloquium on serious game design, and I expect to use this blog for that purpose. Watch this space for further announcements and designs, and as always, feel free to share your thoughts in the comments section.