Saturday, March 16, 2019

Painting Middle Earth Quest

Here's a project that was years in the making: painting the miniatures from Middle Earth Quest.
Middle Earth Quest is an asymmetric one-vs-all game, one player as Sauron and the rest as heroes of the Free Peoples of Middle Earth. The setting is between The Hobbit and The Lord of the Rings, as Sauron is establishing his foothold while the Free Peoples are trying to rally to face him. This is a game that I remember being fun, although I haven't played in years. How do I know that? Well, if you go way back to a post from December 2014, I mentioned that I had painted Argalad, one of the heroes, and I know I primed and based them all in one go. I have another post from July 2015 in which I described how I painted the Witch King. Some time after that, I painted Gothmog, and I started the Ringwraiths, and then the figures sat around for years, first on my painting table, and then in a box. Some time in between, Fantasy Flight Games removed all reference to the game from their site, so it looks like there won't be any more printings or expansions; it must not have sold well.

I actually had Argalad, the Witch King, and Gothmog sitting on my desk the whole time, at first because I liked how they looked and wanted to inspire myself to continue the set, and then because of inertia. A few months ago, #2 Son finished reading The Hobbit, which got me thinking about this game again. Would he enjoy playing this game, now that he's older and knows a little bit of the LotR lore? Maybe it was time to finally get back to the Ringwraiths and finish this set.

Without further ado, here's a quick summary of each of the figures from this game. I'll start with the ones I painted some years ago and then move through them in the order I addressed them.


Argalad is the token elf. Or is that Tolkien elf? I remember being quite pleased with him at the time, and I spent quite a while on him. It's worth noting that all these figures were primed in black, because that's how I was working at the time, when I was less than a year back into the hobby. I'm sure it was all done with layering and some washes, but the only part that really sticks in my memory was the silver embroidery on the cape. That was done by mixing silver paint with Future Floor Polish, which flowed really well into the engraved pattern on the cloak.


There's a whole post about the Witch King of Angmar, so I'll just leave you to check the link if you want to read about that.

Here is Gothmog, who I painted some time between winter 2015 and now, but certainly nearer then than now. I know I gave him quite a bit of time, but I don't remember much else about the process. He's pretty dark, but I did want all the villains to have clearly dark color schemes. I remember that, playing the game, it was sometimes hard to tell at a glance who was heroes and who was villains. This is particularly difficult for the two upcoming horse-riding figures. As you'll see later, I think I did meet the goal of having the "sides" of the game clearly contrast.

Now we move into the figures that I have painted in the last few weeks. It was interesting, but also frustrating, to come back to these figures—especially the Ringwraiths. At the time I started this project, I thought it was a good idea to prime in black (as Dr. Faust does) and also to work on the bases first. All of these figures were in jet black primer but with painted bases. If I were to start the project today, I would build the bases, use zenithal priming over the whole figure (as I started with Massive Darkness), paint the bases, and then paint the figures. I thought about re-priming the figures, but I decided it would be a fun to try to keep them as I started them, as a sort of artistic archaeology project.


Getting into the Ringwraiths reminded me why I stopped this project. Horses are the worst. These figures are pre-assembled, so there are lots of hard-to-reach areas on the miniature. The sculpts are not very good, with flat-faced hooded men. It's also black-robed guys on black horses. It's hard to think of anything that could make this a worse project! I had indeed started painting the horses some four years ago, but it was hardly noticeable: I had done very faint layered grey highlights and then given up.

I recently watched Dr. Faust's episode on highlighting black in which he mixed a few different tones of black onto one figure. (Ironically, that figure is not primed in black!) He suggests that in most cases you wouldn't mix blacks on one figure, but looking at the Ringwraiths, I thought this would be good exercise if nothing else. I mixed a warm black by adding some VMC Flat Brown, which I used for the horses, and then a cool black by adding VMC Deep Sky Blue, which I used for the robes. I overhighlighted the robes in the first pass and then used a black ink glaze over the whole thing to bring it down. Unfortunately, I used my Liquitex Glaze Medium for this, which left the robes super glossy. I knew my matte varnish would take that away, but it made it really hard to compare the blacks of the robes with the blacks of the horses because the glossiness contrast dominated the viewing.

These guys sat on my desk while I painted the rest of the series too, but in anything but the best light, they really just looked like a black blob. I did go in and touch up the highlights on the robes, even before varnishing, and I also punched the leather parts way up. I had been going with a dark leather look, because wouldn't you put dark leather on your black horse if you were a black rider? Yes, you would, but it wouldn't help your miniature any.

In the end, they still look like a bit of a black lump, but at least they're done, which is better than they have been in years.


Here's the Black Serpent. The last thing I wanted to paint after the Ringwraiths was more horses, but decided that getting the horses done was actually the best thing for me to do. I think it turned out fairly well, although it was working on this one where I made a conscious decision that the rest of them would only be "good enough." I was much more interested in getting this set done than in making any showpieces. Also, these sculpts are not very good. The game is from 2009, which is before these sorts of miniatures became centerpieces of marketing strategies a la Kickstarter. The frustrating was exacerbated by the fact that I didn't used to spend as much time cleaning mold lines and flash off of the minis, so some have some awkward tags and creases.

Let me mention a funny thing about the capes. All the caped figures have designs on the capes, but these designs are actually etched into the cape. This makes them look decent unpainted, but it's not actually how a design on cloth "works." Also, this is a place where the detail is not very clear, so it's hard to see the real edges. Argalad's cape turned out pretty good, and I thought about doing something similar with the Black Serpent. However, as I worked on the cape, I realized that if he's truly "The Black Serpent," then he really should have a black serpent on him, shouldn't he? I'm glad I used the etched design as a border here rather than anything else, since it provided convenient lines to paint in, and the paint job makes the actual etching invisible.


The last of the villains is the Mouth of Sauron, who I remember being one of my favorite cards in Middle Earth: The Wizards, which of course remains the grandest Middle Earth game adaptation ever created. In any case, in keeping with the "make the villains wear black" theme, I decided to go with a cool black and warm, maroon trim. This provides a nice contrast both in temperature and in hue but keeping relatively similar saturation. I think the metallic and bone provide more visual interest to the figure as well, so although he's only a few colors and relatively simple, the result is nice.


Which hero to start with? The one one the horse, of course. This is Eomer, clearly from Rohan. He took the most time out of any of them, in part for his size and in part because of his detail. There are lot of greens, browns, greys, yellows, and golds on the card art that don't exactly match those of the sculpt, but it comes close. I also think I did a decent job of the horse, which was done almost entirely with layering.

The cape is the weakest link. I tried to reproduce my approach from Argalad and make it look like there is gold embroidery on his cape. The details on this cape were too ambitious for the casting, however, and it's kind of hard to tell what's going on there, unlike the simple border of Argalad's. I decided to keep it rather than repaint it. Good enough, I say, and most importantly, no more horses.


Next up is the figure that I have long considered the worst of the bunch: Thálin. He's basically a lump of plastic. All right, maybe he's not all bad, but if he stood up and let his arms rest, they would reach past his knees. Also, his back is clearly sculpted to be a natural material of some kind, but who would want to leave their back so unprotected? It's an odd piece.

Whereas the cloaked heroes have etched designs on their cloaks, Thálin's are in his ... is it a tabard? A loincloth? I'm not sure. Anyway, on the card art there's a clear yellow-on-red pattern here, and I copied that idea onto my painting. This part is actually really strong, adding visual interest to an otherwise leather-and-metal warrior.


Berevor is a ranger who provided a chance to play with shades of green. I think the different greens look good together, with enough difference to be visually interesting, and the brown leather bits give her a good earthy tone. The slightly bluish color also adds a bit of cool contrast to the more dominant warm greens.


Last up is Eleanor from the White City. She has really sharp contrasts between her dominant colors, which I think came out strongly in the painting. I originally had the tree on her chest painted in white also, but looking again at the card art, it was clearly silver. I painted that over in silver, while the rear is simply white. I think hers is another case where a fairly limited palette makes her visually distinct and interesting, without needing lots of bells and whistles.

I'm glad to have this set complete, and I look forward to getting these to the table some time soon. Re-reading the rulebook, I was reminded about how it is a bit fiddly, which makes me question whether #2 son would find it interesting or frustrating. We'll see. A nice thing about games is that they don't go bad, so I can always put these guys back in the box they were in for so long, and revisit them when the kids are at the age and interest to want to try it.

As seems to be customary for many of my painting posts, I want to close with a comment about the photography. My first two rounds of photos did not turn out well for the usual reasons: using the default camera app, I was getting light and dark lines from my lamp's frequency, and using Open Camera to adjust the shutter speed, I was getting poor color temperatures and washed out images. I ended up doing a different physical layout, as shown in the photo below:
I tried to make sure the lamp was pointing at the figure, which leaves the background slightly in shadow. If you look again at the photos, you see a slight gradient, which I think is fine though not artistically intended. Using Android's default camera app, I was able to eliminate the lamp-lines by raising the exposure to 0.7. Doing so manually meant that I could not refocus without it dropping back to 0.0, so I tried shooting all the figures without refocusing. This worked on some but not others, the latter of which I had to shoot again. If I were to do it again this way, I would have just focused each time and then adjusted the exposure, so as not to waste time taking out-of-focus shots.

Thanks for reading this tale of my multi-year project to paint Middle Earth Quest! As always, feel free to leave a comment.

Friday, February 15, 2019

Counting plays with a specific player on Board Game Geek

I started logging all my board game plays on BoardGameGeek in January 2016. In writing my end-of-year game reports (2016, 2017, 2018), I have wondered how many games I played with specific players during the year, but this proved to be a difficult number to find. The unofficial BoardGameGeek app that I use tells me how many games I have played with players in total, but this cannot be filtered by date. (Incidentally, I just discovered, when attempting to link to the app, that it is currently missing from the Google Play Store but that the author is working on getting it updated.) I asked a few friends over the years, but nobody knew of a way to get the data I wanted back out of BoardGameGeek. However, I am working on a scholarly piece for which this is important information, and so I decided it was worth some effort to figure it out.

BoardGameGeek provides an XML API for programmatic access to its content, including play data. Let's say for the sake of example that there is a user named "sample", and I want to get all their plays between January 1, 2016 and December 31, 2018. This query will provide a good start:

 https://boardgamegeek.com/xmlapi2/plays?username=sample&mindate=2016-01-01&maxdate=2018-12-31  

The results are limited to 100 per page, but there is no way to ask how many entries exist. Hence, to get all the data, one has to go page by page until the page is empty. Accessing a particular page of data is a simple matter of adding a page=N parameter to the end of the query.

I'm not interested in all plays, though; I am only interested in plays with a particular user. In my case, I am looking for matches by a given name and not a BoardGameGeek username, since this is how players are most easily added through my app. If I wanted to get all the plays with, say, Norm, than I need to dig into the XML and only count those where there is a "player" element with the value "Norm". For this, I can use XPath via xmllint.

After refreshing myself on some Bash fundamentals and finding this beautiful, idiolectic way of creating a do loop, I ended up with this script, which counts plays between BoardGameGeek fake user "sample" and his imaginary friend "Norm" between January 1, 2016 and December 31, 2018.
 #!/bin/bash  
 COUNT=0  
 PAGE=1  
 while  
   echo "Processing page $PAGE"  
   xml=`curl -s "https://boardgamegeek.com/xmlapi2/plays?username=sample&mindate=2016-01-01&maxdate=2018-12-31&page=$PAGE"`  
   plays=`xmllint --xpath 'count(/plays/play)' - <<< "$xml"`  
   pagecount=`xmllint --xpath 'count(/plays/play/players/player[@name="Norm"])' - <<< "$xml"`  
   COUNT=$(($COUNT + $pagecount))  
   PAGE=$(($PAGE + 1))  
   echo "Page $PAGE result is $pagecount out of $plays, so total is $COUNT"  
   [ "$plays" -gt 0 ]  
 do  
   :  
 done  

I learned a few interesting things writing this script and sharing it with friends. Script-wizard Ben Dean helped me revise the script so that it would not generate temporary files on the filesystem, and in pursuing this goal, I learned about bash herestrings. I had also not previously scripted with XPath via xmllint, only within larger applications. This script got me what I needed to know: I logged 937 plays with my eldest son in the three full years I have been keeping track, but not counting the plays since January 1 of this year.

Hopefully this script will be useful to you. If nothing else, it will be useful to me next time I need to ask this question!

Tuesday, January 29, 2019

Global Game Jam 2019 and Kaiju Homecoming

This past weekend was Global Game Jam 2019, and I was the site organizer for Ball State's location. It was my third time serving as site organizer here. I want to capture a few thoughts about the weekend's events here before they get too far way.

We had 21 people register, almost all of whom showed up. There were several people who came on Friday but then didn't return, unfortunately. I have seen this each year, and I think it tends to be people who are interested novices. They don't know how to move from idea to implementation, and they are not familiar with the little failures that are to be expected along the way. I take a laissez faire approach to site organization: I don't manage teams or themes, but instead I encourage people to use the whiteboards around the room to gather interested people. I've thought about whether I should include more didactic interventions, but the truth is that teaching is already my job: the jam is about jamming after all. I would never kick an amateur musician out of a jam session if they wanted to participate, but I would also not be surprised if they left. Perhaps it sounds a bit harsh when I write it out, which makes me think I need to work on better packaging for my pragmatic philosophy.

This relates to something I want to share about this year's keynote. I had a sense that none of the four speakers were actually talking to my audience: mostly novice jammers. The thoughts they shared might resonate with people who already know what they are doing, but that's not helpful if you have no grounding. Let me pick on one particular example. Rami Ismail was the most on-point of the three speakers, but he made a claim in his presentation that you cannot do a game jam wrong. I firmly disagree: there are many, many ways to do things wrong. Here are several: being an unpleasant team member, for example by refusing to compromise on your ideas or by not keeping your commitments; focusing on accidentals such as title and credits screens rather than the essence of the game; taking too long to get to a minimally playable state; thinking that ideas have value rather than implementations; not considering packaging or deployment until just before the deadline. These are the kinds of mistakes that I have seen jammers make and, more to my point above, that I see novices make in my game design and game programming classes. In fact, I think it's much easier to do it wrong than to do it right—regardless of the quality of the final product. I think a rhetoric that says you cannot do things wrong sets novices up for mistakes and frustrations.

At the end of the jam, we had four playable projects. There were two more that were "done" but not uploaded, one of them because the jammer disregarded my and others' advice to stop trying to add features and instead to figure out how to package and upload his work, and the other because he could not make it on Sunday and didn't get around to uploading his.

Last year, my oldest son came and participated in the jam, making two games of his own in the time it took me to break one. This year, I encouraged him to come again, but I told him that I really wanted us to work together on something. He actually started working on his own anyway right after the theme announcement, but once I reminded him that I really wanted to collaborate, he was up for it.

We struggled to turn the theme ("What home means to you") into a game idea. One of our best sketches was of a game where the family stands around the counter, waiting for Mom to look away, reaching out and grabbing tidbits to eat before the siblings can. I imagined we might even use a polka soundtrack and be able to name it after the Shmenge hit, "Who stole the cabbage roll?" As we kept talking, though, we somehow pulled inspiration from Terror in Meeple City and the idea that kaiju have a home as well... and there are people in it, and we don't want those people there. This was the idea that became Kaiju Homecoming.

We built a minimally playable version in Unreal Engine just to make sure the pieces would fit, just dropping a plane onto some cylindrical columns and throwing a ball at it. We laughed from the get-go. I asked my friend Emma if she could make up some digital meeples, and so she and her friend Jessica provided these models. The next day, my son worked on some more original art for the floors and he did all the level design. We had to tweak the level a few times because we couldn't get the floors to stack nicely on the columns in the UE4 editor. Playing the game, you can see the physics solver create wobbles and even occasional collapses as a result. There are probably snapping or simulation sleeping settings that we could use to fix that, but I am really happy with the look and feel, given our constraints. My son was also in charge of all the audio direction except for the soundtrack: I found the music on Kevin MacLeod's site, and he reviewed sound effects using the free weekend access to SoundSnap. We had the game mostly finished on Saturday, and because of other family obligations, we only had a few hours to work on Sunday, right before the deadline. We got as much polish in as we could, and the game was complete.

The game runs best as a Windows 64-bit executable, and you can download the project for Windows from the Global Game Jam site. We also produced a Web build that I've hosted on GitHub: it's more accessible, but lacks the performance of the native version. All the source code and assets are in a repository on GitHub as well, although we actually used Perforce Helix for version control during development. Finally, for those who want the quick overview, I recorded this short gameplay video:


It was a fun weekend jamming with my son and seeing what the other jammers put together. I've started some conversations that might lead us to a different location for the 2020 jam, but that's a long way off to worry about planning. Now, I need to return to all the tasks that I put in the "I'll take care of this after Global Game Jam" bin.

Tuesday, January 15, 2019

What do you make?

I've dabbled with different approaches to having students introduce themselves. In my game design classes, I've generally asked students to share their last great play experiences. This allows us to share some lighthearted stories and start discussing the relationship between "play" and "game." In my HCI class the last two semesters, I've asked students what they want to learn. Last semester's experience with this was not very interesting, since most students said some variation on "how to design interfaces." It's not a bad answer, but it's also not surprising.

This semester, I decided to take a hint from my good friend Easel Monster, as explained in the first minute of this video:
Easel recommends that when you meet someone, you should ask what they make. I did this in both of my regular courses this semester. In the game studio course, which includes a multidisciplinary undergraduate teams, a lot of students gave games-related answers: video games, stories, fictional settings for tabletop roleplaying games, music. This was about what I expected since I recruited these students specifically to produce a video game: of course they would be makers at heart.

My other class is Human-Computer Interaction, an elective for Computer Science majors and minors. I have been out of teaching the prerequisite course (CS222) for several semesters. Where I used to recognize half or more of the students coming into my upper-level elective courses, this time I only knew a handful. That means only a handful knew me as well, although I do wonder if they thought they knew me through my reputation. In any case, on the first day, I took my customary mugshots, having each student hold up a sheet of paper with their name written on it. Having these photos makes it much easier for me to learn which names and faces go together. As they stood in front of the room, I asked them to give their name, where they are from (as broadly as they wish to answer), and to answer the question, "What do you make?" Some of them answered that they made Web pages or software, one in particular referencing software made at his job. Others said they made stories or, again, fictional worlds for tabletop roleplaying. Three specific answers jumped out at me as being especially interesting. One student answered that he makes sandwiches. That's a great thing to make! Someone in the class asked if he made special sandwiches, and he said he made a mean PB&J. Another answered that he made friends. You could practically hear the smiles break out among his classmates—is there anything better to make? Finally, one student said, "I don't know what I make, but I'm trying to figure that out." There's a curious one. On one hand, I say he's in the right place, so higher education can help him sort it all out. On the other hand, I wonder if it should be an entry exam to ask, "What do you make?" to help students think about it.

Thanks, Easel Monster!

Monday, December 31, 2018

The Games of 2018

With just a few hours left in 2018, I am going to go ahead and write up my "Games of 2018" post. Should anything change before midnight tonight, I'll quietly come in and edit the details. This is the third post in this series, the other two being 2016 and 2017.

Let's start with the numbers for the year. In 2018, I played 103 different games across a total of 548 plays. That's significantly more plays than last year (505) over about the same number of games (104). While my scholarly h-index barely crept up from 12 to 13, my games h-index rocketed from 15 to 20. My h-index for the year was 11, and here are the 11 games that I have played 11 or more times in 2018:

  • Gloomhaven (55)
  • Stuffed Fables (23)
  • Thunderstone Quest (22)
  • Go Nuts for Donuts (20)
  • Rhino Hero Super Battle (19)
  • Bärenpark (16)
  • Camel Up (16)
  • Champions of Midgard (12)
  • Carcassonne (11)
  • Clank! (11)
  • Rising Sun (11)

Summer 2018 was the Summer of Gloomhaven. My oldest son and I had a great time playing this BoardGameGeek chart-topper. I shared my painted base characters back in March, and I have also painted almost all the rest of the characters—all those we have unlocked. I'll make a spoilerful post about those once they are complete. The 55 plays are not all full-length games: some were short defeats. I remember one of the earlier scenarios was a terrible match for our characters, and we had several false starts on it. The last time we played was over the Summer break, and we haven't had the game to the table since. I was hoping we might finish the campaign over the Winter Break, but other activity has taken precedence; I have favored playing games that incorporate more people rather than just the two of us with Gloomhaven.

Stuffed Fables was a family Christmas gift in 2017. I shared my painted minis in June, and this is one I have played with my three older boys. The third son is particularly tickled to be involved in this kind of "big kids game" I think. We have really enjoyed it, and the several times, the writing has made us laugh aloud. We have one story left to go before finishing the book. Tracking plays of Stuffed Fables is actually a bit of tricky business. I decided to track each page as a play, following along the idea of each attempt at a Gloomhaven scenario being a play. One might argue that I should have counted "sessions" of Stuffed Fables. Yet, only once has one of our sessions been an entire story; usually we do a few pages and then store our stuff in baggies for another day.

Thunderstone Quest was a recent arrival via their second Kickstarter, as I described in my post about the painted minis. We have enjoyed this immensely, and I have taught it to a few friends. It and Clank! are easily my two favorite deckbuilding games in this genre, combining Dominion-style deckbuilding with some spatial puzzles. The number of plays of this game and of Champions of Midgard correspond to my second son's rising up into the next tier of complexity; these are joined by Runebound, which he only recently learned but we've played a lot of these past weeks. He still struggles with some of the more complex interactions—and even more so with sitting still for longer games—but overall I've been surprised with how well he manages the games. Like any 8-year-old, he will sometimes get tunnel vision on a plan and not roll with the punches, but I think this is something that games will help him learn to do better.

Rhino Hero Super Battle, Carcassonne, and Camel Up are joined by Go Nuts for Donuts as games that anyone in the family can play, and so most of my plays of these are with the younger two boys.

The notable thing about Rising Sun getting to the table eleven times is that many of those were with friends rather than family. Almost all my gaming is with my family, and I love playing games with them. Having them here probably makes me a bit lazy about reaching out to my friends to have them over. However, I had a small group of friends who really caught on to Rising Sun and came over for several game nights this summer. I feel really good about that and need to keep that up.

A few notable games of 2018 did not make the cut into the top 11. I bought Charterstone as a family Christmas gift, and my two older sons, my wife, and I are four games into the campaign. We have been enjoying that, and I look forward to seeing where the game goes next. My third son received Ticket to Ride: First Journey last year for Christmas and we played that quite a bit. A few weeks ago, he graduated to Ticket to Ride, which we played five times together, before I taught him Ticket to Ride: Europe, which I think is the far superior game. We have now played Europe five times as well, and he asks pretty much every day to play it again. The kid loves trains.

Looking at the list of games I only played once this past year, it makes me wonder if I should be even more aggressive about getting rid of games. I remember my wife sharing a story with me about a collector who got rid of all but 10 games, and that he was happier with the ten he really loved than the many he rarely played. My brother also recently tried a thought experiment of what games library he would build with just $250. To me, the answer is clear: two copies of Mage Knight: The Board Game, Ultimate Edition.

Each year, I've written a little about tabletop RPG in this post, and once again, I was able to do a very little bit of RPG gaming, but not much. I ran two sessions of Index Card RPG during the year. One was a game with my three older boys, themed around the "Magic Sword" fantasy realm that my second son spent many years imagining. I actually haven't heard him say much about it, even in the months leading up to our summertime session, but for a long time all of his imaginary play revolved around a world of knights, dark magic, and dragons. My third son particularly enjoyed the session I think, and he regularly asks to play Magic Sword, but I haven't made the time to spin up new adventures for them. The other session was a challenging design for a big family vacation. With some help from the ICRPG community, I designed an adventure that would scale to a variable number of players. As it turned out, only my two older boys and their older cousins were interested, so I had a manageable table of four. I think the session was a great success in many ways, and it was a good way to spend some time with my niece and nephew.

As has been my custom, let me wrap up by looking at the games that comprise my overall h-index of 20:

  • Gloomhaven (55)
  • Animal Upon Animal (54)
  • Crokinole (47)
  • Carcassonne (36)
  • Camel Up (35)
  • Rhino Hero: Super Battle (35)
  • Labyrinth (31)
  • Clank! (29)
  • Terror in Meeple City (29)
  • Runebound (Third Edition) (28)
  • Dumpster Diver (23)
  • Race for the Galaxy (23)
  • Red7 (23)
  • Reiner Knizia's Amazing Flea Circus (23)
  • Stuffed Fables (23)
  • Thunderstone Quest (22)
  • 4 First Games (21)
  • Flash Duel (21)
  • Go Nuts for Donuts (20)
  • Samurai Spirit (20)
This was the year that Gloomhaven overtook Animal Upon Animal. I feel like has to be a milestone in the growth of my family, that a heavy fantasy strategy game overtakes a light HABA game. 

Thanks for reading. I hope 2018 was a good year of gaming for you as well. Here's to a happy and playful 2019!

My Notes on "Make It Stick: The Science of Successful Learning"

Several weeks ago, I finished reading Make It Stick by Brown, Roediger, and McDaniel. It was recommended to me by a good friend with a heart for improving education. The book aims to explain what we know about learning from cognitive science and how this can impact the practices of teaching and learning. I found the book to be inspirational, and I mentioned the book in several recent essays and presentations. I happened to meet local cognitive science and student motivation expert Serena Shim this semester, and she affirmed the findings and value of the text as well.

One of the most important findings that came up throughout the book is that spaced practice is better than massed practice. I think we all recognize that it is true: of course studying throughout the semester is more effective than cramming. However, the science is more nuanced. Massed practice is actually better for short-term recall than spaced practice, but spaced practice is better for long-term recall. This has a fascinating corollary: if our courses contain high-stakes tests, then it is a good tactical decision for students to cram.

This implies to me then that we instructors have to make a real choice between I want students to pass this test and I want students to remember this a year from now. I have a rule of thumb that I have only recently had to articulate, which is that I only want to teach content that I think students should know in five years. My general pedagogic approach favors spaced practice, but perhaps I can do more to support this. However, recent conversations made me realize that this perspective is not universal. I was involved in a somewhat heated discussion about a master syllabus revision with a colleague. The particular syllabus had, in my opinion, too many low-level learning outcomes. I argued that students don't learn these items, and he argued that they do. As evidence, I cited that they could not repeat their achievements a year after taking the class, and as evidence, he cited that they passed the final exam. Here are the loggerheads of higher education: we both believed the other to not just be wrong, but to be holding the wrong value system.

I was reminded by the text of the value of testing as retrieval practice. I had read this before but tried to dismiss it; however, the presentation by Brown et al. makes it hard to ignore. Learning improves through retrieval practice, and testing is perhaps the simplest way to practice retrieval. I mostly gave up on using tests many years ago, favoring instead continuous authentic work. However, I also see my students not remembering to apply fundamental lessons early in the semester into their work later in the semester. I need to review my use of quizzes and tests, as well as how I prepare students to do their own self-testing.

Another theme of the book that knocked my proverbial socks off was that immediate feedback is not always better than delayed feedback. I think that in the educational games community, it is taken for granted that feedback is simply good, and that quicker feedback is better feedback. As the argument goes, if feedback is good for learning, and games are feedback machines, then games can be good for learning. This is not wrong, but it is also superficial. Not all feedback is created equal. The authors cite studies that show that delayed feedback can lead to better learning. As I understand it, the actual reason for this is not understood, but the prevailing hypothesis is that immediate feedback makes the feedback indistinguishable from the task itself; this leads to a result where when the feedback is not present, knowledge of the task breaks down. This sounds an awful lot like "I can do it in the game, but I cannot do it outside the game." I wonder how many empirical educational game research projects have investigated feedback delay as a dependent variable, and if not, how one would construct such a study. After all, a player expects that if they press 'A', Mario should jump right away.

Reading the section on delayed vs. immediate feedback made me think of two other salient examples where immediate feedback may be causing problems. The endemic one is automatic spellcheck and grammar check: we all know that students do not learn to spell or write by letting their word processor do the work, it just builds a reliance on the word processor. The other, related example is IDE for novice programmers. As with automatic spellcheck, the IDE will add red squiggles to invalid code, and students can right-click on it and change it to whatever the IDE wants—often without regard for whether it is what they should want.

Chapter 8 of the book provides a series of helpful summaries that are organized for different reader demographics. It's a valuable chapter, and so I will spend a bit of time on it here describing what caught my attention and where I think it should take me. In the section for teachers, they recommend explaining to students how learning works. The following quotation is a good overview:
  • Some kinds of difficulties during learning help to make the learning stronger and better remembered
  • When learning is easy, it is often superficial and soon forgotten
  • Not all of our intellectual abilities are hardwired. In fact, when learning is effortful, it changes the brain, making new connections and increasing intellectual ability
  • You learn better when you wrestle with new problems before being shown the solution, rather than the other way around
  • To achieve excellence in any sphere, you must strive to surpass your current level of ability
  • Striving, by its nature, often results in setbacks, and setbacks are often what provide the essential information needed to adjust strategies to achieve mastery
Another tip for teachers is to teach students how to study. This has been on my mind quite a bit, along with the question, "Where does the buck stop?" I teach primarily junior and senior undergraduates, and I estimate that 5% of them have any real study tools. Indeed, I think a good description of the Ball State demographic is, "Students who are smart enough to have gotten this far without having developed study skills." Including direct instruction on study habits is an investment in their future learning, but I doubt I would be able to reap it in my own courses, so it's taking away from time on topic. More frustratingly, I have seen for years that I can teach good processes for learning and software development in a course like CS222, only to see that a year later, the students have never touched any of those techniques because other faculty do not expect them to. For example, I can teach the value of pair programming or test-driven development, present the students with research evidence that these increase productivity, and require them to deploy these techniques; but a year later, when I ask them to do these in a follow-up course, they say that they have not used these since CS222. Why should I be more optimistic about study skills, when inertia is powerful and habits are so hard to override?

The section of tips for teachers returns to the theme of "desirable difficulties" that came up throughout the book. Here are some specific desirable difficulties that they recommend:
  • Frequent quizzing. Students find it more acceptable when it is predictable and the individual stakes are low.
  • Study tools to incorporate retrieval practice: exercises with new kinds of problems before solutions are taught, practice tests, writing exercises reflecting on past material and related to the aspects of their lives; exercises generating short statements that summarize key ideas of recent material from text or lecture.
  • Quizzing and practice count toward course grade.
  • Quizzing and exercises reach back to concepts and learning covered earlier in the term.
Again, this is a valuable summary. Each of these items is covered in the text with explanation and citation. It's clear what actions can come from this list as well, and it makes me look at opportunities in my upcoming HCI class in a new way. I also recognize in it the value of several things I already do in the class, such as having students connect readings to their experience and writing reflections of development experiences. Given that I tend to divide semesters into a content-oriented first half and project-oriented back half, I need to be more conscientious about designing assignments and quizzes that reach back to the early part of the first half; this should help students deploy these ideas more readily in the second half.

The final bit of advice in Chapter 8 is to be transparent with students about incorporating desirable difficulties into the class. I have always been a fan of white-box pedagogy, although it's not every semester that I see students take interest in why I am teaching the course the way that I am. Student teaching evaluations often reveal quite mistaken models about my intentions as well. Sometimes I get these excellent reflection sessions as I described in Fall's HCI class, but the irony here is that they generally come after students have completed their evaluations.

I highly recommend Make It Stick. It is written clearly and precisely and organized in a way that emphasized the important points. Crucially, it avoids educational fads in favor of empirical research. Chapter 8, as I have said, provides a great synopsis that turns the ideas of the book into potential action items for practice. 

Thursday, December 27, 2018

Planning CS445 Human-Computer Interaction for Spring 2019

Around the Christmas celebrations, I have spent many hours the past two weeks planning my Human-Computer Interaction course for Spring 2019. I wrote my reflection about the Fall semester back on December 8, and I just put the finishing touches on the Spring section's course plan before lunch today. Now, I would like to write up a few notes about some of the interesting changes I have in place.

First, though, a funny caveat. Since I originally designed the HCI class for CS majors, it has been CS345. Due to administrative busybodies, the course will now be numbered CS445 instead. Which label should I use for my blog posts? I think I'll switch over to "cs445" now, and I'll have to remember to use both codes when I'm searching for my old notes.

Canvas Grading Caveat
Prompted in part by my frustrating experience at the end of the Fall 2018 Game Design class, I have a more explicit statement on the course plan telling students to ignore Canvas' computed grade reports. I would always say this in class, but I did not have it explicitly in the course plan before. Also, I found out that I could mark assignments as not contributing to the final grade, in which case students will be able to see their assignment grade but not a false report of their "current" final grade, so I need to remember to mark all the assignments that way. Also also, please take a moment to consider the epistemological tragedy that is the concept, "current final grade."

Writing
I was surprised in the Fall when one of my more talented HCI teams brought up their project report, highlighted a place where I pointed out grammatical and spelling errors, and asked if they had lost points because of it. There are two things wrong with this question. The first is that it assumes the students had some points to lose in the first place, which is simply not true. I don't take away points from anyone; instead, I award points for demonstrating competence. You cannot take away something that someone doesn't have. The second, more pragmatic problem is that the students also had significant conceptual and descriptive problems in their report, but they seemed more concerned about the spelling and grammatical errors.

Last semester, I included a link to the public domain 1920 version of Strunk's Elements of Style, along with advice to read it. This time, I've made my expectations more explicit. On the evaluation page of the course plan, I have explained briefly the importance of writing and the fact that Elements of Style will provides my expected standards. I also explain there that I expect to give feedback on both conceptual problems and spelling or grammar problems, along with a primer about how to interpret that feedback. I thought about making an assignment around Elements of Style, but I decided against it, partially because I did not want to shift my early-semester plans ahead by a day. My professional opinion is that the book should be remedial to anyone who has a high school diploma, but I am also a realist about the variable quality of writing instruction these students may have received.

Software Architecture
It was a little disappointing to see so few teams really engaging with principles of quality software construction last semester. I have written about this before, and the students are aware that the culture of other classes is one that values only software output rather than software quality. I have carved out some design-related topics from the HCI class to make more time to work through examples of refactoring toward better architectures. I'm still working on the exact nature of these assignments, but I have a few notes to draw from. The schedule I have online right now actually goes right up to the point where I want to switch gears from design theory to software architecture practice.

Specifications Grading
After a positive experience in last semester's game programming class, I have converted my HCI class project grading scheme to specifications grading. I laid out my expectations for each level of poor (D), average (C), good (B), and excellent (A) grades. This was an interesting exercise for me, especially around the source code quality issues. Last semester, students could earn credit for following various rules of Clean Code, and a mixed grade simply meant that they got some and not others. Now, I have put all of these rules at the B level, to reflect the fact that "good" software is expected to follow such standards. For the A level, I've included demonstrated mastery of the software architectural issues mentioned above.

I had some fun with the Polymer-powered course site as well. My new custom element for presenting specification grading criteria uses lit-html to concisely express how they are presented. It took a bit for me to wrap my head around lit-html, but I think I have a good sense of it now. The other fun new feature I added was the ability to download the specifications as Markdown. The specifications are internally represented in a Javascript object, and that object is transformed into the Web view. Of course, with this model-view separation, it's reasonable to provide other views as well, such as Markdown. I used this StackOverflow answer to write some functions that convert the JSON object to downloadable Markdown. I hope that this makes it easy for students to write their self-evaluations in Markdown. I did not use checkboxes on the Web view, as I did for game programming last semester, because they don't copy and paste well. I hope that having the Markdown version available removes the need for students to manually copy over each criterion into their self-evaluation.

More Time on the Final Project
In addition to switching the evaluation scheme, I am giving more time to the final project. I decided to keep the short project as a warm-up, since it also provides a safe-fail environment as students pull together ideas such as personas, journey maps, user stories, and software quality into a coherent process. Some of my dates for the final project aren't quite sorted out yet, as I'm debating whether to take a purely agile approach or a milestone-driven one. The advantage of the latter is that I can specify exactly which design artifacts I want to see at each stage, and the level of fidelity I expect. However, I expect that I will follow the more iterative and incremental approach, but I've put off the final decision until the semester gets underway and I can get to know these students a bit.


We are continuing our relationship with the David Owsley Museum of Art, who have been a great partner. I look forward to working with them and seeing what my students can develop during the semester.