Something like a Summer Devlog #5: Fluttering back to Thunderstone Quest

TL;DR: New Thunderstone Quest Randomizer with combo support. Check it out at https://doctor-g.github.io/ThunderstoneQuestRandomizer.

Background and Inspiration

By all accounts, the plan for this week was to work on a plan for transitioning the third of my three Fall classes to its online mode. I had done some of the prep work required and started conversations with colleagues about it. Then I received my New Horizons expansion for Thunderstone Quest. Normally, this wouldn't derail a whole week, but I was excited to add the new cards to my Polymer-powered randomizer app, TQR, which is online here and which I wrote about here. I was pleasantly surprised how easy it was to add the new cards: just dropping them into the JSON configuration file let the rest of the engine find them and incorporate them into the user experience. I rarely maintain my old code, but it was a good feeling to have something I wrote in May 2019 hold up.

...well, except for one detail. The expansion added a new kind of Market card: Allies. The expansion rulebook suggests adding the ally randomizer cards to all the rest of the market randomizers, and then adding the Ally cards to the "Any" spaces in the market when they come up. This struck me as curious when I read it originally, since I never thought to mix all my market randomizers together: I keep them separated by type (Item, Weapon, Spell). Indeed, this kind of separation is encouraged by the physical dividers provided in the base set. More relevant to this story, the TSQ implementation assumes that you figure out the distribution of Item, Weapon, and Spell first, then fill the market. That is, since there were always three of two types and two of the other, it determined that first, and then drew from the corresponding decks. There was no way for me to add Allies without gutting this part of the randomization, which was also tightly coupled to the whole user experience.

I began to imagine what it would be like to reimplement the randomizer in Flutter. I taught myself enough Flutter this summer to be able to include it in my HCI course plans, and I developed a sample solution to a two-week project for that course. It was, of course, a simple project, and I had in the back of my head that I should do something more significant. Why not a new Thunderstone Quest randomizer? This would give me the opportunity to get a better grip on Flutter, plus I could add one of the key features that didn't make it into the original implementation: combos. I have seen various Thunderstone communities reminisce about a particular randomizer app from Thunderstone Quest's predecessor, Thunderstone Advance, for its support of card combos. My mind idly went through a couple of potential implementations for combos, and I convinced myself (without building any technical demos or writing the actual requirements) that I could get a version of this working. That was enough for me: Monday morning, I was off to the races and working on a new Flutter-powered randomizer.

A point of clarification: My old randomizer is officially named "Thunderstone Quest Randomizer" and abbreviated TSQ. After some failed attempts to come up with a catchier name for my new implementation, I also named it "Thunderstone Quest Randomizer." When it's not clear which one I might mean from context, I will refer to the old one by the abbreviation and the new one by the full name. At least, I will do this until someone comes up with a better name for it, or until I get really excited about "Thunderpants" as a memorable name. Lion-O would dig it.

Data Wrangling

One of the first things I did was transform my TSQ JSON data file into YAML. It wasn't too bad to add the new data to JSON, but there is a lot of cruft. I happened to be recently back into using YAML, which I traditionally like better than JSON, but the prevalence of JSON has me using it more often. More recently, I've been tinkering with GitHub workflows, which are specified with YAML, and it reminded me how much I prefer it for structured data. I used an online translator that was fairly painless and found a convenient API for parsing YAML in Dart (the programming language of Flutter). Adding new libraries to a Flutter project is as simple as it should be in a modern programming environment: drop a line into the awkwardly named pubspec file, which is itself YAML, and you're off to the races.

I wrote most of the parser using TDD, and I found the testing environment to be similar to ones I've used for Javascript: easy grouping and idiomatic matchers that benefit from higher-order functions. I decided from the get-go that I would not make a fundamental architectural mistake that plagued TSQ, which was the very Webby assumption that the JSON data should be used directly by the view. Instead, I added a semantic model layer in between, so that in the new randomizer, the YAML file is parsed into a list of Quest objects, each of which has Heroes, Monsters, Items, and so on. I didn't see a way to automate the population of the domain model as I would do with a library like GSON, but it wasn't much code to grab the data I needed from the YAML parser. Subscript notation for random map access also meant this was very short to write, compared to Java, for example.

This was also a place where I learned to be a bit more idiomatic with Dart. I originally had repeated code of this form:

if (node['Hero'] != null) {
  for (var entry in node['Hero']) {
    // Parse hero
  }
}

This sort of defensive programming was required because attempting to access a key that doesn't exist, in the YAML library implementation, produces an exception, not null as I might expect. After reading some documentation on best practices for Dart, I came across the curious "??" operator, which I have not seen in any language before. It allowed me to change the code above into something like this:

for (var entry in node['Hero'] ?? List()) {
  // Parse hero
}

This means that if node['Hero'] is null, then use an empty list as the value instead, which of course terminates the iteration right away. It's much more terse and expressive: less imperative cruft. I will mention, though, that throughout the implementation, I was nagged by a feeling like there were better, idiomatic ways to do things, and that if I had a code review by a Flutter regular, they could point out where I might grow next. I am sure I can help my students in the Fall go from Novice to Advanced Beginners, but the gap between Advanced Beginner and Competent is foggy.

Let me dig into that just a little bit more to show what I mean. Here's an excerpt from the 1.0.0 release:

for (var entry in node['Heroes'] ?? empty) {
  Hero hero = new Hero();
  _parseCard(entry, hero);
  _quest.heroes.add(hero);
}

for (var entry in node['Items'] ?? empty) {
  Item item = new Item();
  _parseCard(entry, item);
  _quest.items.add(item);
}

There are eight blocks basically like this for the eight types of cards. The guts of the parsing is the same for each card, so it was easy to extract a function that. However, the actual runtime type of each card is different, and they get added to different lists within the Quest. It feels like a terrible repetition to have almost the same code eight times, but it's not obvious to me how I could extract this to have fewer lines of code, especially knowing that Flutter does not allow for runtime reflection. I could put the parsing logic into the domain model classes, but that pollutes the model with data concerns, and making separate per-class parsers for each would add more code than it removes. If this looks like a fun refactoring challenge to you, I welcome you to open a pull request!

Combos

The combo system that I implemented operates on a matching of two kinds of data. Each of the keywords on the cards (like "Humanoid" or "Edged") are encoded in the YAML file and stored in the domain model. Additionally, each card that references these data on other cards has encoded a combo list. For example, Stormhand has a bonus for Edged weapons, so his combo list includes the term "Edged". When choosing a card, then, the combo can arise in two ways: either the current card has a keyword that is in the combo list of a card on the tableau, or the current tableau has a keyword that is in the combo list of the current card. I introduced a combo bias to the application, which is a probability that a card would be filtered out for not being a combo.

It turns out that this is not entirely sufficient, since not all powers trigger off of keywords. Some trigger off of other properties, such as the presence of diseased wounds or certain words in the card title. To handle this, I added a "meta" category to the card database. These are folded into the keywords when searching for combos. For example, Blizzard gets more powerful the more frozen wounds you have, so this card gets "Frozen" in its combo list, while cards that give frozen wounds, like the Tundra Wolf Pack, have "Frozen" in their meta.

In human language, it works a bit like this. Choose a random card to potentially add to the tableau. Roll the dice to determine if we're looking for a combo. If so, check if there is a combo, and if so, keep the card; if we're not looking for a combo, just keep the random card. Then, see if the card can actually fit in the tableau (e.g. the Marketplace is not full).

This is why I called it a combo bias and not a combo chance, because finding a combo does not mean that the card will be used: it's more of a probability of applying a high-pass filter. The app currently allows the user to set the combo bias between 0% and 95%. In casual testing, I've never had a case fail to match or take an observable amount of time except in cases where I specifically chose quests that have no combos, such as using just the Bandits of Black Rock quest, which does not have enough cards to fill a tableau.

There are parts of the implementation that are woefully unoptimized, such as the on-demand querying of which keywords are on the tableau. I expected this to run much more poorly than it did, but the truth is that because the N is so low, and computers are so fast, it made no observable different to performance whether I cached the data or created and merged new Set objects each time.

The sequence of card selection is important for combos, of course, and I tinkered with this a bit throughout development. As of this writing, the sequence is Heroes, Guardian, Dungeon, Marketplace, and Monsters. We did not encode any combo data for the Dungeon tiles, since they generally key off of things that are always in play, like the four classes, light, and gear tokens. This means that the hero selection is arbitrary, subject to the selected strategy, but anything else except Dungeon Rooms down the chain can be part of a combo. I think this is probably a good configuration, but I'd be happy to hear other perspectives on this.

State Management

Naturally, I had some frustration with Flutter as I tried to wrap my head around it. The little sample project I mentioned before existed on one screen with a single stateful widget, but for this project, I knew I had to handle state more carefully. The documentation gives a few pointers and then says, essentially, do whatever you want. It's nice to have that freedom if you have time and mentoring. I decided upon the approach recommended for simple applications, which was to use a ChangeNotifier to propagate changes to the user settings through the rest of the application. Near the end of development yesterday, I got to wondering if I had correctly discerned the difference between using the Consumer and Producer objects, and I may yet go revisit that. In the meantime, though, the basic architecture of the application is that the randomizer screen is its own stateful widget which is the generated tableau, and the options in the Settings page are shared via a ChangeNotifier.

The SettingsModel and its corresponding screen are another area where I see a lot of repeated code, but it's not obvious to me how to use the tools of the language to factor out the duplication. I can handle cases like showing one checkbox per quest to have it selected or not, but it's less clear to me how to handle the suite of checkboxes around the application appearance.

Implementing different hero selection strategies was done with the Strategy design pattern, of course. I was happy to be able to implement this pattern in a very direct way, and it worked as expected. It did result in a strange separation of concerns, though, where the thing called "Randomizer" delegates to a strategy that is, for convenience, defined alongside the SettingsModel. This is another area of code that is a bit gross, could benefit from refactoring, but also works pretty well if you just leave it alone.

Flutter is famous for its support of contemporary user experience design, including ample animations. I knew I wanted to add something along these lines, although going into the app I didn't have a concrete notion of what it would be. As I got into the implementation, I decided to keep it simple: use fade transitions to show when a new tableau is generated. Note that the animation between the main randomizer page and the settings page was essentially "free" since it is built into the default app routing behavior.

Animation

It took me many, many efforts to get the animation working properly. The documentation made it seem like FadeTransition should be easy to put into play, but I tried that, several other things, and then came back to that. After hours of hammering at it, I finally had a sense of how the animation worked, and this revealed to me that I had deeper problems with how I was conceiving of the animation. When I stopped to consider what was scratching at the back of my head, I realized that I had to separate and synchronize the notions of fade out, generate tableau, and fade in. It seemed to me that I should be able to do this with a chain of futures, but the animation framework seems to only provide listeners, not futures. The result is a bit of confused code in the main randomize screen: the logic for the sequence of events is not laid out sequentially in code, but spread across a few different functions to handle different cases. I don't like this, and it's not clear to me if it's a property of the framework or a limitation to my understanding.

Another piece that caused me minor trouble, but is worth mentioning, is the asymmetry in how animation events are reported. When you play an animation forward (fade in), it ends with a "completed" event. That makes sense, so I assumed then I could listen for the same thing when playing an animation backward (fade out). It doesn't, however: when an animation is finished playing backward, it fires a "dismissed" event. Dismissed? I had seen this in the documentation assumed it meant, you know, that the animation was dismissed. I would never say that a reversed animation was dismissed, and this nomenclature surely doesn't show up in any of the game engine's I have used, where reversing animations is commonplace. Heck, I can't even find in the Flutter documentation now the definition that this is what they mean by "dismissed." It's a really odd choice to me.

In the end, though, I got the fade animation working. Originally, I had some idle thoughts about card selections flying in from the side, but really, I think the fade transition fills the bill.

Tool Notes

Let me say for the record that flutter doctor is amazing. Indeed, the whole flutter command-line tool is a master class in how to conveniently manage a development environment. It was frustrating, then, that there were a few cases where Visual Studio Code did not behave as I liked. I regularly had to kill and restart debugging sessions. This would not be such a frustration except that I had to do all my development in Windows because of a problem with hot-reload on Linux. I was using Windows because Linux wasn't providing me with hot-reload, but the hot-reload frequently failed on Windows, so what then? Also, I had bad behavior from the debugging sessions when changing my assets, but it wasn't clear if that's an expected behavior or not.

The biggest frustration about Visual Studio Code, though, was that for the life of me I could not figure out how to tell it to run all my unit tests. I could easily choose a suite of tests to run but not the whole directory of tests. Dropping into the shell, flutter test did the trick, and I ended up regularly just doing that and, when failures were reported, going back into the IDE to run the suite with the failure so I could get the more readable error report.

It took me more than halfway through development to realize the power of the Ctrl-. shortcut, but once I found it, it had a dramatic improvement on my development speed. This shortcut brought up a menu of options that I hadn't even thought to hope for, the real beauties being "Wrap with Widget" and "Remove Widget". Flutter code can end up with lots of nested widgets, which yes, really ought to be extracted into more manageable chunks. Be that as it may, manually wrapping or removing widgets is a painful process of braces-matching. These shortcuts were brilliant time-savers.

Also, the GitHub workflow I wrote about the other day to automatically publish on GitHub pages worked like a charm.

Final thoughts

I have dubbed this release 1.0.0 and posted about it on BoardGameGeek, where it's already generated a few thumbs-up and positive comments. I will be adding a link to this blog post there too, for anyone who wants to read the long form commentary. I would be remiss not to include my gratitude to my eldest son, who helped with encoding the combo data and keywords into YAML. He did a great job, learning the notation and the ideas quickly.

Awkwardly, my own set of Thunderstone Quest is not currently in a state to use a randomized tableau! Before receiving the expansion, we have been playing the Epic variant, so I have several cards pulled from their stacks for this purpose. Maybe after we play the authored quests that came in the expansion, I can excite my kids to do a bit of sorting so that we can play some games with different combo biases.

Right now, the fact that a given card was part of a combo is not exposed in the user interface. You can actually see them logged if you open up the console: for example, on Chrome, press F12 to open up the developer tools, and you can see combo card selection reported there. I've thought about making this an option in the UI, but this goes into the "that's enough for this week" category. Maybe I should open an Issue about it on GitHub so I don't forget.

I have not taken a careful look at internationalization nor accessibility of the new randomizer, but these are significant topics that I would like to investigate. I want to include at least some coverage of both in the Fall HCI class, and one of the reasons for making a bigger Flutter app was to motivate the desire for such things. I don't think I will do it right away, though. I worked on Thunderstone Quest Randomizer more-than-fulltime since Monday, and today I need to step back from it and do some family painting.

Thanks for reading. I hope you enjoyed the commentary and that you find some utility in the Thunderstone Quest Randomizer. It's free software, so please feel free to use it, share it, study it, and modify it to your heart's content.