My video game preproduction class formed teams and started work on their final projects a few weeks ago. Each group started working on digital prototypes before they had really settled on the core gameplay. I encouraged them to move toward paper prototyping so that they could make changes faster, even radical changes. I am pleased to report that they have embraced nondigital tools as part of their process, although I'm still concerned that they are not iterating fast enough.
This was on my mind when I watched Steve Smith's latest video from the Modern Software Engineering YouTube channel, which reminded me how the DORA metrics can positively influence software development practice. It got me thinking about the metric change lead time, which measures how long it takes from a change to go from committed to deployed into production. I realized that this is essentially the same idea as I have been advocating for their prototypes, whether they are analog or digital: the less time it takes for an idea to go from implemented to tested, the better. It doesn't matter whether we are committing in the sense of version control or in the sense of having made a coherent change to a paper prototype. I suspect that the DORA metrics would still hold, that a team that reduces that time is going to be more productive.
In today's meeting, we talked about what this means for us as game developers. I borrowed part of Smith's analysis, discriminating between activity metrics, output metrics, and outcome metrics. The students got the idea that the outcome metric is most important, and that this could be something like profit or it could be something like delight. That is, if we are making a game that is designed to bring joy to people, then the amount of joy we bring the player is the intended outcome, so we should measure that. In fact, we often call that playtesting.
This led me to hypothesize that we can use change lead time metrics as a tool to determine when to switch between forms of prototyping. I have that students sometimes have difficulty understanding physical prototyping and other forms of lightweight prototyping when they are developing video games. The DORA metric points toward a solution: one should change modes of prototyping when it reduces change lead time. That is, given an intended change to the game system, one can ask which mode of prototyping allows us to go most quickly from implementation to playtesting. While that is paper, stay with paper. At some point, though, there will be more systems or crafted aesthetics than the low-fidelity form is accurately conveying, and so we use higher-fidelity prototyping instead.
There are at least two potentially confounding factors within this hypothesis. The first is that it assumes that we can accurately assess whether a particular system can be accurately playtested in a given prototyping medium. Drawing a physical card feels different than tapping a depiction of a deck, for example. The second is that it may require an amortized analysis when getting started. Engineering a system that allows for rapidly changing design components requires some up-front work. The point remains, though, that one cannot build software that is infinitely malleable, and the low-fidelity prototyping ought to inform which parts of an implementation are expected to change and which are not. Like the first challenge, this deals with the inherent risks of software development: the only way to know exactly how to build it is to have already build it.
