On Measurement in Computer Science

In a chance encounter last semester, I befriended an undergraduate honors student who was not involved in any of my courses. She has become enamored of the notion of measurement—what it is and what it means. She asked me to contribute an essay on measurement in Computer Science for a collection that will become her honors thesis. It was an honor to be asked, and although I had plenty of inspiration, I found the actual writing to be difficult. After a few discarded drafts, I decided to focus on the paradox of the measurable and the unmeasurable within Computer Science, while trying to stay apart from political debate (we need more Computer Scientists!) or research loggerheads (agile vs waterfall). Without further ado, this is the final draft I sent to her yesterday, although I wonder if I may tweak it a bit before she publishes it.

The discrete and formal measurements of Computer Science reflect its foundations in mathematics and electrical engineering. Computer programs are composed of unambiguously parsed statements and expressions, which are compiled into atomic hardware directives. Information is transmitted over networks by measuring electronic pulses and radio waves, interpreting these as bits and bytes. Computational theory tells us that all computable problems belong to a single space-time hierarchy and that some problems cannot be computed at all. This leads us to the biggest question of Computer Science—the question of whether P=NP—which is fascinating not only because we do not know the answer, but because we also do not know if it is answerable.

These discretely measurable aspects of the discipline complement a more humanistic side. Creativity and ingenuity are required in determining what line of code to write, what proof technique to use, or what software architecture to follow. First attempts almost always fail, and so we learn to approach problems iteratively, using formal and informal heuristics to determine whether we are moving closer to a desirable solution. That is, we recognize that doing Computer Science is a learning process, that building a system is the process of learning how to build the system. Unlike in engineering and architecture, where models are built for others to manufacture, the computer scientist deals directly with the elements that compose the artifact. This creates a rapid feedback loop for the reflective practitioner, giving rise to the model of expert computer scientist as master craftsman.

Building any nontrivial software system requires a multidisciplinary collaboration, where the tools of the Computer Scientist are combined with approaches from the social sciences, humanities, and the arts. Research consistently shows that teams with strong communication practices produce better results; how to actually conduct these measurements is a perennial research question. Martin Fowler—a recognized industry leader software development—has gone so far as to claim that productivity cannot be measured at all, and that any attempted measurements will only produce misinformation.

My own scholarship involves building original educational video games with multidisciplinary teams of undergraduates. When interviewed about what most contributed to their successes, teams inevitably split their answers between the measurable and the non-measurable. They find that formal processes and communication patterns, quantitative feedback, and technical structure give them the confidence to build and revise complex software systems. At the same time, they recognize that empathy has the most dramatic effect on both team productivity and the quality of the product. These two sides, the scientific and the humanistic, are not opposed to each other: they form a synergy that can propel a team to success or, in their absence, drag a team to a near standstill.

This harmonious balance of the measurable and the unmeasurable characterizes Computer Science. Alistair Cockburn captures this paradox in his cooperative game principle in which he defines software development as a cooperative game of invention and communication. Both kinds of moves are necessary for the team to create a solution of value to stakeholders and, thereby, to win the game. A winning team must make measurable and unmeasurable moves, creating technical and artistic artifacts while continuing to build empathy, all the while reflecting on how they are learning through this process.