Learning that story maps are harder than they look

Some time last semester, my chair asked if I would teach a section of CS691, a new graduate course for my department's Masters of Science in Software Engineering program. The course is titled Software Requirements and Design, so right off the bat I was a little put off by the sheer waterfallness of it. I have been mostly uninvolved in the MSSE program, which struggles to get started for various reasons. Be that as it may, requirements and design are definitely concepts in my bailiwick, so I agreed to put together the best course I could. He told me to expect relatively low enrollment, given the low number of students in the program, but we had to offer it anyway because these students need it to graduate.

As I was procrastinating designing the course, I received a call from my friend Kate who is working with a local non-profit, ecoREHAB. She's leading a team of students who are revising the organization's marketing materials to help ecoREHAB better explain its mission and values. Working with their board, she had some ideas for potential software products to help. This aligned very well with the learning objectives of CS691, so we agreed to a collaboration. I didn't yet know how many students I would have or what their background was, but ecoREHAB was very understanding of this: though I couldn't promise a product, I was sure we could build a prototype, and that would be enough that we could see if it was worth pursuing to production or not.

This brings us to the point of the blog post: yes, once again, it's story maps! I decided to use this technique in CS691 to capture and track high-level requirements, and so I assigned Patton's book as required reading. With my plans in place, I awaited the start of the semester. Turns out, I only have three students in the course, two of whom need it to graduate and one who is taking it as an elective. During the first week of the semester, we read the first few chapters of Patton and completed his "morning routine" story map, where a team builds a story map of what they do in the mornings. The argument in the book is that people already know how to make story maps, they just need to learn how to do it in a disciplined and pragmatic way. This exercise went well, although the students pushed pretty heavily toward making a taxonomy rather than a map. That is, they wanted to classify and make categories rather than focus on discrete stories. I pointed this out, and we moved on. We made a second story map, based on a straightforward problem a friend of mine had last semester: he wanted to know which courses the faculty in his department wanted to teach or did not want to teach, and have that information readily available when scheduling. Here, too, I don't think what we came out qualified as a "story map" as defined in Patton's book, or at best, it was an awkward one. I pushed the students to think about this, but they believed it was satisfactory. We put these "starter" maps aside as we spent the rest of the month in a kind of race through background books, papers, and ideas. According to the schedule I had laid out, by early February, we would be returning to our community partnership; I figured we would come back to these issues then.

During our month working together, I came to truly respect my students. Part of it may be the small class size—there's nowhere to hide, and everyone gets to speak their piece. Regardless, these three are excellent students: they keep up with all the readings, they ask great questions, and they voice and justify their opinions. One has industry experience from overseas, one has local small-team experience, and one comes directly from undergrad and a masters in another discipline. I don't think I could have picked a better mix of students. I say all that in part to qualify my comments below: I have no doubt that these students are doing their very best.

I decided I wanted to give the reins to the students when we did our planning meeting with members of the ecoREHAB board. To bridge our background work to the project, I did another story mapping session. I'm working from memory now, and I don't quite remember the context I picked, but again, I felt like there were pieces that the students just weren't getting, like sliceability and the difference between an activity and a category.

This took us to our big meeting with the community partner. A student volunteer led the session, and he did an admirable job for a first-timer. It's easy to forget that this stuff is hard, just simple things like keeping people on track during a meeting, welcoming different perspectives while being mindful of the time. Also, we have no courses that help students with this kind of process; if I were to teach this course again, I would put some of this into the course description, since where else would students get the chance except when studying requirements gathering?

One of the team could not make this meeting, so we recorded it. This proved to be quite useful, since the team agreed that our one-hour meeting with ecoREHAB could not have possibly produced as rich a story map as we wanted. I challenged each student to make their own story map, reviewing the recorded video as needed, and we came together in our next meeting to share what we had found. I kept my Socratic composure as I pressed the students to evaluate their and their peers' maps. They had differences of course, yet they were still pretty confident... but I was not. In an inspired moment of which I am rather proud, I decided not to tell them what I thought was wrong; instead, I assigned them an essay, to return to Patton's book and then to think and write critically about what they know about story maps and how they know it. This was on a Friday, so I tried to put my own concerns on the back burner until we could get back together.

On Monday, the first student I met in the hallway shook his head and said something like, "That damned essay." He proceeded to tell me how much he disliked it because it forced him to realize how much he didn't know. I tried to turn that ship around and tell him that this means the essay worked! As I was doing this, the second student came in, muttering with half a smile, "That essay..." As you might guess, all three students came in a bit dejected, but I think much smarter than when they had left on Friday. All three wrote brilliant essays about what they thought they knew, and about how they realized that their assumptions were wrong. We had an excellent discussion about this, including trying to think of factors that led them to believe what they now had come to doubt. I was joyful to see them begin to dissect the differences between a story map and a hierarchy. We talked about sliceability not as a requirement of a map but as a property of a well-designed map. Perhaps most critically, a student voiced a real concern: if this technique was so hard to learn, was it worth it? For them, of course, it's an open question. For me, who had also made mistakes in my first attempt to learn the technique, I have already paid the cost of learning and reaped the benefits in my studio class. It will be interesting to see what happens to their skepticism as we move forward with our project.

That led us to the real problem: what do we do about this community partnership, given how much trouble they had already had? We agreed that I would make a story map to the best of my ability and we would start into the project using that one. This would give the team the opportunity to work with a reasonable map (I'm still learning too, of course) and reflect, particularly after an iteration is complete, on the impact of this technique. I made the map, with some student input, on this past Monday. We were sure to include conditions of satisfaction with each story, as I wrote about before. but then I headed out to a conference and, after that, Spring Break week. I expect to write about what happens in the back half of the semester later.

Follow-up on Story Mapping: Sprint 2 Planning

My team had our Sprint 2 Planning meeting yesterday, and I started with the story map on the wall that I wrote about over the weekend. Most of the questions the team members had were covered in the conditions of satisfaction that I had written on the note card version of the stories. The team decided to migrate two stories below the cut point, pushing them to Sprint 3 instead. The first of these was the external authentic testing of the software system: since the team was committing to making a high-fidelity paper prototype, they were rightly wary of also trying to budget the time for authentic testing. Since we would focus on a paper prototype, we also pushed down the task of making a design guide to specify fonts, colors, and so on. This was also a prudent move, since if the paper prototype evaluation went poorly, we would need the time to clean that up before investing time in a design guide that would likely have to be done again.

Much of this conversation went as I predicted, but the task planning ended up with some interesting results. Several new stories were added to our map, most of them planned for the next iteration.

These new stories dealt primarily with user interactions that we could foresee only after talking through the extant map and considering the bounds of the stories we had committed to. The new stories include:

• Continue uninterrupted if the page is reloaded
• Install the game as an app (manifest.json)
• Continue after pressing the browser's back button
• Browse previously unlocked content
• Clear previously unlocked content
• Finish the story for Prairie Creek

That last one is not a user interaction, but simply our team noticing that we had neglected to include a story for the creative work of finishing our narratives! By the end of this sprint, we should have significantly reduced risk around our core game mechanics and theme, but there still will be necessary work in fleshing out the narrative.

Another attempt at story mapping in the academic studio

About a year ago, I wrote about how my first attempt with story mapping did not go as well as I had hoped and that my team eventually abandoned them. Last semester, I decided to invest some time in building a better understanding of the technique, so I did what any reasonable contemporary developer would do: I bought and read Jeff Patton's book. One of the key insights I got from this book—perhaps something that's obvious to regular practitioners, but I didn't see at the time—is that story maps can replace one-dimensional backlogs but not Scrum-style task boards. That is, while the story map provides the big picture of the project plan, teams still need other modes for tracking smaller-grain tasks. Among other factors, this made me want to revisit story maps in Spring, and so I have.

This semester, I am once again leading a multidisciplinary undergraduate team in an immersive learning project. I wrote a bit about the project when I wrote about sustainability game design and the conclusion of my game design colloquium. This semester, my team is building a Web-based, geolocative, educational game to complement the educational themes of Camp Prairie Creek. We just wrapped up our first sprint, which resulted in the team building a proof-of-concept experience using Polymer while exploring various narrative themes. Before the sprint, I had laid out a story map based on my vision for the project, and the team had some input into the overall shape of it.

Sprint 1 Story Map

During sprint planning, the team decided where to slice it. We took the stories for this first experimental sprint and transcribed them onto new cards for the task board.

Task Board at the start of Sprint 1

The sprint ended this past Friday, and although not every task was complete, we deemed it a success: the team was able to learn the fundamentals of Polymer, including building fluency with command-line git; we have a technical artifact that shows how various pieces fit together; and we have a more coherent vision for a narrative structure and theme. However, throughout the sprint, I didn't see any students actually using the story map. They would congregate around the task board around the time of the stand-up meeting, using it to direct their communication and activities. The story map, which is physically far from the task board, did not seem to get any attention.

One of my responsibilities yesterday was to recreate the story map based on what we learned. The first pass looked reasonable.

First pass at revised story map

During Friday's Retrospective meeting, the team decided that their work would be improved by the inclusion of conditions of satisfaction for each story. Usually, when using Scrum, I write the names of stories (in Mike Cohn's format) on the front and the conditions of satisaction on the reverse. It was not clear to me if these fit with the story map format, so I had left them out in the name of simplicity. I have one returning team member who was in last Spring's studio, and this student was one who strongly recommended including the conditions of satisfaction. So, after creating that first pass at a story map, I sat down with my large-format index cards and starting re-writing the stories to them.

This process led me to consolidate several stories. This wasn't quite what I expected, but the new articulations were much cleaner, in my opinion: they were more encapsulated rather than spreading one action across multiple notes. For example, "See a map with location markers" and "See my own location on a map" became consolidated into "See annotated map," with conditions of satisfaction indicating that we want to see both our own position and marker positions. These could remain separate, but I was thinking about how they would migrate to the task board: the two small stories looked like they would take up more room than they were worth, whereas one story would just have a few extra rather small tasks. It may be worth noting that the team already has this mostly done in the tech demo, so much of the story will be revising how this feature is implemented within the revised visual design.

Revised story map

I've included below a photo of the task board cards that I expect the team to take in Monday's planning meeting. Of course they will be able to negotiate on what goes above and below the slice, but I suspect that we won't see much change here.

Stories with conditions of satisfaction, for the Task Board

[See follow-up post for more]

Leaving story maps behind

I wrote on the first of this year about how I was excited to try using story maps as a management aid for my Spring Studio team, and in the middle of February I followed up with some of my findings from one iteration's use. The team just finished their second iteration, and after talking about it with them during our end-of-iteration review and retrospective meeting, we are going to drop the story maps and go to Scrum.

The problem with the story map—as we reified it—is that it presumes that the reader already has knowledge of the fundamental processes that are required to complete a story. For example, we had an activity on the story map called "Narrate my friend's story" with a story underneath it called "Read conclusion and result." (Story, Conclusion, and Result are all key terms from our game design and are defined in a design log.) When I look at that story, I can imagine the steps required to complete it: sketch the UI, make sure I have actual conclusions and results, make sure it fits with the overall game UX flow, rough it in code with placeholder assets, start replacing placeholders with real assets, test the integration. The problem is that I am not the one doing these steps: rather, it is a team of people who never did this before. What happened in the first two iterations with the story map is that they would look at "Read conclusion results" and, in the discussion, I would explain that these steps were required. These steps weren't tracked explicitly, though, and so we ran immediately into cognitive overload, and the team moved forward with the illusion that they knew how to proceed. Note that this is basically the same phenomenon as in a traditional class, where you can tell the class something and they will nod but not take notes, believing that they understand; of course these students then fail to follow the proper steps when it comes time to execute the task independently.

I am hopeful that switching to Scrum will make it easier for the team to track its progress on smaller tasks. It will mean more attention devoted to planning meetings, as we break down stories into individually-measurable tasks. One reason I was hoping for a leaner approach using story maps is that this is only a three credit-hour course, unlike the six credit-hour studios of the past few years, and so this will eat even more time away from production. However, if we're not producing the right thing, then it doesn't matter how quickly we do it.

Spring 2016 Studio: Story Maps vs Scrum Board

I mentioned in my last post that I decided to experiment with story mapping as an alternative to Scrum-style task tracking boards. My team had their first release on Wednesday, which led into an excellent playtesting session at Connection Corner. However, the team only was able to make the release due to the herculean effort of a few technical team members, who worked through the day on Tuesday to add missing critical features to the prototype.

It's worth dwelling on that phenomenon itself for a moment. This is part of a three credit-hour course, which means--according to federal standards--that we should expect nine effort-hours per week from the student during the fifteen weeks of the semester. The team has scheduled meetings for six hours per week, which I set up in order to maximize face-to-face time for this multidisciplinary team. At least one student put in about nine hours on Tuesday alone to help us meet this deadline. He didn't have to do this, in the sense that there would be no punitive grade assigned, and the shame at not having met our goal would have been shared by the entire team. Worse, we would not have been able to conduct our playtesting session, which would mean we would still be grappling with fundamental uncertainties today. On one hand, we need to congratulate the students who put in significant extra effort to help the team meat its goals (and indeed we gave a round of applause on Wednesday); with the other hand, however, we need to admonish ourselves for putting us into a situation where so much work was required before an important deadline. How did we get to that point? It seems there are only two options: either it was unclear to team members that we were headed toward missing the goal, or they did not care that we were going to miss the goal. The behavior of this subset of students who put in extra hours leads me to conclude the former.

On Monday, many stories remained on the spine of the story map, all or nearly all with student names on them as responsible parties. On Wednesday--the day of the deadline--I wrote a note on the board in the morning to ensure that the story map was up to date for our review & retrospective meeting, and even then, the story map was untouched.

Every student team goes through some growing pains where they realize that they need to be more careful about communication and planning. The retrospective meeting on Wednesday confirmed that the students were aware of the need to coordinate more prudently and tactically. I am left wondering, however, how much of this is related to the story map? I could look at the story map and tell that we were not making progress, as notes were not moving, but I think they were blind to this. I suppose this iteration will demonstrate to us whether they will hold themselves more accountable to the practice or not. During the retrospective, a student contributed what I think will be a valuable action item: immediately after the stand-up meeting, gather around the story map to review where we are. Today was a planning meeting for the next iteration, and so it will take until Monday to get any sense of whether this helps or not.

Another item that gives me some unease is the tracking of stories (in a story map) vs tasks (with a conventional Scrum board). User stories are necessarily brief, but to address their need requires a decomposition of the story into discrete tasks. Using Scrum, this was done explicitly during the Sprint Planning meeting. For example, given a story like "As a player, I want to create my character" might break down into a design and an implement task, to make it clear that we should figure out a good user experience before programming it. However, in the last iteration, we saw several UI components that were thrown together by programmers--surely with good intention, but just as surely without any background knowledge in usability and game interface design principles! As above, there's too much ambiguity to drive a particular conclusion: did the students not know that they should step back and work with the team to design a good UI, or were they just working quick-and-dirty on a two-week prototype? The truth may be somewhere in the middle, and I suppose I can hope that if it is purely ignorance that this two-week prototype was a learning experience about the importance of critical design!

This leads me to one last point that I want to record here, so I don't forget. This two-week prototype was "slapped together" intentionally. The team agreed on several methodological rules governing source code, but they also agreed that these rules would not hold for the two-week prototype: this could be done "quick and dirty." Late in the iteration, I started looking more carefully at the code being committed to the repository, and I was struck by how one developer's "dirty" might be another developer's "abominable." When I said dirty, I meant we could be lax about things like having rigorous unit tests, or allowing several parameters to constructors rather than introducing a builder. Others clearly had a different idea! This strikes me as a fascinating study for software engineering research: how do different developers approach a project when told they can do it "dirty" vs. "clean", and what are the contributing factors in their decisions, and what are the inherent properties of the code they produce? Maybe someone has done this already, but if not, it sounds fascinating---and it clearly resonates with the kinds of discussions I have daily in CS222.