A proposal for the disclosure of the use of generative AI in scholarly writing

I recently encountered a scholarly manuscript in which the author acknowledged ChatGPT as being instrumental in composing the work. This is fraught with similar problems to students' use of such generative AI tools: how is the reader to trust that the author actually means what is written? 

I propose that those who wish to publish such manuscripts must also release their original draft, prior to its being passed through AI tools. This recognizes that the purpose of these tools is to translate between natural language dialects, and so they should be treated like other translations. While I am reading an English translation of Francois Mauriac's Viper's Tangle, I can access the original French in order to distinguish between the connotations of the original and the interpretation of the translator.

Reading "Fifty Quick Ideas to Improve Your User Stories"

I recently finished Fifty Quick Ideas to Improve Your User Stories by Gojko Adzic and David Evans. The book was strongly recommended by Dave Farley on his Modern Software Engineering channel. I have been using user stories for many years as part of my teaching and practice with agile software development, and I hoped that the book might give me some new ideas and perspectives. I read it with a particular interest in helping students manage game development projects.

The book itself is laid out simply: every two-page spread presents one of their recommendations. Each has a narrative, a summary of benefits, and then practical recommendations for getting started. From the title, I expected the book to be short and punchy like The Pocket Universal Principles of Design, but it contains a lot more detail. The authors are clear that the book is not an introduction to user stories; they assume the reader is already familiar with user stories and the jargon of agile software development. This is good for a reader like me, except that I had earlier lent the book to a student who was just learning the fundamentals. I would use a different resource for that purpose if I could do it again.

Here are suggestions from the book that struck me as potentially helpful to my teaching and mentoring. I have included the page numbers for ease of future reference.

Describe a behavior change (p14)

This is the second time recently that I have come across Bill Wake's INVEST mnemonic for user stories—that stories should be Independent, Negotiable, Valuable, Estimable, Small, and Testable. The other place was Vasco Duarte's No Estimates, which I wrote about earlier this year.

The idea behind this recommendation is to quantify change. For new capabilities, "start to..." or "stop..." are simple quantifiers. I feel like there's something useful in here for introducing students to user stories.

Approach stories as survivable experiments (p18)

This tip is all about the perspectives of what stories allow. I often see students mistake stories for traditional requirements, probably because traditional requirements look more like school. Framing the stories as experiments may help students see that this is more creative and exploratory.

Put a "Best Before..." date on stories (p24)

I wish I had thought of this one. I have mentored student teams where they have a story in the backlog that only makes sense to do by a certain date or milestone. It's an easy one to remember, and it falls into a common pattern in the book: make the stories your own.

Aside: It reminds me of the advice about managing my curriculum vita that I received from my colleague Mellisa Holtzman many years ago. She said that your vita is your own, and that you should use it to tell a story rather than slavishly follow a template. She was right, both philosophically and rhetorically. My university recently moved to using a one-size-fits-none digital solution, and I was disappointed that there has been no discussion about the impoverished epistemology of such a move. 

Set deadlines for addressing major risks (p28)

This is similar to the previous one, and it references it as a technique: one can use learning stories that have "Best Before..." dates.

The authors distinguish between learning stories and earning stories, referencing an old presentation by Alistair Cockburn. This is a nice distinction as well that I am sure I can use.

Use hierarchical backlogs (p30)

Having spent the last year mentoring three teams with flat backlogs, it made me miss story maps, and I look forward to going back to them. Story maps come up explicitly on page 36 under the recommendation Use User Story Maps, which is just after the recommendation on page 34 to Use Impact Maps. Gojko Adzic also wrote a book on impact mapping. I have read most of it and find it intriguing, and I can see how it could be useful in a business environment. It didn't strike me as immediately helpful for my needs teaching the game design and development or the basics of agile software development.

Set out global concerns at the start of a milestone (p40)

The authors acknowledge that user stories cannot and should not capture everything that a team has to do. Writing emails or reading applicant vitae are examples of crucial work that is not directly related to a user story. 

Cross-cutting concerns such as security, performance, and usability should still be discussed, but they should manifest as something like a checklist that accompanies the whole process—not embedded or wedged into a user story.

Use low-tech for story conversations (p54)

The recommendation for whiteboards and sticky notes comes as no surprise to me. What did surprise me was that they caution against using a room with a large central table. Such a configuration makes people sit and opine rather than converse, and what comes out of the conversation (what we often call the "user stories" themselves) are just markers of that conversation.

Diverge and merge (p58)

When user story analysis is being done by a large team, they recommend breaking the team into smaller groups. Each group comes up with examples of user stories, then the groups compare their results. They suggest 10-15 minutes of divergence. 

In last year's game production studio, one team was much larger than the others, and I recommended that they break down story analysis by feature domain. I wanted each group to have representatives from Computer Science and from Animation. It worked reasonably well, but it was slow, and there were still a lot of holes in the analysis: cross-cutting ideas were lost. (Also, most of the students didn't know anything about user stories, and some harbored significant misunderstandings.) Next time, I will keep this advice in mind and have them work on the same context rather than different ones. I regularly have students push back on this kind of activity as redundant but this is because they have not yet experienced the productivity drop that comes from realizing later that the requirements were wrong.

Measure alignment using feedback exercises (p62)

This one jumped out to me because it sounds like it came right from a book about active learning techniques for the classroom. 

During user story analysis, rather than ask if anyone has questions, give the discussants a sample edge case and ask them an open-ended question such as, "What happens next?" Each writes their response, and then the team compares them. This shows whether there is shared understanding or not.

Reading this gets me fired up to use it in all of my teaching from now on. That's awkward since I won't be teaching again until Spring 2026. That gives me all the more reason for this blog post.

Split learning from earning (p90)

As mentioned above, this separates two kinds of stories. Learning stories still need a concrete goal that is valuable to stakeholders. An interesting corollary to this advice is that every story should be either learning or earning; if not, it is not really a user story at all.

Don't push everything into stories (p98)

I foreshadowed this one earlier as well. They advise strongly against tracking non-story work with user stories. An example they give is insightful: if such work is tracked in the backlog, and the stakeholders sort the backlog, then they will deprioritize work that does not generate value. The non-story work won't get addressed, then, until it is a crisis, at which point it becomes the highest priority, so there was no value in having it in the backlog.

Avoid using numeric story sizes (p102)

The two-page spread summarizes some of the key arguments behind the No Estimates movement. Here, they recommend that if someone uses T-Shirt sizes, to only use at most three different sizes. Better, they prefer Goldilocks sizing: a story is either too big, too small, or just right. Their conclusion is similar to Duarte's: just count stories.

Estimate capacity based on analysis time (p106)

If we know that planning for an iteration takes 5%-10% of the iteration time, then we can timebox the actual planning and then only work on the things that could be covered in that planning meeting. That is, use the planning meeting's duration as a heuristic for the scope of an iteration. This is clever, and the authors acknowledge that the moderator needs to prevent the meeting from getting too far into the weeds.

Split UX improvements from consistency work

They recommend having UX changes explored by a "mini-team" that explores and prototypes. The output of this mini-team's work are the user stories for the whole team, not exhaustive design specifications. The work of this exploratory team is on the learning side of the learning/earning divide. The mini-team can slipstream back into the main team to help with implementation.

This interested me considering how often I see teams struggle with questions of gameplay and engagement. "Will this be more fun?" is a good design question that can be approached through prototyping. Unfortunately, my teams often want to do this work as if the answer is a fait accompli. I wonder if this recommendation, together with the learning/earning split, will help me frame better for them what it means to answer the design question with a prototype first. The outcome would be the stories for the whole team to actually engineer a solution. I suspect I will encounter much of the same resistance I mentioned previously, where teams assume working on the "product" is more efficient than working in prototypes because they have never seen a product fail under bad design or bad engineering. I think this is why I prefer working with students who have tried hard things and seen how mistakes have consequences beyond a scolding or a bad grade.

Check outcomes with real users (p116)

The authors proclaim this as potentially the most important thing a team does. The important corollary here is that this requires one to write user stories the way one approaches TDD, or even how one ought to design learning experiences: focus from the beginning on an assessment plan.

The outcomes of the 2024-2025 Game Production Studio

I have been working with a multidisciplinary cohort of students from Computer Science, Art, and Music in the Game Production Studio for the past three semesters, team-teaching the experience with Antonio Sanders from Animation. At the end of this process, we had three games released for free on itch.io:

• Food Truck Cartel
• Milky Way Market
• Weatherbreak: Climate Overdrive

I am proud of the work these students have done, but it was a very different experience from the previous cohort. Our third cohort of students in the Game Design & Development concentration are currently wrapping up preproduction, but I have not been involved in that course. I am out of the sequence for a year as Travis Faas steps up to lead the sequence, which frees me up for Fall's sabbatical that I mentioned yesterday.

One of the primary differences between last year's and this year's cohorts was scale. Mentoring a single team of six students is very different from mentoring 25 students on three teams. Some things I did intuitively needed to have been more structured. The clearest example is the asset pipeline. Both years, there was a studio policy that everyone needed to be able to open and edit assets and get them into the engine. It was easy to confirm this with my smaller team, and for the larger one, I took students' word for it. It became clearer to me late this Spring that a significant number of students could not, and never did, know how to do this: instead, they relied on manual hand-off procedures. The point here is that, with a small team, I could simply observe them in every step and know they had met expectations, but with the larger team, that did not scale. "Trust" and "verify" needed to be coupled, especially since it wasn't even clear that the students who had previously certified their compliance with the protocols understood what it was all about. Release engineering and quality assurance were two other areas, along with asset pipelines, where I know that I will need more formalized approval processes in the future. I don't know yet how much those processes will look like industrial ones, with some kind of threat of firing or demotion, or academic ones, wearing traditional pedagogic trappings.

Managing three simultaneous teams made it difficult to determine the timing of iterations. We ended up keeping all the students on the same schedule, which meant that there was one day when each team was doing review and retrospective meetings. One of the best decisions my co-teacher and I made was to have both of us sit in on the review meetings, and that's something I need to keep for next time. However, in reflecting on the semester, I think I should have also been there for the retrospective meetings, at least at the beginning of production. This would help me model the process for the students. Also, it made me reflect on how in my immersive learning classes, it was the retrospective—not the review—where I made a lot of my pedagogic insertions. Once a team identified a pain point, I could direct them toward an industry standard solution via just-in-time teaching. If I have another large section, I need to consider staggering their iterations so that I can be at both meetings.

We gave our students a short written final exam that was inspired by the questions we used during team retrospectives. I took notes about their responses to the final question, which was to identify something that they would do differently during production, knowing what they know now. I consider this one of the most important outcomes of the entire three semester process, and so I will summarize their responses here. I have clustered their responses and removed any identifying information, of course.

Work management. We used HacknPlan to manage work, but each team had slightly different stories about how this didn't go as well as they would have liked. Several students commented that using physical, visible task tracking was far preferable. It makes me want to bring in the same kind of task boards that I used in RB368 for my immersive learning classes. How to do this with equitably in a shared lab is an issue to figure out.

Scope management. Several students commented on how their scope was not well under control. I have been thinking about this as well, especially since I recently read Vasco Duarte's No Estimates book, in which he advocates for having proximal work in higher focus than distal work. This is somewhat contrary to the advice given by Lemarchand in Playful Production Process, which is our reference text for the studio. Lemarchand advocates for having a full production plan at the end of preproduction, but at the same time, he recommends breaking down a production phase into iterations—without really explaining how one might do that. I think Duarte's model is going to be more helpful here, and I'm thinking about abandoning the entire "macro chart" element of preproduction in favor of epic stories.

Code quality. A few students commented on how they needed to have better control over their technical implementations. Some mentioned a bad habit of pulling the first tutorial they could find into their production code, and how this would lead to an unmanageable mess of ideas. Others talked about how they should not have tried to reuse the preproduction code since it was too quick-and-dirty. A few recognized that introducing more automation would have saved them a lot of trouble, but also that their failure to refactor meant that their code was not amenable to testing. 

Iteration. Some students recognized that they needed more iteration on their core design loops and on their assets. Our students had a bad habit of taking an inspiration directly into production without low fidelity prototyping and experimentation. It was good to see them talk about how they should have done more sketching and playtesting. I do wish they had appended "as you instructed us to do" to their answer.

Vision management. One of the strangest things to me is that, despite my repeated suggestion and admonition, I don't think any team had a vision statement: they didn't articulate design goals or experience goals. This either caused or is a symptom of significant problems. Conversations about design alternatives became about force of personality rather than alignment with a vision. It was good to see that some of the students recognized this and highlighted the need to articulate vision as something they would do differently. It is the kind of thing that, in my producer hat, I can demand of future teams, and in my professor hat, stick a grade on so they know I am serious.

Marketing. Some students recognized that decisions made in preproduction lingered far longer than they should have. This is strongly related to the previous point about vision management. From my perspective, it looked like these students were on the road to recognize that they were rationalizing the past rather than being intentional about the future. I see this frequently with my students, that they will tell themselves a story about what their vision must have been because it aligns with what they already did. It's very human. It's also dangerous for collaborative creative work.

Specificity. This final pattern in the students' responses caught me by surprise, and I was delighted with how students articulated such an interesting problem. Many recognized that their discourse throughout the project was often at a level of abstractions but that the design and development work they needed to do was at the level of concretion. Again, I think the lack of clear design and experience goals contributed here, but it's also a case that makes me think about techniques like Stone Librande's one-page design documents. Looking back at the three projects my students made this year, each one of them could have been expressed in a one-page design and hung on the wall.

It was a trip working with these students for the past three semesters. They should all be proud of their accomplishments: making games is hard! Some of them really impressed me by learning new things, working through social and technical challenges, and devoting themselves to solving hard problems. I love teaching this kind of class, but it will also be good for me to take a little break. I look forward to what a sabbatical can do for my perspective on some of these ideas.

Reflecting on CS222, Spring 2025 Edition

This semester's CS222 class was unlike any other. My department is once again involved in a 1+2+1 program with a Chinese university, meaning that Chinese students complete their first undergraduate year in China, their next two years in the USA, and then their final year back in China, culminating in dual degrees. It was a vibrant program many years ago, and I am glad to see that it has come back. In the Spring semester, I had all of the eligible 1+2+1 students in my section, which meant that half of my enrollment consisted of domestic students and half consisted of Chinese nationals. I used to have some in my classes, but I've never had half of the enrollment in a class be international students before. It meant that on most days, there were more native Chinese-speakers in the classroom than English-speakers. 

One of the things I realized this semester is how often I speak in idioms, using phrases whose literal translations don't have an obvious meaning. For example, I told one of the teams that they had gotten "in a pickle." Then I laughed and wrote that expression on the board, gesturing to it and pointing out how ridiculous it was. I explained that it meant that the team had gotten themselves in trouble, possibly of their own creation. 

Unfortunately, moments like this were not as powerful as I would have hoped. Turning to my class, the domestic students were smiling as they contrasted the literal and figurative meanings of the phrase, but almost all of my Chinese students were as they always are: eyes glued to their monitors or smartphones. A guest speaker's presentation allowed me to sit in the back of the room and verify my suspicion about what was going on: most of the students had their phones propped up on their laptops and were running voice translation software. They were not listening to the speaker in any significant sense but reading in Chinese what it thought the speaker was saying. Occasionally, a student would switch to the laptop to visit a site or verify a term, but mostly they were reading real-time translated transcriptions of what was spoken. Not all of them did this, but most did. Some listened and took notes. One watched Chinese television. In this way, they are not unlike domestic undergraduates.

As usual, we completed a major project lasting about 8 weeks, and it was split into three iterations. I gave lengthy feedback to each team's submission, and sometimes teams even read and responded to it. An unusual frustration from this semester was that none of the teams really nailed the final iteration. I expect 20% or fewer to get the process right the first time, and then maybe 40% on the second iteration, and I usually get 60% or better addressing the fundamentals by the third iteration. I didn't have that this time: each project had something fundamentally wrong that I had already pointed out to them in a previous iteration. One conclusion from this is that I may need to rearrange or remove some of the elective content from the class in favor of more supervised in-class practice. Another consideration, though, is that I hadn't accounted for the extra labor done by international teams. I strongly encouraged—but did not require—international project teams, and most of the teams were. This means that in addition to tackling the significant challenges of the course, which include trying to change habits and conceptual models around programming, these teams were also dealing with language and cultural differences. That work is real work but it was outside the grading scheme. When I laid out their final grades yesterday, I decided to add a flat boost to the international teams to compensate them for undertaking the challenge. The resulting grades looked more appropriate to me for how well I knew the relatively small class. If I were to teach another class with such a high proportion of international students, I would need to revisit the question of whether an up-front incentive might be fruitful.

Course achievements were once again counted as regular assignments rather than their own category, as I did last semester. I do think this works better since it lowers the cognitive load of the course. My department currently has unprecedented growth in student organizations, and I need to bring back some of the achievements related to those. 

Counting achievements as a regular assignment has helped students navigate that part of the class, but I have yet to crack the problem that many students who ought to resubmit work don't do it. The idea is that students resubmit work until they have shown an understanding of it, and I am not sure what impedes students from doing this. It's possible that it's too liberal for Canvas to handle. Canvas won't tell them that something is due because they can resubmit any past assignment once per week. I suppose I could make a separate weekly resubmission assignment, which may solve the problem that some students fail to understand what "one per week" means. It feels like catering to their LMS addiction to me, but maybe it's what they need to help them through the challenging content.

I'm still stuck on the question of whether the content of the course would work well as a portfolio. Take a principle like naming: I could require students to submit in a portfolio an evaluation of something they wrote before 222 (which I do as an assignment) along with a sample of work that demonstrates their following the principles. Such a portfolio would be a powerful testament to what they have learned, but I've struggled to figure out how to pace such a thing. When I first started using achievements in CS222, all the assignments were from a grab bag of achievements, and in many ways, it was a portfolio-style assessment. As I put more emphasis on the project, I had to put that aside. I mention it here in part because, as I understand it, the 2026-2027 academic year will see a transition from a 15-week to a 14-week semester here. That will require me to blow up the course and rebuild it since there's no way I could just trim a few things out and have the rhythm still match.

The last note I'll mention here is something from my planning document, where I keep short notes about how things went and what I want to change the following year. This year, I had several teams undertake projects whose nature did not fit the requirements of the final project. The reasons undoubtedly come from a combination of background, culture, language, and generational differences. To cut to the chase, I realized I need to make it even more explicit that the final project needs to do something computationally interesting: being a clever interactive app is not enough. Some of the projects students wanted to undertake would have been great in a course on web and mobile apps, but they were not good contexts for exploring TDD for the first time.

I will be on sabbatical in the Fall, which means I'll be stepping away from this and all my other regular courses for a semester. It will be a good chance to catch my breath, and I won't have to carve out a week over the summer to rebuild the course.

What we learned in CS222, Spring 2025 edition

I usually like to make the What we learned exercise the last thing in the semester before the final exam, but this year, I had to move it up a little. I had two seniors come to class today to talk about their experience, and that meant that the remainder of our meeting was just enough time to do this exercise.

In 30 minutes, the students came up with 109 items. I gave each student six stickers, and they voted on the items that were the most important to them. These five rose to the top:

• Test-Driven Development (13 votes)
• Clean Code (11 votes)
• Programming intentionally (6 votes)
• Model-View Separation (5 votes)
• Canvas stupid (5 votes)

The students recognized that many of these top items are categories rather than particulars and so tended to attract more votes, but that's fine with me. The list is still remarkable in two ways. First, the third most important item to this population was programming intentionally. I don't remember this coming up as an outcome of the course before, but it's a fascinating sentiment. It is different from saying "We are using Clean Code" or "We are using Mob Programming." It is a statement of how we even go about making those kinds of choices, which is great. Maybe if I ever pull all my ideas together into a book of programming advice, I'll call it Programming Intentionally.

The other noteworthy thing on the list is the last one. It's the first time I remember a "joke" entry showing up as a top item. Any good class is going to have some funny items on the list, especially once they relax into the exercise of reflecting on the semester. In this case, "Canvas stupid" was my shorthand for a student's much longer comment, which was reflecting on my telling them how the way that Canvas deals with points is stupid, that you cannot deal with small numbers nor large numbers adequately. In my particular case, I believe I was ranting to the class about how I want to normalize scores into the [0,1] range, but how Canvas has hard-coded two decimal places. I even reached out to Canvas support earlier this semester to see if we could enable more somehow, and I was told it was hardcoded into their implementation. 

Article in Well Played special issue, "For the love of games"

I have an article in the latest issue of Well Played. It is a special issue with the theme, "For the love of games," and the editors invited articles from games professionals about a particular game that impacted our career paths. My article certainly has the best title I have written: The Thief of Fate and the Devil Whiskey. It was a delight to write, and I hope readers enjoy it.

Making Dart test helper methods show up in the structure view of Android Studio

Why such an awkward title for this post? It is the kind of search I was doing the other day when I found no hits. I'm writing this in hopes that I can save someone else the trouble I faced when the solution is, in fact, quite simple.

My Dart project contains many unit tests, but many of them use helper methods rather than calling test directly. Unfortunately, I was deep into this approach when I realized that these tests were not showing up in Android Studio's Structure view, which is otherwise a good way to navigate files.

Here is an example to illustrate the problem. Notice that the call to test is nested within the helper function, testExample.

 import 'package:test/test.dart';  
   
 class Example {  
  bool isEven(int value) => value % 2 == 0;  
 }  
   
 void main() {  
  final example = Example();  
   
  void testExample(  
   String description, {  
   required int value,  
   required Function(bool) verify,  
  }) {  
   final result = example.isEven(value);  
   test(description, verify(result));  
  }  
   
  group('Example.isEven', () {  
   testExample(  
    '2 is even',  
    value: 2,  
    verify: (result) => expect(result, isTrue),  
   );  
  });  
 }  
   

When this is opened in Android Studio, the Structure view looks like this:

Notice how the group is empty. 

The other day, I tried different searches to find an answer. It seemed to me that there had to be some way that unit testing libraries communicated their structure to JetBrains IDEs; it could not be the case that the JetBrains engineers were doing simple string matching in source files. Yet, I had no luck. In my confusion, I even turned to ChatGPT, which confidently told me that the only way to do it would be to refactor all of my tests into (yet more) higher-order functions so that I was calling the standard test function at the top level. I asked it for a reference, hoping to find the documentation I had unsuccessfully been searching for, and it unhelpfully pointed me toward two web sites that don't address this issue. Still, I put a potential refactoring into my project plan, although with over a hundred tests and counting, and with a rather eloquent current solution, this would have been both tedious and disheartening.

One of the reasons I started writing helper methods in this particular style—that is, by sending named functions as parameters—was because this is how the bloc_test package handles testing. A day or two after having tried to search for a solution to my problem, I was in a test file and noticed that all my blocTest calls did show up in the Structure view. How was that possible? Thanks to the MIT license of bloc_test, I checked the implementation and found the @isTest annotation, which comes from the meta library. Quickly reviewing its documentation, this was clearly exactly what I needed. I included this little annotation to my project.

 import 'package:meta/meta.dart';  
 import 'package:test/test.dart';  
   
 class Example {  
  bool isEven(int value) => value % 2 == 0;  
 }  
   
 void main() {  
  final example = Example();  
   
  @isTest  
  void testExample(  
   String description, {  
   required int value,  
   required Function(bool) verify,  
  }) {  
   final result = example.isEven(value);  
   test(description, verify(result));  
  }  
   
  group('Example.isEven', () {  
   testExample(  
    '2 is even',  
    value: 2,  
    verify: (result) => expect(result, isTrue),  
   );  
  });  
 }  
   

The result was that my Structure view looked exactly how I wanted it.

There you go: the solution in Dart is to annotate the helper method with @isTest. The documentation of that annotation make it clear that it solves my problem, but I must not have hit the right keywords with my original search. I hope that this post helps anyone else who is caught by this issue.