I had a break from teaching CS315 Game Programming last Fall while I was on sabbatical, and now I'm gearing up to get back into it. It's a fun course that has always had the same mission but has taken many different forms. I'm feeling pretty good now about my plans for Fall, and so I want to document some of the key points here. The draft course plan is already online. As with my CS222 course, there are still some areas where I want to rework the prose, and I have only posted activities for the first few weeks, but I feel like the structure is where I want it to be.
Grading
Three years ago was the last time I used checklist grading in CS315 in its original form. Two years ago, I switched to a different model inspired by a book I had read. I wrote about how it had some good qualities, but it caused friction with Canvas, and it also had me doing a lot of regrading. Thinking about my plans to kick Canvas to the curb, I figured I might get rid of the interface problems. Yet, the more I worked on my plans, the more I kept drifting back to checklist-based grading. I drafted the first four weeks of exercises this way while I tried to identify why I still felt unsettled.
At some point, it struck me that a completely different way to handle functional requirements articulation is through user stories. This is the way I do it in the prerequisite course (CS222), and it's also how I coach teams to do it in the CS315 final project. Why not deploy user stories in the weekly exercises as well, thereby also giving example of something I want students to do later? It seemed sensible, and so the only missing piece was to map this to grades. I decided to go with the same simple model I started using recently in some CS222 sections, where the grade is based on the number of completed stories.
I am happy with the articulations I have put up for Exercise 0 and the three increments of Exercise 1. The only piece that gives me some pause is that the grading scheme does make it seem like missing a story is punitive rather than that meeting stories is rewarded, but I think this is part of the systemic problems I have inherited. At least I don't think I'm reinforcing them, and I can explain to students that they still have the option to skip lower priority stories and earn a lower grade if their situation makes that prudent.
As part of my Canvas-kickery, I'll be using GitHub issues to give student feedback. This is a natural place to give feedback in the first half of the semester, when students are working individually on projects. I haven't thought much about the final project yet, but I suspect that once the semester is running, a path will open up for me. CS315 is not a course where we traditionally use Google Drive, but since I'm already collecting students' GitHub usernames, I can also collect Google Account names in order to set up a dashboard system like I described in my previous post.
Style
There are two major benefits to teaching students to have good style in this course. First, there is the intrinsic value to the student of learning to follow an established style guide. Second, it makes it much easier for me to read a lot of student projects if they follow the same structure. It is not so different from requiring a particular style in order to approve a pull request.
For some time, the exact style requirements have been distributed through the early semester exercises. This had the advantage that it allowed me to ramp up expectations as students became more comfortable with the game development ecosystem. The downside was that students had to build up knowledge piecemeal about the expectations, remember where to find reference material, and potentially build up bad habits until the right models were shown.
This year, I've done something I've wanted to do for several months: I constructed my own style guide. It has its own page on the course site, and all of it is required from the second week of the semester. It consolidates many aspects that previously were spread out across different grading levels in the checklists. It also puts almost everything in one place, which I hope will be useful. One place I could not easily bring into one site is the actual code style, for which I am simply having students follow Godot Engine's published conventions. For commit messages, I used to ask students to read Chris Beams' article on commit messages and follow his guidance, and this was frequently a source of frustration for students: some didn't understand version control well enough to follow his arguments, many skipped reading crucial parts of it, and some just had bad habits. Now, I have simply included his seven rules and linked to his article for more information.
Accessibility
The course site is built from the same Astro configuration as the CS222 one I referenced earlier. Writing and revising in Markdown is quick, and component-oriented design can be useful. One of the main reasons I want to use Markdown is that I want to easily include source code in the exercises. Unfortunately, the default Astro formatter violates WCAG 2.1 AA requirements that are now being required for all teaching material.
It took several hours to sort out a solution, and I ended up using the github-light-high-contrast Shiki theme in order for my code to pass muster. It does not have the aesthetics that I wanted, but at least WAVE says that it passes all the requisite checks.
As described in my course revision post in June, the overall structure of CS315 Game Programming was unchanged from previous semesters: half the semester was spent on weekly projects designed to build skills and confidence, and half the semester was spent on larger projects.
The most significant change was in how those weekly assignments were evaluated. The past several years, I have used checklist-based evaluation, but I was hoping to find a fix for the problem of students doing the checklists wrong. This takes something simple and makes it into more work for me than if it was just a point-based rubric. Unfortunately, the strategy I used did not make things any simpler. Instead of checklists, I gave students a list of the criteria that needed to be met in order to be satisfactory. Their work then was assessed as Satisfactory, Needs Minor Revision (fix within 48 hours), or New Attempt Required. New attempts could be made at the rate of one per week, as I've done for years in most of my non-studio courses. I ran into a bit of the same problem as I wrote about yesterday, where Canvas' "Complete/Incomplete" assessment combined with no-credit assignments leads to a bad user experience, but it was not among the dominant frustrations. Those frustrations were two: students not submitting satisfactory work, and students not submitting work.
The first of those is the most disconcerting. As with checklist-based grading, I gave the students the precise criteria on which a submission would be graded. All they had to do was to meet those, and most of them did. Sometimes it took minor revisions or a new attempt or two, but these were no big deal: handling and correcting misconceptions is exactly what the system is supposed to do. The real problem came from students who submitted things that were wrong multiple times after I had told them what was wrong. In a strict reading of the evaluation scheme, this means the work was still simply unsatisfactory, whereas in other schemes (including checklist-based) they might have gotten a D or C for the work. I am still torn on this issue: was the system unfair to students of lower ability or was it the only fair thing to do with them? Put another way, is it better to give a student a C when they still have serious misunderstandings, or is it better to clearly tell them that they should not advance until they understand it? I don't interpret any of the criteria I gave as strictly "A"-level. That is, it did not require excellence to meet those criteria. What it required was rigor.
The other problem, of students not resubmitting work that needed to be resubmitted, seems unrelated to the evaluation scheme chosen. Speaking with professors across campus and institutions, this seems to be part of a generational wave of challenges. I have a few hypotheses about root causes, but the point of this blog post is not to opine on that topic.
Some of my early-semester assignments take the form of multi-week projects. For example, the set of assignments involve creating an Angry Birds clone. It is submitted as a series of three assignments with increasing complexity, and the complexity is scaffolded so that someone who has never made a game before can follow along. I had a student in the class this semester who fell behind, and then he wondered if he could just submit the final iteration of that three-week project as long as it showed mastery of each week's content. I ended up declining the request. One of my reasons is that the assignments double as a sort of participation credit. It makes me wonder though if it's worth my separating these things. For example, something I've done in other courses in the past is make it so that the final iteration's grade supercedes earlier ones if it is higher.
This was the first semester that a colleague offered a different section of CS315 during the same semester. Looking at his students' games, as well as some recent conversations in the game production studio, made me realize that I should probably emphasize the build process more in my section. Rather than simply running their games in the editor, I should ensure that they know how to create an executable or a web build. It's an important skill that's easy to miss, and there's a lot to be learned by seeing the differences between running in the editor and outside of it.
Now that we've grown the number of games-related faculty in my department, there's a chance I may not teach game programming again until 2026. I expect I will come back to these notes around that time. The biggest pedagogic design question I will need to consider is whether to return to checklist-based grading (with its concomitant frustrations) or move to something else, like a simple point distribution.
Game Programming is such a fun course to teach. In the best of cases, I get to bring interested students along with me on a journey through the fundamentals of the craft. This semester, I had several such students who clearly understood what we were doing, why we did it, and that it was interesting. These students were great, and I hope I take nothing from them by focusing on areas of the course that disappointed me. The challenge is always to keep what works and change what doesn't.
Checklists
The first topic to explore is checklist-based grading. Game Programming is the course where I pioneered this approach, and I have generally been keen on it. However, I have never had so many cases of students failing to follow protocol as I have this semester. No matter how many times I reminded, remonstrated, encouraged, or commanded, a significant number of students checked items that were incomplete.
I wish had more data to distinguish between cases where students thought they had items that they didn't vs. cases where students were either lazy or trying to get away with something. The idea of Save Points—where students get an opportunity to resubmit work and address one of the missed criteria—is for the former, not the latter. Responsible students do not struggle with this: they make mistakes and fix them. The mystery comes from irresponsible students.
My lovely wife made a suggestion to me just the other day based on her observation that there is no penalty to the students for checking all the boxes and forcing me to deal with it. There's a solution, then: impose a penalty on checklist errors. I wonder if that one simple fix will help students see that I am serious about checklists being their responsibility, that they should not check an item that they are sure is satisfied. The impact of honest mistakes could be softened by being able to use a limited number of Save Points on such errors.
I also wonder about the mode of checklists. I wrote before about the complications of having the checklists in version control within the project (although I cannot find it right now to link to it). I suspect part of the problem that irresponsible students run into is that they download the checklist and start marking things, and possibly the project changes afterward. For multi-iteration projects, I am sure that students just keep the checklists in place and do not re-evaluate them. I could circumvent these problems by requiring that checklists be completed on paper. That is, I could make it easy for students to download printable checklists, and then have them turn these in at the deadline. It would be extra paperwork to manage, but it would be worth it if it helps them understand the gravitas of checking a box.
Scoping and Slicing
I feel like I had more students struggle with how to slice game programming problems this year than in the past. It's a hard problem! I do have them submit project plans to me, which I comment on, but it's not clear to me that they are acting on my feedback. That is, I may point out that their plans for a user story are too big, for example, but this doesn't seem to stop them. As above, it's hard to distinguish between whether they don't know how or aren't taking the feedback.
It is not clear to me what I could do to improve this aside from give them more practice at this. At some point, it may simply be a combination of scale, difficulty, and lack of experience. That is, with this many students, doing something that is legitimately hard, without knowing that it's hard, it leaves me in a difficult position. Put another way, one way out of the problem is to declare it instead as a learning objective. This is potentially useful as this course, which used to be terminal, now leads into our preproduction class in the Game Design & Development concentration.
However, it's also possible that there may be other cognitive tools I can teach the students to help them understand the difference between a half-baked cake and a half tray of cupcakes. Too few students come out of CS222 with an internalized understanding of the agile approach despite the use of incremental and iterative development there.
One way to help them with this would be to spend more time together. That is, we could take a page out of the College of Fine Arts playbook and make this a studio class, with double contact hours. I've been teaching studio courses like this in Computer Science for many years. I don't think my colleagues are aware of it, which is only relevant when you're actually spending twice the contact hours with a class than otherwise. At the risk of stating the obvious, it's twice as much time. If we made Game Programming a studio course, I would have twice as much time to work alongside with students and help them out. The cost is that I would essentially give up half a day a week to the endeavor. I still struggle to see exactly where the optimum is on the cost-benefit curve here.
Engagement
I'll close with an odd little story. The penultimate meeting of the class was my opportunity to introduce some topics of artificial intelligence to the class. Specifically, I talked about behavior trees, which is a topic for which I have had a group of students doing research with me this semester. This presentation had to start with some lecture about the material. Once I got the basics out of the way, I gave the students a challenge to try drawing a behavior tree for Tag. I told them to get into groups of three or four to see if they could sort out how to combine selectors, sequences, and actions to write a simple in-engine simulation.
About half of the students clumped up into two groups and began approaching the problem, either on paper or at a whiteboard. The other half? Well, one of them looked like he tried to get the guy sitting next to him to talk, but he was rebuffed. The rest of them just sat there, faces into their laptops, exactly as they had been a moment before. This isn't the kind of course where I can just say "Laptops down" since we were working together in a demo repository, but clearly, there was a mismatch here of expectations.
I was faced with a dilemma: should I call out the people who were disengaged or should I ignore them? I decided to simply ignore them. The day before the final presentations, I just cut off half of the room and stopped talking to them while the other half of the room and I explored the nifty space of behavior trees. I'm not exactly happy with this decision, but I think it was the right one. It may also be worth mentioning that a third of the class was not there, so really there were three categories, not two. It will surprise no one to hear that there's a near perfect correlation between the performance of students across the categories with respect to measurable effort and results.
The problem is that this was just a clear symptom of the disengagement that had happened well before. Part of the issue is the room in which we had the class. It is a great space for studio work and a terrible place for interactive workshops. My usual room makes it easy for me to walk around and see what people are doing: it is practically or literally impossible in the room I was in. I leave this note here in part so that I can make sure next year that I am back in the teaching lab. I think if I had a more clear pattern of walking through and seeing what's on students' screen, I would not have had a whole side of the class physical and mentally far from me.
I'm pretty sure I've never done a "Summer course revision" this early in the year, but I recently had a radical idea: what if I did my job while I'm getting paid instead of when I'm not? Sorry, I don't mean to be too cynical. In fact, I am grateful. I have an opportunity right now to do some of the work that I'd normally have to push off until summer.
The danger of doing any kind of course planning this early is that teaching assignments can still be shuffled around. Right now, I am scheduled to teach one of my favorite courses: CS315 Introduction to Game Programming. Reviewing my notes from last year's class... it turns out, it was pretty good. The changes I want to make are minor, and I think I can mostly take last year's plans and repeat them. One of the big differences is the release of Godot 4.0, which means I will either have to recreate some videos or link to them with caveats.
The course plan is already up, and I have the first four weeks of projects revealed. The following weeks should be similar to the previous year's class, but I don't like to have all of that open at once. This policy gives me a chance to plan ahead while also having the opportunity to get to know my students and see where their interests lie.
As a result, there's really not much of a post here. This post is mostly here so that if I later check to see when I worked on planning Fall's CS315 class, I'll find this.
Following up on my previous post, here is another short reflection on one of the courses I regularly teach: Game Programming, also known as CS315. My students are taking the final exam right now, so it seems a good time to record some of my own thoughts about the class.
I did not make any major changes to this class, and it continues to be one of my favorite classes to teach. Godot Engine is just the right level of abstraction for junior-level CS students who are just getting into game development: they can make things reasonably good-looking and fast while still getting into some serious details of game programming. Checklist-based evaluation means that the work is fairly easy to grade and the students by and large understand what is expected of them. Yes, there are always some hiccups here, but there's not enough to warrant radical change.
The class involves several sessions where I am demonstrating concepts and students are thinking about how the pieces fit together. They did a good job with that this year, and they asked better questions than I remember past students asking. Of course, some of the questions could just be answered with, "Try it and let us know," but still, I didn't want to prevent their asking questions. The flip side of this, however, is that they were terrible at taking notes. The fallacy that they will remember lectures is strong with them. I am not sure how I can use the class to help fight against this ridiculous misconception that plagues so many students.
There were a few sessions that I expected to finish in one class period that ended up taking more than two. For example, I put together an interactive lesson on shaders, and for some reason, it filled up a lot more time than I expected. I'm simply noting this here so that I can keep that in mind for the future.
There are some bolts yet to be tightened on the course. The final project has too many checklist items that are easy to cheese. For example, given an item like "Add pop to your UI using AnimationPlayer," some students do a lame fly-in and call it done. I am not quite sure how to invert this. That is, what I want is for them to do something they are motivated to do, and I give it course credit. One of the downsides of checklist-based grading is that some will do the minimum required and present this as if it demonstrates mastery. It's not obvious to me where the dividing line is between those two, especially in the absence of a game design prerequisite. That is, for this course, they can want to make something pretty dumb, and it's hard to distinguish then between whether what they wanted was dumb or whether they were bad at making something better.
I look forward to continuing to teach this course as a generation of students comes up through the Game Design & Development concentration. I am hopeful that we are bringing students who are even more excited to excel at these ideas rather than just barely pass the bar. I expect I will continue to use Godot Engine, and I am hopeful that next year, it will be Godot 4. Indeed, right now I am evaluating whether I want to build a prototype in Flutter or in Godot, and a large part of it is because Dart is just such a nicer language to express ideas than GDScript 1.0.
A few weeks ago, I wrote about how I removed a cyclic dependency from my CS315 class by moving the project report from a git repository to a GitHub wiki. This solution was clear, simple, and wrong. It turns out that GitHub wikis are only available to public repositories when using GitHub Free. Over the weekend, a student working on the first project reached out to me, pointing out that the wiki feature didn't seem to be available. I confirmed the problem and set about seeking a new solution.
After some consideration, I decided the simplest solution was to remove the checklist item that says that a student has submitted their work on time. This item was the crux of the problem, since it forced someone to either leave it unchecked at the time of submission or to check it prematurely. I did not want to change actual course policy, however, so I had to add a general clause stating that work has to be turned in by the deadline to receive credit. This is a little less elegant, since now a student's grade is determined by a combination of checklist items and a general policy. However, it's a familiar kind of policy, and so I hope students don't mind the inconsistency.
It turns out, this is actually a change in policy, although I didn't think about it until writing this blog post. Previously, someone could use a Save Point to resubmit work that was late, since the deadline itself was in the checklist. Now that it is not in the checklist, the course plan gives students no option to earn credit for submitting late work. This was not quite my intention, since the whole idea of having a Save Point is to deal with unforeseen circumstances, and such circumstances do sometimes lead to the inability to submit work on time. I suppose I better get back to editing the course plan to add another general rule, that Save Points can additionally be used to submit work late. That is even more inconsistent and inelegant, but I have not found another approach to fill the bill.
Incidentally, the first "easy" solution I came up with was to have students just submit two things on Canvas: a link to their repository and a separate project report. Unfortunately, Canvas does not seem to support this. Even though the teacher's interface makes it look like you can require a student to submit both items, what students see is that they can only submit one or the other.
My recent blog post on the topic confirmed my recollection that I've been generally happy with the Game Programming class. I've just published the initial course site, which includes the first three project specifications. I expect these will be followed by a midterm and the final project, but I want to meet the students before I commit to that. I made some minor wording changes to the overview page but no policy changes.
I did make one important change to the projects: I got rid of a race condition. The students will still be participating in checklist-based assessment, where I give them a checklist and then turn it back in to let me know what they did. They will also be using GitHub's Release feature, along with semantic versioning, to mark the end of each iteration of multi-iteration projects. Previously, however, these two requirements were at odds. The checklist was part of the README.md file, and the requirement to make a Release was one of the items. This means that a student had to choose one of the following:
Check the box before making the release, commit that change, and then make a release. This means that they have checked the box before doing the work, which is contrary to the idea of checklists.
Make a release, then check the box and make a commit. This means that the README.md file was not up to date in the release.
I puzzled over this a bit and, after some experimentation, decided that a good solution was to remove the project report (which includes the checklist) from the README.md file entirely. Instead, students will now be completing their project reports using the Wiki feature on GitHub. The wiki is still maintained in Markdown, which means I can keep the model wherein students download the checklists from the course page. However, it is not in the source repository but is actually a separate one alongside it, a fact that can be confirmed by cloning one of the wikis.
The next Ludum Dare once again aligns with the second iteration of the second project. Like last year, then, I will be allowing students to either complete the second iteration as given or participate in Ludum Dare. Last year, this option was nested into the Project 2 Iteration 2 specification, which left it unclear exactly how it would be evaluated. I have pulled it out into its own heading and given it its own specifications. It will be interesting to see if students take this option.
It was a fun semester working with about 25 students in CS315 Game Programming. Last Fall, the course was online and asynchronous, and it was good to be back in the lab again. One particularly fond memory was at the end of the first project, when I had the students rank each other's projects for a Student Choice award. It was fun to watch them engage with each other's work, an when the votes were tallied, it came out to a tie. We had the two tied students come to the front of the room for an electric game of Rock Paper Scissors to see who would win. That's the kind of serious educational business that you just cannot replicate online.
On Tuesday, the students took their final exam, in which they wrote project reflections that featured three "What went well" entries and three "What went poorly" entries. I shall move forward in the same spirit.
Godot Engine. This was the second time I have taught CS315 using Godot Engine, and I am even more convinced now that it is a great choice for this class. Students can see how ideas they have learned in prerequisite courses manifest when writing games in GDScript, and the few students who have already completed the capstone can similar bring those lessons to bear. We only pursued 2D games, which may be considered a mild weakness if you consider 3D to be essential, but frankly, I don't. I am confident that an earnest student could easily learn the 3D-specific tools and techniques with the 2D fundamentals they learned this semester. The quality of some of the games the students made was really impressive considering their experience level and time constraints.
Personal Progress Reports. I have already written about the frustrations I had in the first iteration of the final project with respect to my traditional self- and peer-evaluation format. I switched to the Personal Progress Report format that I learned from Jason Wiser, in which students report on what they planned to do, what they accomplished, whom they helped and who helped them, and external resources used. I required this to be a multimodal document, including at least screenshots. The students completed three of these weekly reports. Many students struggled with the format initially, particularly the use of multimodal composition: several students simply dropped in images with no explanation, no context, no obvious connection to the narrative. Through repeated feedback, I was able to get pretty much everybody on board. Reading these took more time than expected, but it was a good glimpse into students' behavior. I also was able to clarify any doubts I had about teamwork and consistency of contribution. The few students who referenced Personal Progress Reports in their final exam reflections acknowledged that it was a bit of paperwork to do, but that they also enjoyed having to think about what actually happened during the week. Overall, then, I think it was a worthwhile change, and now I'm wondering if something like this should also be backported into CS222, where I often struggle to get an honest glimpse into what's happening on teams. Note, however, that I did not explore the idea of allocating a fixed amount of points across team members for grading purposes; I have still been unable to dig up any good research here.
Short Intro, Long Final Project. We started with just a few weeks of fundamentals, expressed in three projects and a one-week midsemester ex(j)am. I cut a week here from last year's plan, removing a project that dealt with 3D development. As usual, I gave the students a lot of freedom in designing their final project. Since the course doesn't really cover "game design" but only technical issues, students could pursue anything that would be reasonably in scope. Generally speaking, I was impressed by what the teams put together. I am eager to see if the quality of projects continue to improve as more students conceive of CS215 Game Design and CS315 Game Programming as a logical sequence.
Semi-Public Showcase. To celebrate the students' good work, we agreed to hold a semi-public showcase of their projects. I sent invitations via email to every CS faculty member and major, and I also invited a few other faculty friends. I bought a bunch of cookies, and my son baked two excellent homemade batches of cookies and brought them, too. We got the college social media person to come and take pictures. In the end, though, we only had about about five people from outside the class attend, including zero students. Now, I understand that everyone is busy and students in particular are inundated with so many invitations that they ignore basically all announcements. That said, I couldn't help but feel disappointed, in part because I had spent so many hours planning and in part because so few people were able to play these games, get inspired, and congratulate their makers. It is possible that, in a future year, we could do something more like the CS120 Art Show, having the games someplace public to be enjoyed. I'm not sure I have the endurance for planning that kind of event, though, so I'm really not sure right now where the next logical step is here.
Checklist Grading and Design Quality. Checklist-based grading worked well for the technical matters that are expressed in the syllabus, but it's really not enough to help students think about iterative design. What to do with a team that makes a terrible design decision, which I give feedback on each iteration, and which they never change, to the detriment of their project? As mentioned above, game design is not actually in the learning objectives, but it sure seems like a good opportunity to reinforce some ideas of user-centered design. At the end of the second iteration, I decided to have them try the five-point rubric I talked about in a previous post: clarity, innovation, immersion, flow, and fiero. I explained to them that I didn't quite agree with that taxonomy, but that the goal was to just get talking about design issues. Indeed, teams formed up into clumps and had useful conversations that were then referenced by a few students in their final reflections. It would be easy enough to bring this more formally into the course with just a bit more structure and scaffolding.
The 3D Blind Spot. My supposition is that a student who wanted to learn Godot Engine's 3D features could just do it, and this is based on the same kind of belief as my idea that a student could learn Unity or Unreal Engine based on their experience in the class. However, I have no real data about that. Nobody in the class seemed to mind skipping over 3D. Indeed, students could have done 3D games as their final projects, but nobody seemed to want to. Instead, games had the look and feel of either retro Flash games or 2D indie games. I do wonder, though, as my department is looking at more strategic collaborations with the Animation department, how exactly a student from this course would be able to jump into a collaboration with a 3D artist. Maybe we should try to set this up in a jam experience to see what happens.
Of course, we had all the normal pros and cons of a semester, with some students really excelling, some falling behind, some showing up every day and asking good questions, some refusing to show their faces. That's all pretty normal. Overall, this course was a great joy to teach this semester, and I had a wonderful time working with the students on their projects. I'm hopeful that they are inspired to carry on, join some jams, and keep sharing stuff with the community.
A few weeks ago, I had an amazing session of CS222. When I got to my office, I wrote down three key phrases on my whiteboard with the intention of writing a blog post about the class. When two weeks or so had passed and I made time to write the story, I could not reconstruct the event from those phrases. Drat! Now, instead of sharing a story of an amazing CS222 class, I'm instead going to share with you one of the great frustrations of last week: the first iteration of the Game Programming (CS315) Final Projects.
First I need to back up a little. I am teaching three different courses this semester, and each one has a similar structure: the first half of the class introduces concepts, and the second half of the class is a deep dive into a project that manifests these concepts. As I have written about before, I am exploring ideas of labor logs, that students earn credit simply by spending time on their work. In CS222 Advanced Programming, students are maintaining project labor logs, where they summarize each week what they did and how many hours they invested. In CS215 Game Design, students are independently maintaining project design logs and submitting weekly labor statements. These labor statements are inclusive of all course effort, including research, attending class, and helping each other, in addition of course to active labor on the final project. One of the reasons for the different policies is that CS222 involves group projects, and their labor logs are kept in a group document; my hope here is that this would give positive peer pressure and accountability. In CS215, students are working on their own projects, so I'm just asking them to tell me how many hours they spent on the class, in hopes that they will reflect on how this impacts their commitment and experience.
As we started the final project in CS315 Game Programming, I had not made any solid decisions about how to ask these students to track or report their hours. This class has a mix of individual and group projects. In fact, I had written on my planner to engage the whole class in a discussion of how they wanted to track hours, but I kept forgetting to check my planner, so we didn't have that discussion until about halfway through the first iteration.
I showed the Game Programming students how I was managing the other two courses, and I explained my rationale. I was particularly clear about my disappointment with last year's game programming class, when in the final iteration, so many promising projects failed because teams put off their work until the day before the deadline. That is, I tried to impress on them the difference between constant forward progress in an iterative process and submitting work by a deadline.
I let them huddle up and talk for ten minutes or so with the charge of coming up with a fair process for us. The responses were interesting. Very few—maybe none—were in favor of following the practice I set up for the other classes. This was not surprising, and their rationale was a mixture of explicitly not wanting to have to track hours and, unless I'm mistaken, implicitly not wanting to commit or be held accountable to commitments. Several students brought up techniques that they are using in their capstone courses. For example, one group was in favor of following the path of a particular capstone instructor, who has students keep meeting minutes in the repository. I pointed out that this was interesting but didn't solve the same problem, which they acknowledged. Another group responded that they did not want to follow what they had seen in the capstone because they did not see it as being necessary for the task at hand.
One response resonated most with me: a group pointed out that our environment was significantly different than last Fall's asynchronous online class. Because we are meeting together and having in-person showcases of our progress, we can simply use positive peer pressure to ensure that each group is meeting their commitments: no logging, no reports, simply relying on the human success mode of being proud of accomplishments and being good citizens. I decided to take this approach, at least for the first iteration.
Well, was I disappointed when I graded their submissions late last week. Looking at the git commit histories, about half of the teams made no or essentially no progress until the day before the deadline. Keep in mind that this is my course in which we use checklist-based specifications grading, so in theory, students can know the grade of their work at the time they submit it. Indeed, everyone (or practically everyone) who started work the day before the deadline filled out the checklist and claimed to earn an A. My response to them was that they had failed to maintain the contract of the course, so they were ineligible for checklist-based grading and had to be graded ad hoc instead. I still feel badly about the disconnection between checklist-based grading and this ad hoc management, but it seems like it was the right thing to do.
Unfortunately, my disappointment does not stop here. I also required the students to complete self- and peer-evaluations, following the same format I use in CS222. Indeed, I gave the same policy statement as well, that a student's iteration grade would be capped by the sum of the means of their self- and peer-evaluations. I was shocked when I sat down to my spreadsheet and ran the pivot tables to find these sums of medians: practically everyone had full credit. There were maybe two students with a few percent less than full points, but not enough to be meaningful—and not enough to account for the stories that at least one team told me about the communication problems they had among members. Putting this together with the previous disappointment is a double whammy: not only did half the teams not start working until the day before the deadline, they also claimed that each of them was perfectly fulfilling their obligations for the previous three weeks.
They lied. There is no other way to put it.
I know that sometimes, under pressure and facing temptation, anybody can make a mistake of judgement. This, however, was a systemic misrepresentation of truth, and now I need to decide what to do about it. I will need to talk to them about it on Tuesday, though I'm still not entirely sure how I want to approach it. What difference will a chastisement from an authority figure make if one does not show them the respect of telling the truth? Or is Dad Mode really what these students need?
I also need to decide if I want to change the policy for Iteration 2. If they do not learn from their mistakes, then the whole enterprise is a failure. Would making them actually track their hours force them to face their deceit or delusion? Or would it only push them toward digging themselves deeper graves?
There is a positive story that came out of this, however. A few hours after I posted grades, I got an email from a student, and the email opened with a statement that they were not looking to change their grade. Instead, the student humbly acknowledged that they had screwed up, that they got the grade they deserved, and that they would not repeat this error. This acknowledgement was, in a way, an attempt to show gratitude for holding them accountable to standards. Keep holding on to hope, friends.
The first of my summer course revisions is CS315: Game Programming. The online course plan can be found in the usual place. This was a fairly easy revision since I have recently revised a manuscript dedicated to the course. The paper has been accepted for publication in the Journal of Computing Sciences in Colleges, and I will be presenting it at the annual meeting of the Midwest chapter. Writing and revising that paper required me to carefully read through notes about the planning and execution of the course as well as my reflections, which are publicly available in a blog post. The paper includes some thoughts about what might be done in the future, so planning the course became a matter of following my own advice.
I decided to keep Godot Engine for Fall even though we will be back in the lab. I discussed this choice with my Spring game production studio course, which included students who had taken CS315 in Fall 2019 using UE4 and in Fall 2020 using Godot Engine. They unanimously agreed that using Godot Engine was a better choice and that focusing on 2D games in particular helped control project scope.
I kept the overall course structure from Fall 2021, including the scaffolding of projects. The instructions are all based on Godot 3.x, and if 4.0 is released before the semester starts, I will have to think about which version to use in class. I removed the one-week 3D project from the course plan. Of all the work from Fall 2020, this was the least interesting. It will be interesting to see if students really care about this or not. It's not clear to me if 3D has the mystique to them that it once did for undergraduates. It's so common now that perhaps they will understand that it's fundamentally just like 2D, but with more asset work required and therefore slower cycle times.
Because we will be meeting in person, I have removed the requirement for web builds. I am a little sad to see this go because it's an interesting process, but it is no longer required when we can instead go back to showcase-style presentations. I can always add continuous integration as an option for the final project, which will be the first place where public dissemination becomes desirable.
I plan to have a one-week midsemester jam, potentially to be called the "Midsemester (Ex|J)am." This was a fun break for the students last year, to have a week to just make something that shows off specific technical skills. It should provide a useful transition to the final project. The descriptions of both of these are commented out of the course site right now since they will require a bit more revision. It does not seem worth that revision until I have a chance to get to know this crop of students, and Projects 0–2 should nicely fill up the first six weeks of the semester.
The schedule for the weeks will have students presenting their work on Tuesdays, and so in my plans, I have sketched in a few short tutorials for Thursdays. Some of the things I had in ad hoc videos last Fall I can do in class meetings, with the benefit of live Q&A. Without quizzes or attendance, there's a chance that students may decide to skip these meetings and just work by themselves. It certainly makes more sense for me to reuse the lengthy project descriptions I wrote for my asynchronous online semester rather than to write up shorter ones that are supplemented with ephemeral live demos. I intend to be open with the students here, to get a sense of what excites them and trying to use our time together most fruitfully, whether that means tutorials, Q&A, or philosophical discussions about the role of higher education in 21st century America.
One major addition I intend to make for the final project is the inclusion of development logs. I used design logs in Game Design in Fall 2020, where students had to log their efforts to earn a grade each week. Here, I would like to do something similar to make clear a distinction that some students seem to miss: the difference between "something is due in two weeks" vs "you are expected to make continuous progress on this over the next two weeks." It was the post-Thanksgiving, final iteration of the Fall 2020 game programming projects where I really became aware of this confusion. I hope that using development logs will help me get out in front of the problem. If nothing else, it will prompt me to have a discussion about this with the students, and perhaps I can even learn more about their experience through that.
The university is planning on having all our courses back in-person in Fall, and this means that my CS315 Game Programming class will once again be able to meet in the lab. Last year, I switched from UE4 to Godot Engine in large part because of the lack of access to lab machines: every student has a personal machine that can run Godot Engine, but many cannot run UE4. This Fall, I could just go back to UE4, but I have spent a lot of time with Godot Engine, and it has some distinct advantages for teaching.
I shared with my CS490 studio, via Slack, that I was thinking about the tradeoffs between these two engines. There are students in the virtual studio who have taken CS315 in both forms: some took it in Fall 2019 with UE4, and some in Fall 2020 with Godot Engine. Additionally, several of these people have since started learning different engines, such as some students who learned UE4 in class now using Godot Engine, and some who learned Godot Engine now using Unity. One student provided an excellent overview of his experience, but sadly, no one else has engaged in my request for comments.
Last Friday was the end of the third sprint for CS490, and we all met on Zoom so that the three teams could show their increments to each other. It so happens that during the discussion, a few items came up that are salient to the issue of engine selection. One team, which is using Godot Engine, had underestimated the time it took to write an AI for some of their game characters. In another project, also using Godot Engine, a student pointed out that the original compositions seemed to blend together well when switching between them. This was a happy accident and not by the design of the music nor its playback.
So, if UE4 has all this cool stuff, why not just use it? As I discussed with my students during the meeting, a lot of this great engine content is beyond what a student can get to in a one-semester introduction to game programming. They would be useful things to know now, having wrapped up the third iteration of a project and having already studied the basics, but I've never seen a student in CS315 get as far as really needing advanced audio or behavior trees. For behavior trees in particular, when I have offered some A-level credit for exploring them, it's always been a bit disappointing because it's the tail wagging the dog: students make a simple behavior tree to show they can rather than use them to solve an actual problem they have.
Godot Engine, by contrast, makes it really easy to stand up a prototype. Additionally, as one of my students put it, 3D in Godot Engine is fine, but 2D in Unreal Engine is a pain. A student also suggested that most students in CS315 would rather make 2D games than 3D anyway, especially given the extra complexities inherent in 3D game production.
I am not yet putting together formal course plans for Fall—that will came later. For now, I am undecided but leaning toward sticking with Godot Engine for another year. I have been approved to run another full-year immersive learning collaboration with Minnetrista: we will make prototypes in the Fall and produce a game in the Spring studio. Although CS315 is not part of this sequence, many CS majors come into CS490 via CS315. This will be a factor in my decision. I also have two outstanding grant proposals that would impact how I want to devote my own attention next academic year: it's much easier to teach an engine I am currently using than one I am not.
This is the first of my traditional reflection blog posts, in which I share some of the successes, failures, and mysteries of teaching. The good news is that all three my courses went surprisingly well, given the unprecedented constraints that we were under.
As I shared in my planning post, the game programming class switched to Godot Engine this semester. This ended up working quite well. While I enjoy working with Unreal Engine 4, it is fiddly in ways that trip up novices, and I need to seriously consider whether Godot Engine is a good permanent replacement for UE4 in CS315.
The students had the usual confusion around intellectual property licensing, and it's not clear to me how much I need to get out in front of it vs. respond to their misconceptions. I fear that time I spent trying to explain this up-front would be wasted, as so many students would still get it wrong that I would repeat it all anyway.
The checklist approach to grading projects continues to work well, and I think that this clarity of expectations was especially useful in the absence of shared lab time. I have gotten better at documenting these, which also helps. One question that remains in the back of my mind is the distinction that Nilson makes between more hurdles and higher hurdles. She suggests one or the other, while I continue to do both. I wonder if there is an alternative model for game programming projects that would take the "more hurdles" approach, although it also seems to me that then you're just doing regular points-based grading where only a subset of points is required.
In the manifestation this semester, I played some more with the idea of "stars" at the A-level, so that students could see some of the variety of actions that could be rewarded. Unfortunately, most students clearly settled on the ones that required the least effort, which makes the endeavor seem like a presentation of false choices. Next time around, I could spend some time reconsidering this, although I do not think it's that bad to leave it as it is. A few students were able to be rewarded for doing some unconventional things, and that is the main point after all.
The heartbreaking moment this semester came when in grading the final iteration of the final project. This spanned the two weeks after Thanksgiving break, when the university shifted entirely to asynchronous online instruction. In my mind, this should have been the great push, where students bring together everything they learned and make some really interesting projects. What actually happened is that almost everyone made zero progress for most of those two weeks. Almost all the activity on the projects took place in the 72 hours before the deadline. Some students basically lied about their efforts, though the git logs gave away their true patterns, while others humbly acknowledged not having done adequate work: both groups earned the same grades, but I am proud of the latter while bitter about the former.
I sympathize with the students' bad situation. I have some confidential messages that provide more evidence for my suspicion that a lot of faculty did not handle this semester well. Some students will always complain about end-of-semester workload, but when essentially all of them do, one has to wonder if there is a systemic problem. My defense to my students was that we had the same pattern of expected behavior all semester, and indeed, some students have thanked me for that; however, this doesn't change the other stressors in their lives, or the potential of uneven distribution of labor expected from other faculty who appear not to consider the consequences.
Despite the expectation that my students maintain steady progress for each two-week iteration, I did not create any incentives for this behavior. It makes me wonder if, next time, I should incorporate something like labor logs, where students are graded based on self-reported time-on-task. I tried this approach in my game design class, which I will write about later in this series. This would give them positive feedback in the form of grades for those weeks where there is no deliverable. Another alternative would be to have weekly check-ins or status reports rather than biweekly deliverables. I will need to consider the costs and benefits of these approaches next time I teach the course, which I hope to be in Fall 2021.
Another puzzling factor in this class, and in both my other classes, was that many students stopped turning in material and yet did not withdraw. My intuition is that there were more of these students this semester than in the past. That is, there are always some, but I believe I had a higher proportion of them this semester. I wonder if this is because of the online asynchronous nature, if students deluded themselves into believing they could catch up, when all the evidence was that they did not have the conscientiousness to keep up. I will be interested to talk to my colleagues about this, including those outside my department and my university.
In summary then, it was overall a great experience in teaching game programming, and some of my students made some really great little games, for what they are. My favorite is the one that used the sprites from Tyrian, which ratcheted up the nostalgia for me. The experience was slightly soured by disappointments in the final iteration of the final project, but at least I have some ideas of how to mitigate this next time.
Note that the students are completing the final exams this week, so the course is not yet completely over. I will post an update if there's anything in the final exam responses that make me rethink anything I have shared here. Thanks for reading!
I just finished reading the final exam responses from the CS315 Game Programming class that I wrote about yesterday. One of the questions invited the students to write about their understanding of depots, changelists, and workspace mapping in Perforce Helix. A surprisingly large minority of responses equated version control to backing-up files. That is, students said that the main purpose of version control was, essentially, to have a back-up of your project in case you need it. This strikes me as a particularly naïve perspective. In every case, I provided written feedback encouraging the student to think about the project in the depot as being the real one whereas anything that is checked out is just a shadow of that.
I wanted to mention this here on the blog in part because I am getting back into teaching CS222 in the Spring. I have had some hallway conversations with the professor who is assigned to teach the other section, and we've been talking about how to improve students' understanding of version control as a critical piece of contemporary software development workflow. One of my experiences this semester that inspired me to do this was the realization—confirmed in an honest conversation—that the students in my upper-division HCI class were terrified of pull requests. Their experience in CS222 had inadvertantly taught them to avoid contemporary best practices of version control rather than to depend on them. This could be related to the misunderstanding that I saw in these CS315 exams, where students see version control as an awkward back-up system rather than, well, version control.
My students are currently taking their final exam, so this seems like a good time to start my end-of-semester blog post about CS315 Game Programming. This was my third Fall semester teaching this upper-level elective course using Unreal Engine 4. The semester ended up consisting of four "mini-projects", each about two weeks, and one six-week final project, which was completed in two iterations. By and large, I am happy with how the semester went: students learned how to work with some contemporary tools, including Perforce Helix for centralized version control, and they made interesting final projects. What I want to document here are some of the struggles, because it will do me more good when planning next year's class than focusing on the successes.
It turns out that almost all of my frustrations from the semester stem from my decision to use Specifications Grading again. For people who are not familiar, you can easily hop over to the course plan's projects page to see what they are. Briefly, I laid out all the criteria for which student work would be evaluated ahead of time, and like last year, I asked students to submit self-evaluations in which they graded their own work.
This leads quickly into the first problem: students did not seem to understand how to use checklists. It feels so strange to even type that, but it's true. As part of their submission, the students had to complete a checklist, and then based on what was satisfied, they could know—and had to say—what their grade would be. However, more often than not, I would read through the student's submission and have to point out that they didn't actually satisfy some criteria. I designed a little leniency for students who legitimately did not understand a criterion or two, but what I didn't expect was that several students made the same mistakes again and again and again. I forced the students to rotate partners during the Mini-Projects, thinking that this would ensure that mistakes would be caught by the partner; instead, what I saw was that the misunderstanding (not the understanding!) spread to new partners.
I suspect that a major reason for the checklist problem is that students are so deeply brainwashed into the "turn this in and hope for points" model that they cannot conceive of an alternative. Certainly, in my years of teaching, I've had plenty of push-back on unconventional things I do. (I continue to do unconventional things, partially because I want students to learn to question conventions.) I can work on clarifying the language around the specifications themselves of course, but I feel like this is treating a symptom rather than a cause.
There is one place where my instantiation of specifications grading contrasts, as I recall, against the presentation in Nilson's well-known work. She describes making a choice between more hurdles and higher hurdles, but my version of specifications grading is both more hurdles and higher hurdles: students have to do more and better work to earn higher grades. This is sensible to me, but I wanted to mention it here because it is a lever that I could pull in an experimental assignment or section.
Another problem I encountered with the specifications grading this semester was that a minority of students were able to follow the specifications I provided to earn relatively high marks but, in my professional opinion, without really meeting the learning objectives. For example, I had a B-level criterion which was, basically, that the project should have all the parts of a conventional video game: a title screen, gameplay, an ending, and the ability to play again. An alarming number of teams did not handle mouse input controls properly, so that once you click in the game, the mouse is captured and the cursor made invisible. This means that technically you can still navigate a UI menu, but without being able to see the cursor, it's awfully difficult. Their conventional solution seemed to be to use Ctrl-F1 to release the cursor so they could see it again: an editor kludge for a runtime problem. Did such teams satisfy the criterion? Well, yes, but also no. I liberally allowed it, leaving notes in my review that they should fix this, which almost nobody did. I could, of course, add text to the already-wordy criterion to say "If you are developing for a Desktop application, and you are using mouse navigation, make sure etc." That's just one special case of a particular environment, though. What I think I'm really running in to is the problem if specifications grading in the face of creative, wide-open projects.
Several students took my examples wholesale, brought them into their projects, and then submitted them as satisfying the relevant criteria. For example, I showed C++ code to count how many shots a character fired; student teams put this into their game and then checked the box saying that they included C++ code. Technically yes, but without any semblance of understanding. Another student took my dynamic material instance example wholesale and put it into his final project. Again, no indication of understanding the pieces, just copying and pasting it into his project and claiming that he included dynamic material instances. Yes, they're in there; no, there's no evidence of understanding. Some of this could, in theory, be cleaned up by changing the specifications, but then it gets into the same kind of problem as measuring productivity in programming. Exactly how different from my example does a student's work have to be to demonstrate that they understand the comments? "Exactly" is the key word here if the specifications are going to be objective.
I'm left with this sinking feeling that specifications grading are not worth the effort and that I should return to my tried and true minority opinion on grading: use triage grading for everything. This allows me freedom to say something like this: "Use dynamic material instances in your project in a way that shows you understand them." Then, I can fall back on saying that a student's work either clearly shows this (3/3 points), clearly doesn't (1/3 points), or is somewhere in between (2/3) points. This clear and coarse-grained numeric feedback can be combined with precise and crystal-clear written feedback to show students where they need more work, which appeals to me much more than my grimacing at the student's submitted checklist, then at the student's code, and then saying, "Yeah, I guess."
Back in 2007, my first Computer Science education research paper was published in the proceedings of SIGCSE. The paper provides an overview of how game programming can be used as a motivating context for learning design patterns. One of the examples in the paper, which I have used for years, is the behavior of the player-controlled bombs in the inimitable Every Extend, which can be used to explain the State Design Pattern. In the paper, I argue that the player's pawn can be described by a state machine like the following.
In yesterday's meeting with my CS315 Game Programming students, I was inspired to create a UE4 tutorial video about using state machines for state management. In particular, I wanted to show how an enumeration can be used to define the possible states, and then behavior can be switched depending on the current state. This is not, of course, the State Design Pattern, but it's a heck of a lot better than an ad hoc collection of Boolean variables. It also follows from Michael Allar's Gamemakin' UE4 Style Guide that is mandatory for my students, which says:
Do not use booleans to represent complex and/or dependent states. This makes state adding and removing complex and no longer easily readable. Use an enumeration instead.
To prepare for the video, I have spent a few hours the last two days building a simple playable framework similar to Every Extend. Over a decade ago, I did a similar exercise in Java to produce EEClone. This time, I did it in UE4. (A bit of an aside: Every Extend is like my kata for new game programming frameworks. I use the essential gameplay of it as a case study to understand how game engine functions. Of course, then, when I first started learning UE4, I tried making an Every Extend clone, but it went quite poorly. After a few years of working with UE4, it's become a lot easier to get the game up and running!)
With my gameplay framework in place, I started working on the diagram shown above to use in my series of game programming tutorial videos. However, as I looked at the diagram, I realized I had a rather fundamental problem: these are not the states of the player bomb. It's true that these are, abstractly, states of the game, but they are not the states of a single stateful object. Instead, what I had programmed was leading into the following state model:
That is, a player bomb is created, which has it play a spawning animation. When that is completed, then the player becomes vulnerable to collisions but can also press Space to explode. However, unlike in the initial analysis, there are no other states after this. If the player presses Space or is hit by an obstacle, the player's bomb is destroyed: that actor—that object—no longer exists. It is the game mode that either counts down to respawn or waits until a chain of explosions is complete and then counts down to respawn. These states are not part of the player pawn.
This observation shoots a hole in my plan to make a video using the Every Extend example. I still want to make a video about state analysis, but I need to find a better example—one where a single actor's state is interesting enough to be modeled but simple enough to fit into a tutorial video. Please feel free to share your suggestions in the comments.
Before the Fall semester fades away into memory, I wanted to capture a few of my thoughts about its CS315 Game Programming course. Interested readers can examine the course plan as well as my summertime blog post describing the plan's construction. What I will do here is start by addressing the major points that I brought up in the other post.
More Small, Structured Projects
This went well, having four two-week projects that increased in technical complexity. I prevented people from choosing the same partners for the first three, which I think was also successful in both building community and sharing knowledge. Curiously, it was not entirely successful in helping some students avoid unproductive team members in the final project, but at that point, the onus was on them, and so it was their learning opportunity.
Some students wanted even more fine-grained assignments and closed-form assignments. An example might be to take a skeletal project and add a new actor or component to produce a desired behavior. This suggestion seemed to have influence with students who had middling performance, and this makes it worth considering. Those students who are still on the road to learning how to learn may need more fine-grained scaffolding. On the other hand, I was always available to help them move forward on the projects, and so it's not completely obvious that they simply weren't being responsible with their time.
Specifications Grading
This was, by and large, a great success. The students knew what they had to do to earn the grade they wanted, and there were very few questions about ambiguous criteria. I did end up giving each student a "Quick Save" which allowed them to fix one criterion and resubmit, addressing the concerns I expressed in my September post. I regret that I didn't think to call them "Save Points", since that's what they do.
A good addition to the specifications grading approach was to require a project report in which students told me what grade they earned. This made my evaluation into a verification rather than a bug-hunt. I like how this forced the students to go back and look at what they had done and follow a checklist of critical ideas. Some students had what I consider unreasonable trouble with Markdown, but at least they learned a new skill along the way. I am thinking of incorporating specifications grading into my HCI class in the Spring, and I will definitely take this approach to fostering students' reflections on their work.
During one of our class meetings after the first iteration of the final project, I asked the students what they thought about the specifications of the final project, and in particular whether they would make any additions to A- or B-level criteria. The one A-level that came up was profiling—a great idea that I had not considered at all. One of the teams used a Post-Processing Volume, and we agreed that this could be a good B-level criteria. A more insightful and critical student pointed out that it would be useful to distinguish between "bare use" and "proficient use." A good example here is Behavior Trees. The grade criteria was simply to "use Behavior Trees," but this could be done with a one-node tree that is just awkwardly recreating what could be done in a simple call to UE4's AI Move To. This is an area that I can try to clean up in the future.
Another area that was surprising to me was how many students had trouble with the criteria to document the licenses under which they used third-party assets. I know this is something that I bring up when I teach CS222, although few people there use third-party assets. In this class, everybody used some textures, models, sound effects, or music that was created by somebody else. I'm not sure if I need to have more documentation about this, plan an early-semester lecture about it, or give more examples. Very few students did it right the first time, and many more made mistakes more than once, which is a bit mind-boggling to me. That is to say, I do not understand which part of the process they do not understand.
Perforce Helix
A lot of students struggled with Perforce Helix, as I expected. However, it did serve its pedagogic and practical purposes: the students got to see a different model of version control from GitHub, and it ensured that all the projects were in one place. I have little sympathy for the one or two students who still could not use P4V at the end of the semester: at some point, they must have given up trying, because everyone else got used to it. I need to review my video tutorials on this topic before next Fall, because I think I may be able to tighten them or get more targeted at the kinds of problems students had. Simple Achievements
I was surprised that a very small number of students pursued the achievements I set before them. I really thought that there was low-hanging fruit here for course credit: learn basically anything about UE4, talk about it for five minutes in class, get credit. There certainly were interesting things learned that I had not planned on, including post-processing volumes and custom animations via bone rotation in Blend Spaces. Perhaps this extra step was not necessary, since when teams demonstrated their work, their classmates would always ask at something interesting, "How did you do that?"
I don't have much to say about the Decorum and Diversity portions of my original planning essay, so I will move on to...
Conclusions
I think it was a great semester. I was really impressed with the students' projects. I expect to teach the call again next Fall, and I will make some minor modifications. If we are able to replace the lab machines with UE4-capable rigs, that would be even better, since then we could do more "lab" exercises. I also learned a lot this semester, manifest in my new video tutorials and my Heroic Uncertainty National Game Design Month project. My department has been on this strange pattern of offering HCI every semester (including summers!) when it used to be a once a year course, and as much as I like that course, this one is much more where my mind is these days. Perhaps I need to talk to the department about trying a Spring section of this course next year.
My game programming class has started their final project, which is to be completed in two iterations. My intention was that they would deploy what they learned in the Advanced Programming prerequisite to develop good user stories and plans, but many of them asked specifically for some kind of lesson or activity to get a better understanding of agile methods in general and Scrum in particular. I spent some time yesterday searching for best practices here, but none of the canned exercises I could find satisfied my goals. I wanted something that would teach the basic structure of Scrum and include the use of a task tracking board. I ended up making some modifications to ideas I found online and deployed them in class today.
To start, I had them play this online Battleship game that is used by some trainers to get people—especially management—to understand the fundamentals of agile approaches. In the game, you can choose how many shots to take at a time before you get feedback about whether or not they hit. I had half the class work in a 40-shot increment and the other half work in 10 shots. As one would predict, the half with shorter increments had much higher scores than the others. Not only that, they were also done faster: the 40-shot students were among the last done because they had to be so judicious with their resources. I asked the class the difference between the two groups, and they correctly identified feedback as the key. I quickly explained how this is a decision we make when choosing, say, a waterfall-based or an agile methodology, and then we moved on.
I had them count off by fours to make four teams who don't normally work together, and I put each one in one of the "alleys" between tables in RB355. I drew a template task board, with columns for Story, Not Yet Started, In Progress, and Done, and had they copy these to their boards. I gave an overview of what would happen: a three-minute planning meeting that would generate tasks on sticky notes, followed by three three-minute "days" of a Sprint, followed by a review and retrospective meeting, and then another series just like that. They had seen some aspects of this before, so we were able to get right to work, and at this point, they were very curious about the activity we were going to do. I handed out their user stories, which are transcribed below, and after giving them about two minutes to read the sheet, we got into the planning meetings.
Product Backlog
Fleet
Build a fleet of at least four paper airplanes that satisfy the following criteria:
A plane must be folded from 8.5”x5.5” paper only. No additional material may be added and no extra material may be removed.
The nose of the plane must be blunted to avoid accidental injury.
The plane must have crisp, clean folds.
A plane that satisfies the above conditions must bear all team members’ initials. This marks the plane as part of the fleet. Once initialed, a plane may not be removed from the fleet.\
Branding
Each team has its own name and logo.
The logo must be publicly displayed by the team on their board.
The public display must have each team member’s initials to indicate their approval.
Hangar
I have a hangar that indicates the starting position of my fleet. The hangar satisfies the following conditions.
The hangar covers at least 11”x8.5” of the floor and be at least 1/2” off the ground.
The hangar features the team’s logo.
Flight
Maximize the number of planes from the fleet that can touch the far wall while still in flight.
The hangar is no more than two floor tiles from a wall.
The plane thrower’s feet must not be more than two floor tiles from the wall.
Sliding along the floor to the far wall does not satisfy the flight goal. The plane must strike the wall while still in the air.
A plane that has met the goal is placed next to the hangar. No other planes may be next to the hangar.
Earn points equal to the number of planes that pass the mark minus the number of planes in the fleet that do not.
This is a customization of the "agile paper plane game" that you can find in various places online, such as here. All versions of the game I came across shared the property that each member of the team had to make one fold in planes. I did not like that approach, since that's not generally how (my) agile teams work. They don't have every piece of work move through every person like an assembly line, but instead, they rely on clear communication of both serial and parallel work. I also took inspiration from Lego4Scrum, where the tasks look a lot more like real software development work. I don't have the budget to buy enough Lego kits and have them sent via expedited shipping, so I combined the two ideas.
I provided both whole and half sheets of paper for their use, as well as the sticky notes and, after they realized that some of the sticky notes were not very sticky, some adhesive tape. The students got right to work and stayed within the bounds of what activities belonged in which timeslice. They were clearly and intensely focused, which made it both rewarding when a plane soared straight to the far wall and comical when so many of them flipped and flitted about before nosediving into the ground.
I noticed as they worked that some teams were using 8.5"x11" sheets to fold their airplanes, but I didn't challenge them on it right away. Rather, at the end, I asked them to review the conditions of satisfaction and have a chance to come clean. One team had recognized at the end of the first sprint that they were in violation of the airplane acceptance rules—one of the emergent leaders pointed out that he only read "8.5x..." and assumed the rest would read "11" and not "5.5". They made tasks to deconstruct their hangar and planes and recreate them to satisfy the constraints and did so in the second sprint. Another team had not read the conditions carefully at all and only realized they were in paper size violation when the other team told their story.
I asked each team to share stories about what changed between the two Sprints, and the results were fascinating. One team discovered the value of cross-functional teams. In their first Sprint, they distributed tasks and as a result only had two different plane designs, neither of which were making it across the room; in the second Sprint, they all worked on each piece and made more progress. Another team pointed out that they jumped into the plane-folding task with gusto because they were excited about that, but that their excitement was not actually producing anything that satisfied the conditions. That team added a more explicit research step in the second Sprint, which helped them vary their designs and produce more useful planes. I pointed out that Scrum allows for "spikes" that are used for just such cases. I wish I had taken better notes, because the other two teams also made changes to their process in such a way that highlighted core agile practices and attitudes, but now I cannot remember them. It's like I tell my students regularly: "I will remember this" is the primary fallacy we tell ourselves.
At the end, I asked them to share their insights about what they learned from the exercise that could be applied to their final projects, and the results of this were excellent as well. Topics that were brought up include the importance of making estimates (and in particular, of making them large enough), giving adequate time to planning, articulating tasks in a clear and measurable fashion, and keeping everyone aware of your progress on your tasks.
I was really surprised by how well the exercise went, perhaps in part because it's so unlike the kinds of things I normally do in class. I generally prefer to do and reflect on authentic work rather than small simulations or throwaway work, but here I think it let the students focus on the process rather than the minutiae of programming. I was really impressed by their reflections on the process and its implications for their project. My own professional preparation with agile software development methods allowed me to hear their stories and contextualize them within more general principles and rules.
There were a few things that I would change for next time. It wasn't clear to me that having a "Stories" task on the board mattered at all, or that anyone used it the way I intended. The exercise would have been fine without it. A couple parts of the handout can be clarified. I had two students come up to me with what I would call "pointy" planes and asked if they were "blunt enough." I started telling them to use their eyes to measure bluntness: if it fits in your eyeball, it's too sharp. This was maybe a bit challenging to interpret, so maybe another measurement of bluntness is appropriate. In my description of the hangar, I meant that it should have a volume of at least 11"x8.5"x0.5" and be sitting on the ground, and I should have just said it that way. Instead, I worded it as 1/2" off the ground, which some students interpreted as elevated, like on a chair or table. One team was also unclear about whether it was the front or the back of the hangar that should be within two tiles of the wall, although this was also the team that tried throwing their planes at the near wall instead of the far wall, so I think this was an intentional trying to game the system and not an honest mistake. (We did not come back to the point that this one group tried to interpret the rules in their favor rather than seek confirmation or favor value for the customer.) Almost every team ended up having one person write each other team members' initials on their planes and logo, rather than what I intended, which was that each person inspect the work and then initial it. I can make that more clear.
That's the story of my game programming class today, finished writing just in time to go to my next class of the day. As always, please feel free to leave a comment about your experience with agile training games or ask about the activities of the day. I expect I will reuse the exercise again in a future course.
My game programming students had their first mini-project due yesterday, and I'm happy with the results. The "Angry Whatevers" project is designed to get students familiar with UE4 and Blueprint by making a simple fire-a-projectile game. It's essentially the same introductory project I gave in Fall 2017, when I first switched the course to UE4. However, this year's projects were on average much more interesting and higher quality. I attribute this to two factors: first, I used a specifications grading approach, which made it very clear to students what they had to do to achieve different grades; second, I am more seasoned with UE4 and was able to articulate, teach, and clarify more easily than last year.
Last year, we used GitHub for version control, but this caused several problems for us, the most pressing of which was that a distributed model doesn't work well for the non-mergable binary assets that constitute the bulk of a Blueprint-based UE4 project. In the Spring, I learned and deployed Perforce Helix with my studio course, and I decided to go ahead and use that for this semester's game programming course as well. It was a little choppy to get started as I had become rusty with some of the core configuration. My first pass involved making one depot per team, but I my conceptual model of workspace mapping was wrong.
When I spent several hours trying to rebuild my understanding of Perforce Helix and how to integrate it with the course, I realized that I could get away with one structured depot. This depot was called CS315Depot, and I wrote up instructions and recorded a private YouTube video explaining how to set up the mapping. One of the crucial steps that I had forgotten was not to use the graphical mapping tool in P4V. Instead, we used the text-based mapping specification, so that if I was working on a Project 1 ("P1"), then I would make a workspace called PaulGestwicki_Laptop_P1 and use my UE4 project as the project root. Then, I use a mapping to the depot like this:
What this does is map all the files in my local workspace to the folder P1/PaulGestwicki on the depot. The real magic, then, is that when I want to grade the first project, I can make myself a new workspace, something like PaulGestwicki_Laptop_GradingP1 and map it to the whole P1 directory:
In one action, I can grab all the student projects into my workspace and then batch-grade them, without having to make N trips to the depot or create N different workspaces. Also, I could easily set up one group (CS315) for all the students and give that group write access to the shared depot and read access to my depot of demos. Note that this does mean that students can check out each others' projects, which I am happy to support; it also means that they can accidentally destroy each others' work, but that's why we have version control in the first place.
One of the common errors I saw with several students was that they forgot the last slash in the mapping, which ended up with them dumping their Content, Config, and .uproject files into the root of the P1 directory, prepending their usernames to each file. Some students recognized the problem, and I was able to just obliterate their old files so they could try again. Because it was so easy to recognize the error by glancing at the depot, I was also able to email students and point out where they had unwittingly made a configuration error.
Perhaps this idea will be useful to you, dear reader. In any case, I hope it will be useful to future-me in case I forget how to do such a thing in the future! I offer my public gratitude to Perforce for their academic licensing that allows my students to use their software, and of course, gratitude to Epic for the very generous licensing of Unreal Engine.
UPDATE: When I went to adjust some grades the other morning, half the projects were gone. I looked at the changelog history, and I found one that looked suspicious. It touched every file in the depot, which makes me think they had a mapping wrong. What was really strange to me was how this affected the tools themselves: when I viewed the project in p4admin, I could see all the files I expected; when I viewed in p4v, half the student folders were gone and could not be mapped. I am still quite perplexed about what could cause this. However, the good news is that after I backed out that changelog, everything ended up back where it was supposed to be. Version control for the win!
Several times during the semester, I have wanted to share thoughts about my game programming course here, but time and other obligations have gotten the best of me. Now that the semester is wrapping up, I am making the time to focus on the last few weeks of the course and the roller-coaster of emotion it has been for me.
Background
To set up the story, it had been two or three years since I was able to teach my game programming course (CS315), and I revised the course design over the summer to center on Unreal Engine 4 and my Collaboration Station project. The students started the semester with a two-week project designed to get them into the UE4 basics. The project was called Angry Whatevers, and students had to create a simple playable game that involved physics and parabolic motion. I started a new playlist on YouTube to support the class during this period, starting with this pair of videos that give an appropriate framework using UE4's 3D primitives and Paper2D sprite system, respectively:
Because I was recording these videos on Windows, I had to learn a new video production pipeline as well. My first attempt used OBS Studio for the screen capture and OpenShot for video editing, in part since OpenShot is what I've been using on Linux for some time. I realized after posting the first edit of the first video that there were significant editing glitches. When I brought this up in class, the students expressed disdain at OpenShot and strongly encouraged that I learn Blender. I didn't even know that Blender had video editing support! It took some time to learn Blender's video editing system, but I ended up becoming fairly productive at doing simple editing with it.
In any case, all the students who did not withdraw from the class did fine in their Angry Whatevers project. Several students withdrew during this period, but none talked to me about their experience, so I have nothing but conjecture about causality here. We had a class meeting after Angry Whatevers during which I gave them the option to either move forward with another tutorial-style assignment or to move into forming teams for a bigger final project. They chose the latter. I prepared and delivered a presentation about Collaboration Station, showing the three minigames that I had not yet implemented, and I encouraged students to choose one of these as their semester project; however, I also gave them the freedom to propose something different, with the cautionary explanation that game design is hard. As you can probably predict, dear reader, all of the teams decided to pursue their own projects.
I had the teams write their pitches in Tim Ryan's game concept format, which I had to approve before they could move forward, and they organized their work as user stories expressed in Trello. More specifically, they used the "front" of each Trello card for the name of a user story in Mike Cohn's format, and they used the "back" to track the conditions of satisfaction, hour estimate, and team member assignment. My intention was to monitor teams' progress via their changes on Trello, but I quickly had to drop this plan as out of scope: I was overwhelmed this semester with the number of different projects I was supervising, and I could not afford this level of oversight. I was surprised when, after the first iteration, the students elected to continue using Trello the way I required for the first iteration. They found value in it, and so they required themselves to show their progress via Trello at each end-of-iteration presentation.
I'll note here that toward the end of the second iteration, the students expressed interest in learning more about the C++ side of UE4. Some students had thought about shifting some of their existing work from Blueprints to C++, but they were unsure how to dig in. I recorded a series of five videos that walked through an example designed for students with no C++ experience, although these were not ready until we were into the third iteration. No students ended up using C++, and avoiding such a thing in their third iteration is wise. However, looking at the video series, the first video has 49 views, the second has only 28, the third has 13, the fourth has 37, and the fifth has 29. If you can make any sense out of that, let me know. For the time being, I interpret that as meaning that students by and large didn't watch the series for their own edification. It took roughly 10 hours to make that video series, so hopefully I can recoup the investment either in a future class or in the long tail of YouTube.
The tumultuous end of the third iteration
I know that was a heck of a wind-up, but now we can get into the story of the last few weeks. At the end of the first iteration, I was pretty lenient in what I considered "satisfactory" since the teams were still really getting their feet wet with UE4. The second iteration, some teams did not have working core gameplay, and so I told them the work was unsatisfactory. What surprised me was that at the end of the third iteration, a majority of the projects had fundamental problems such as broken core gameplay, no audio or visual feedback, or no attention paid to user experience. This made me go look at their version control logs, and these pointed to the idea that many had simply not put in adequate effort. Before grading these final iterations, I went back to check the course description to see how specifically I had defined "satisfactory." To my great embarrassment, I didn't. I honestly remembered having written it up, but either I wrote it somewhere else and misplaced it, or I simply thought about it and never typed it up. In the absence of a rubric, the best I could do was fall back on the clear dictum: "Note that, because this is a three credit-hour course, you should expect to invest nine hours of attention to it per week." Clearly, some teams did not do this, with gaps of 15-19 days between commits.
There's another important piece to understanding my emotional reaction to the third iteration. The teams' presentations were split over two 75-minute class periods, with the day of a team's presentation being essentially random. It turns out that the Tuesday group was the disheartening one: no team presented something that I would call, in any way, "done." This led to an interesting conversation on Slack with a few students who were willing to speak up, though. I expressed my distress that they had not produced shippable games, but some students said that they never wanted to finish a project in the first place—they really just wanted to tinker around. That's interesting for several reasons, one being that they clearly didn't understand what I had tried to teach them about Scrum, but the other is that they claim to have had a fundamentally different perspective of the course goals all along. One student put it very clearly that he thought he was "learning game programming in order to learn Unreal Engine," which is vastly different from my intention, which was for them to learn Unreal Engine in order to do game programming. Students in the former category would probably have been disappointed at my lack of didactic lectures about specific UE4 features, whereas students in the latter category would appreciate the seminar-style of individual exploration and sharing of lessons learned.
After these conversations came the Thursday group, where two groups really nailed it, showing off games that were truly shippable. In their presentations, they shared specific and interesting tech tips in such a way that everyone in the room learned something. (We called this requirement the Feature Focus™). I thought about going back to Slack to try to understand how people in the other teams felt about the excellent Thursday presentations, but I did not. I did not want to come across as crass or petty, although I do really want to know what they thought, as I try to understand their perspective—or what they claim their perspective is, anyway.
This isn't even my finals form
When we wrapped up the second iteration, I gave the students the option of either working on their final project through to the end of the semester or to end a week early and do a different final assessment. The result of our discussion was that the students wanted to be able to choose either a traditional written final exam or a "jam" final. I had used the jam format back in 2011, and they seemed to like the idea when I told them about it.
I tried to make the two equitable, each designed to take four to six hours. I designed the jam format first, using my university's recently-announced new branding campaign, "We Fly," which appears to be beloved of upper administration and generally denounced by students. I put the following constraints and requirements on their submission to this final format:
Playable game implemented in Unreal Engine 4.17 or 4.18
Implemented by an individual student in CS315
The creators of any third-party assets are credited and licenses identified as per licensing requirements
Player input is managed through project settings
Core gameplay includes interaction of multiple actors in a level
Dynamic game state is tracked in the user interface (e.g. score or health trackers)
Clear goal and ending conditions
Include music and/or sound effects
Captures an interpretation of the theme
A casual observer will notice that these include some of the specific features whose absence surprised me in the third iteration.
The students presented their submissions during our university-assigned final exam slot, and I was delighted at the quality of their work. Students whose technical contributions and understanding were unclear to me showed that they had minimum expected competence with UE4, and students who had really pushed themselves showed some impressive jam-quality work. It was fun to see how many students poked gentle fun at the university's marketing efforts, with several games involving the destruction of money, amassing money, or throwing money into bottomless pits in order to create new slogans.
Seven students opted for the written final format, which consisted of three questions. The first had students research and report on binary space partitioning and its role in BSP brushes; we did not address this in class, and so I figured this would be a good assessment of their ability to explore new topics related to game programming. The second question involved deconstructing a classic arcade game and discussing how it would be implemented in UE4, along with a project management plan such as a Scrum product backlog. The third was a post-mortem of their team project in classic GDMag format.
Their responses to the first question were a mixed bag, the biggest problem being that some did not distinguish between BSP and UE4's BSP brushes. In a sense, the assessment "worked" because I was able to see their confusion.
The problems with the second question were more interesting. Some students showed significant trouble analyzing the game to determine its formal components and then arranging those into a series of steps. In retrospect, I had not given them much feedback on the project management portion of the class, and I know from experience with immersive learning teams that this is hard enough just by itself. Similarly, these students generally had no experience with game design, and so they lacked some vocabulary and mental models for how to separate a player experience from formal components. This part of the question may have been "unfair" in that it wasn't something I had really emphasized throughout the semester. There was also variance in the second part of the question, where some students waved their hands and talked about implementation in broad strokes, where others were more specific about what was actors, what was pawns, who should have what responsibilities, etc. I think if I were to give this type of exercise again, I would have to include more a specific description of what a correct answer would contain, since it was hard to distinguish whether students thought their submissions were adequate or whether they were writing and hoping for points (a phenomenon any instructor is familiar with). In the end, however, everyone that did anything reasonable on the written format got a satisfactory mark, so it's not worth fretting the details at this point.
Students' Review of the Course
At the end of our finals week meeting, I ran a quick semester retrospective with the students. I wrote four columns on the board: + (for positives), - (for negatives), Δ (for things to change), and ? (for lingering questions). I encouraged students to speak up and share their thoughts, and I logged them onto the board without commentary except occasionally to ask for clarification. I then asked if there was anything on the board that anyone strongly disagreed with, and the only point of contention was that some people seemed to dislike Blueprints, while others thought they were valuable. Finally, I went through each item and asked for a show of hands to get a sense of whether it was a majority issue or a minority issue. I've transcribed the list below, marking majority issues with a capital M and minority issues with a lower-case m; the one item without a marking was instead discussed as being a mix of positive and negative.
Positive:
Two-week intro (M)
Choice of project (M)
Open-endedness (M)
Working off of the same basis for the two-week project (M)
Video tutorials for the two-week project (M)
Showcases at the end of iteration and the final (M)
Feature Focus™ (M)
Slack (M)
Negative:
Ambiguous grading scale (M)
GitHub
Infrequent use of Slack (m)
Change:
More smaller projects (M)
Require participation in game jams (M)
More emphasis on scoping and estimating projects (m)
Host more class-based jams like the final (as opposed to external jams) (M)
Use "jam" format for intro weeks (m)
More C++ (m)
Smaller projects with specific technology learning outcomes (M)
More videos on specific Blueprint features (e.g. variables, conditionals) (M)
Encouraging more student-presentations on UE4 technology (M)
Theme the entire class, in part to help control scope (m)
Encourage use of Slack (M)
Less distribution of course content between Canvas and the course site (m)
Questions:
I may have learned more from the final jam than from the big project (M)
More jams? (M)
Would having more jams early in the semester positively influence the formation of final project teams, their preparation, their estimation? (m)
Blueprints vs conventional text-based code (esp. for things like data structures) (m)
Production pipeline, e.g. working between technical artists and gameplay programmers (m)
Canvas is better than Blackboard but Canvas is less than ideal (M)
The student who brought up that first item in the questions, he spoke for a few minutes about how the scale of the final project contributed to his putting off working on it. By contrast, he said, the final jam had a very tight timeline and very clear requirements, and so he knew he had to dive right into it. He did not address whether or not he could have done the final jam without having worked on the major project.
At the end of this discussion, I shared with them something that had been heavily on my mind the past two weeks or so. Many of them had experienced problems or complained about the challenges of their final project with respect to scope, estimation, and creative direction. I looked across the room and I asked them to think back to when we were first planning the final projects, and specifically, how I had laid out for them three minigames from Collaboration Station that were already designed and were perfectly scoped for their work during the semester, and how I had warned them that going in a different direction was not advisable for reasons of design and scope. Ah, I wish I could have captured the flood of emotions that I could see on their faces! At first, a moment of shock and surprise, and then a look of understanding, and then nervous laughter as all the pieces came together. When I followed this with, "I don't want to be an 'I-told-you-so...'" they really let loose and had a good laugh. It's true, though, and I told them that I hoped that this was a good lesson for them in understanding how complex this area of study really is. I think we were all able to leave with high spirits despite having been a bit in the doldrums a week earlier.
Jam in the place where you work
Clearly, the idea of "jams" came up quite a bit in our semester retrospective, I'm sure in part because they had just had a generally positive experience working on the final exam jam. As I got to thinking about it, though, I still had some questions about what they were really talking about. A single-person effort isn't really a "jam": the term comes from musicians' jam sessions. The metaphor is also just a sort of fun wrapper around an academic assignment, and isn't it strange that students would say, effectively, "give us more assignments!" Maybe not so strange, given the one student's comment about learning more from the short experience rather than the longer one. In any case, I turned back to the class' Slack and posed the problem back to them. I articulated what I thought were the five characteristics of the jam format, and asked them to thumb-up the ones that they thought were most important or essential. Here's where the votes have stabilized:
Individual completion (1)
Themed (6)
Timeboxed at no more than one week (4)
General guidelines of satisfactory completion, as in the final exam jam format (9)
Presented in class (0)
I am a little perplexed at the zero votes on the bottom one, given some of the other results of the retrospective, but that one was also last in the list just before a block of text, so maybe people missed it.
What stands out to me is that the things they most wanted were the guidelines, which are really standard fare, and their omission from the final project was an oversight on my part. I've been thinking more about specifications grading and how that might be useful for a class like this. My colleague David Largent has been using specs grading in his classes and has started giving some regional presentations about his efforts. I could see criteria for a future iteration of the course looking something like this:
D: I can run the game without crashing.
C: The core gameplay is functional and gameplay state is tracked on-screen
B: The game includes audio and visual feedback for core gameplay
A: The game can be packaged and launched
That's just a sketch, but I think you get the idea. I can set up the hurdles that teams have to cross more explicitly to get the grade they want. Of course, this opens up a problem I had this semester and alluded to above: I had a student come to my office upset with his grade, and I told him the core gameplay didn't work. He asked, "What's the core gameplay?" I would have to be careful in the specifications and future course design to qualify any game design terminology, since clearly not everyone will be familiar with it, even if they are hobbyist gamers.
Wrapping up
The coarse-grained grading scheme that I adopted turned out working OK, although not without its own struggles. When students mentioned "ambiguous grading scheme," I assumed they meant that I had not rigorously defined "satisfactory," but judging from some of the questions I've received the last few days, it's possible they were confused by the variable MacGuffin system. This is another area where I wish I knew what the students actually were referencing, but I'm afraid that moment has passed. It's possible each student meant something different, but they generally didn't like ambiguity, so may as well vote for that one. However, looking at the grades on my spreadsheet, I think they lined up to fairly represent my intuitive understanding of what students learned during the semester.
There were also opportunities for achievements that I didn't think of until it was too late. Regular readers know that I enjoy painting miniatures, and the last few weeks I decided to listen to a few GDC lectures on YouTube while painting. I realized, in retrospect, that I should have had an achievement for students to do the same. The 2016 Tech Toolbox, for example, would have been a great item for students to study and respond to. The way that I had articulated the Connected achievement gave preference to in-person meetings, but there is such a valuable repository of knowledge distributed on the Internet, I could have used the course as a chance to get students connected to that as well.
That about wraps it up for CS315 in Fall 2017. Word on the street is that we may be able to go back to offering this every Fall, now that we have had some successful faculty searches. I would like to get it back on a regular rotation and hone the structure of it the way that I have CS222. This semester felt a bit choppy, but I had some very talented students who were willing to talk with me honestly about it, and I'm grateful for that.
