Wednesday, August 25, 2010

Backhanded compliments... I think.

In the first meeting of CS315 (Game Programming), I was a little surprised that I left with very little impression of the students' feelings for the class. There was neither palpable excitement nor crushing dread. In good faith, I posed this question on Facebook, where I am friends with several of my students:

BSU student friends: after literally months of planning for CS315, I have to ask: what's the word the street after day one? I feel like folks from 345 would have some idea what to expect, but there were a lot of faces in the crowd that were completely unfamiliar to me.

Well, the responses were enlightening. Here are a few of my favorites:

  • "They HATE it! I heard on google that everyone in the class who wasn't in cs345 thinks you don't know what you're talking about."
  • "I hear horror stories. About your class. In Texas. Everyone is that scared."
  • "I won't come out of my bunker, for fear that I might accidentally stumble into this class."
  • "Heard about that massive traffic jam in China that's been stalled for ten days? It's because so many people are trying to crowd around the furthest point from this CS315 class."
  • "People are willing to choose poverty and low quality of life over the class of Dr. Gestwicki."
  • "You're a monster, Paul."

It's good to have friends!

Tuesday, August 24, 2010

Unit testing rendering code in Slick

Several weeks ago, I developed a product backlog for my game programming class, and I tagged the first two tasks --- the highest priority tasks --- for demonstration purposes. Specifically, the plan was to complete these two tasks on Wednesday of the first week of classes (tomorrow):
  • Create a Position component.
  • Create an ImageRenderable component with corresponding ImageRenderSystem.
The first is quite easy and provides a good introduction to TDD. The Position component doesn't do anything: it's just a bounding rectangle in 2D space, so the test is really just over accessors and mutators.

For the second, I wrote a little note to myself that my unit test should load a 1x1 image and check that the color values match as expected. Rather routinely this morning, I decided to make sure I remembered how to do this before the class meeting tomorrow. The best I can figure now is that I never actually wrote this before, I only saw that Slick's Graphics class has a getPixel method. The mere presence of this method means that the test should be a simple one, right?

Well, no. Slick sits on top of LWJGL, and so one cannot simply create a Graphics object: there is all sorts of OpenGL initialization that happens behind the scenes. In fact, a similar thing happens when just trying to create a Slick Image object: this fails if done outside of the context of a Slick Game in a GameContainer since the OpenGL subsystem is not initialized.

Mock objects really won't help here, since the best one could do is check whether the ImageRenderSystem calls a specific rendering method on a mock Graphics object. This would be fragile and only test how the ImageRenderSystem uses the Slick API, not whether it behaves as it should.

After some lunchtime tinkering, I've come up with a solution that seems to work fine. The crux of it is to create one-off BasicGame implementations within the test case and embed the test code within. BasicGame is the default starting place for a Slick game implementation, itself implementing the Game interface. BasicGame is abstract and requires implementation of three methods: init, update, and render. So, let's look at the "easier" one first. This is what ImageRenderable looks like:

import org.newdawn.slick.Image;

public class ImageRenderable implements Component {
 private Image image;

  public Image image() {
    return image;
  }

  public ImageRenderable image(Image image) {
    this.image = image;
    return this;
  }
}

Easy enough, as long as you don't mind my Smalltalk-esque style — using the same method name for both an accessor and mutator, and returning this from the mutator.
This conceptually-simple unit test just checks that if we set an image, that the property accessor returns that image:

@Test
    public void testImageAccessorAndMutator() {
        final String RESOURCE_LOC = "images/white-1x1.png";
        try {
            new AppGameContainer(new BasicGame("Test") {
                @Override
                public void render(GameContainer container, Graphics g)
                        throws SlickException {
                    final Image image = new Image(RESOURCE_LOC);
                    ImageRenderable ir = new ImageRenderable().image(image);
                    assertEquals(image, ir.image());
                    container.exit();
                }

                @Override
                public void update(GameContainer container, int delta)
                        throws SlickException {
                }

                @Override
                public void init(GameContainer container) throws SlickException {
                }
            }).start();

        } catch (SlickException e) {
            fail("Failed to create game: " + e);
        }
    }

Here's a breakdown:

  • The game needs a container in order to run, so we create an AppGameContainer.
    This is the class used when launching games as applications, in contrast with
    AppletGameContainer.
  • The inner class overrides the three methods required, but only render is interesting to this test, so it's the only one that matters.
  • The image variable references the actual Slick image object, which is then referenced by ir. We don't actually need any Entity objects here: we're just testing the component.
  • assertEquals ensures that the image we put into the component is the one we get out.
  • After the test, we call the container's exit method to gracefully shut down Slick.
  • The start() call on the game container actually starts the test game.

Unit tests for accessors and mutators are nigh unnecessary, but this set-up allows us to do something more interesting with the system that processes these components. Here's a simple implementation of ImageRenderSystem, which assumes that its Entity arguments have both Position and ImageRenderable components.

import java.util.List;

import org.newdawn.slick.Graphics;

public class ImageRenderSystem {

    public void render(Graphics g, List entities) {
        for (Entity e : entities)
            render(g, e);
    }

    public void render(Graphics g, Entity e) {
        Position p = e.as(Position.class);
        ImageRenderable i = e.as(ImageRenderable.class);

        g.drawImage(i.image(), p.x(), p.y());
    }
}

Simple. Here's the corresponding unit test (which was written first, natch).

@Test
    public void testImageRendering() throws Exception {
        new AppGameContainer(new BasicGame("Test") {
            private Entity e;

            @Override
            public void render(GameContainer container, Graphics g)
                    throws SlickException {
                ImageRenderSystem sys = new ImageRenderSystem();
                sys.render(g, e);
                assertEquals(Color.white, g.getPixel(0, 0));
                container.exit();
            }

            @Override
            public void update(GameContainer container, int delta)
                    throws SlickException {
            }

            @Override
            public void init(GameContainer container) throws SlickException {
                BasicEntityManager mgr = new BasicEntityManager();
                e = mgr.createEntity();
                e.add(new Position().x(0).y(0));
                e.add(new ImageRenderable().image(new Image(
                        "assets/white-1x1.png")));
            }
        }).start();
    }

This has the same structure as the previous one. I am using the init method to set up the test. Technically, this could be done in render as in the previous example, but it seems that init is a good place to do initialization. The render method calls out to the ImageRenderSystem render method, and then we check that the pixel in the upper-left corner is indeed a white one. To finish the test, we should check that the rest are black, but that's left as an exercise for the reader.

This shows how to set up rendering-related unit tests in Slick. The real moral of the story, though, is: make sure you know what you're doing before you show up in class. This was supposed to take about fifteen minutes, and it took me over an hour to puzzle through all the parts. It's always easier the second time.

Thanks to David Craft for the post explaining how to set up SyntaxHighlighter in blogger.

Tuesday, August 17, 2010

Ready for Fall

Yesterday, the course descriptions for my two Fall courses received their final edits, and I am quite happy with the results. Here are links for the curious:
The CS222 description is straightforward. The more interesting part is the course design, which is not apparent from the course description but is reflected in it. The class meets twice a week for 75 minutes per meeting, and my plan is to have a pre-course reading, in-class activity, and assignment for every day. I have planned out the first two weeks and have sketches beyond. This should carry us to the ninth week of the semester, at which point we will change gears into a six week project, delivered in two three-week sprints. This will give the students the opportunity to perform a formal estimation and make commitments based on it, run a sprint, and then re-estimate and re-promise based on that experience. For most of the students, this will be their first opportunity to work in teams on a software project that includes any kind of formal milestones. We will be spending significant time discussing methodology and "soft skills" before the start of the project.


I have written extensively here about the planning for CS315. In the final version of the course description, I made a few editorial changes but only one change in content: I augmented the evaluation rubric to incorporate non-code contributions. Previously, the rubric had the headings "Quality of Code" and "Commitment". I changed the first to "Contributions", which has a nicer poetic parallel with "Commitment," and it captures the kinds of non-code activities that will be required of some students, such as checking copyrights on media and managing quality assurance testing.


The CS315 course description is complemented by the plans for the first 1.5 weeks. Clearly, CS222 is designed to produce students who can succeed in project-oriented experiences such as CS315, but since CS222 is new, I don't have the luxury of fully-prepped students. (Fortunately, many in the class are coming from my CS345 class last Spring, where they got experience in agile development.) The first 1.5 weeks will be the training at the beginning of a new job, getting the students the bare minimum skill set to start the first sprint. CS315 meets three times per week, and these are the plans for our first three meetings:
  • Monday: Demonstrate Eclipse and Mercurial using the MercurialEclipse plugin
  • Wednesday: Introduce Scrum, Entity Systems, and Test-Driven Development by completing a very small but legitimate slice of the project: pulling a task off the backlog, grabbing a partner for pair programming, writing tests, writing a solution, refactoring, committing, and updating the sprint backlog.
  • Friday: Form teams for the first sprint and bid on the first sprint's tasks.
I anticipate an exciting semester's teaching. I plan to keep using this blog for reflective practice with the goal of one quality post per week.

Tuesday, August 3, 2010

Student evaluation plan for game programming

Continuing the series of posts on planning for Fall's game programming class, I would like to share my student evaluation plan. Thanks to all of my friends and colleagues who have taken the time to discuss these issues with me.


By way of background, I used both self- and peer-evaluation in my HCI class in Spring 2010, where students worked in teams on Android applications. I have always been suspect of both kinds of evaluations --- they are neither reliable nor valid, statistically speaking ---  but it seemed like the best way at getting reasonable feedback. While I believe that most of the evaluations were honest, I found the lies to be much more interesting. Of course, the cynical mind leaps immediately to bloated self-evaluations, and there were some of those. Simple tricks like taking the median of the set of self and peer evaluations can address such issues assuming most people are honest. By the end of the semester, some students had an axe to grind and gave brutally honest feedback. As an observer, I think that much of this anger was justified. However, it was the inflated peer-evaluations that I found more interesting. Without breaking teacher-student confidentiality, I will say that these led to some fantastic discussions where the reason for "inflated" evaluations was not from a desire for reciprocation, but rather sympathy, maybe even pity for those who had made mistakes.


The point relevant to this post is, as Dr. House would say, people lie. Specifically, students lie to teachers. We have set up a cultural system in which it is inevitable, in which cheating is rewarded with material gain and, as The Spirit of Christmas put it so eloquently, "the rewards of virtue are largely spiritual, but all the better for it."


However, there is a force more powerful than material gain, a force that can strengthen leaders or destroy futures: peer pressure.


Without further ado, then, my experimental evaluation method is rubric-based, using the rubric shown below.
Category 3 2 1 0
Code Quality Uses pair programming or all commits are signed by a team member. Usually uses pair programming or most commits are signed by a team member. Rarely follows pair programming or few commits are signed by a team member. Has not contributed code
Commitment to Meetings Attends and is prompt and attentive for all meetings. Attends most meetings but is either late with significant frequency or is not focused during meetings. Attends most meetings but is routinely late or unfocused. Regularly misses meetings.
Facilitation (ScrumMasters only) Facilitated the success of the team. Mostly facilitated the success of the team with some problems such as failure to remove impediments or unnecessarily prioritizing owned tasks before team tasks. Regularly failed to facilitate the team's success Did not adequately serve the team.

The numbers are based on my general grading rubric. In a nutshell, 3=A, 2=C, 1=D, 0=F.

As part of the Sprint Retrospective, students will grade themselves according to this rubric. Then, in a meeting with the team, they will announce what values they assigned to themselves. While students may lie to me in a moment of weakness, I think it's less likely that non-sociopaths would do this in front of their team of peers. Teams can reorganize between sprints, and so I will collect all of the self-evaluations in a spreadsheet that is shared with the other teams.

I do not have any middle management in the course design: there is no one between me (Scrum Product Owner) and the students (Scrum Developers / Team Members) that externally evaluates them. In the event that there are students who are not carrying their weight on the project, I have set up a mechanism for firing them from the studio. Here's how I put it in the draft of the course description:
If I receive reliable reports that a student is not contributing adequately, I will schedule an individual meeting with the student to discuss the matter. Failure to attend this meeting or resolve the problems will result in the student's being fired from the studio: the student fails and may no longer contribute to the project.
I am hopeful that the combination of rubrics and public accountability will contribute to students' sense of ownership over the project. I am putting a great deal of trust in these students, and I want the evaluation plan to reflect that. As I intend to introduce the course, we're not just playing pretend: we are a game studio, and we have work to do.

As always, I welcome feedback and discussion.

Planning Poker with Undergraduates

I have spent many hours working on the product backlog for Fall's Game Programming (CS315) class, in which the students will implement the Morgan's Raid game. Originally, I specified the story points for every user story, but I was concerned that my estimates were not representative, especially since my teams were comprised entirely of novices who are familiar with neither the technology nor architecture.  Following the advice in Clinton Keith's Agile Game Development with Scrum, I decided that a planning poker session could be the best approach for reliably determining story points. Briefly, in a planning poker session, each participant has a hand of cards, and each card has a number on it. For every user story, the participants choose and then simultaneously reveal the number of story points they chose, repeating until consensus is reached. This facilitates communication among stakeholders about the product backlog.

My backlog currently has about 70 user stories, about 25 of which are high priority. An ideal planning poker session would contain stakeholders with areas of expertise relevant to the project. When dealing with undergraduates, there are no experts, but the gulf between novice and expert itself provides a motivation for planning poker. Although I work with undergraduates regularly, I still find that I often make mistakes when estimating how long various tasks will take them, especially when integrating ideas that I have internalized but that are new to them. In a project involving a combination of object-oriented and entity-oriented design, Scrumcontinuous integration, and unit testing --- all of which are new to the students --- there are ample opportunities for estimation errors! A planning poker session, I figured, could help us determine more accurate story points than if I did it myself. This would also be an excuse to explain the various parts of the experience to a subset of Fall's class, giving me practice and seeding the class with students who have already seen some of the ideas.

(A parenthetical story: Last night, my wife asked me about this morning's meeting, and I told her briefly what it was about. My brief explanation was completely insufficient, so I explained to her some principles of agile software development, Scrum, continuous integration, and planning poker. It seems that, despite the effort I have invested in preparing for the course, I only ever really explained it to her as "working on my spreadsheets." Explaining all of these things to her --- who has as much experience with these ideas as many of my students --- was a great experience for me. This also means that she probably doesn't read my blog, so I can safely write about her here, as long as we all keep our mouths shut.)

I sent out an invitation to some of the students I know who are signed up for CS315, using Doodle to find the time that most could meet. I wish I could have had them all, since I chose them for their breadth of experience and perspective, but the best I could do was Tuesday morning with four of the invited. This morning, one informed me he could not attend, and another did not show, so the meeting went forward with me and the two students. Not counting coffee and bagel time, we worked for about 90 minutes on the backlog and covered all the high priority items. I don't know anything about the programming skills of these two students, but I will say that if I have a class full of students as clever and motivated as them, this semester will be fantastic. As one might expect, the first few were a little rocky as I had to explain some of the background and we had to get a feel for the abstract measurement of story points. We quickly hit stride, and the story point negotiation revealed places where more detail or more stories were needed.

It so happens that all three of us at the meeting are gamers, and it was interesting to engage in planning poker as a cooperative game, especially given my recent interest and readings on software development as a cooperative game. Having this meeting face-to-face, it was fun to choose point values and hold the card face-down while waiting for the others to choose a value: there was the tension of anticipation, the climax of revealing values, and verbal and non-verbal negotiation, and the joy of consensus. In fact, in the few cases where we all chose the same value the first time, it definitely felt like a "win." The experience was a game in Huizinga's sense, especially in the construction of a time-boxed community of players in a shared experience. Qualitative technical communication researchers, take note for potential future research opportunities!

I am confident that the values we determined are reasonable and will give realistic measurements of teams' velocity. Based on this positive experience, I plan to have another planning poker meeting with some volunteers once we make progress on the high-priority user stories. I hope that the experience was as enjoyable for the students as it was for me. My only regret is that I did not take a picture of my awesome planning poker cards before I left, but they are in my filing cabinet for another day. Now if that's not reason enough to come back for more blog, I don't know what is.