The Online Student Showcase for CCSC:MW 2020

This year's CCSC Midwest conference ran on September 25-26 as scheduled, although we moved the whole event to be online. (No, not "virtual." It was a real conference, and it was online.) This was the right move to make, given the pandemic, although I found that it took away from me the things I enjoy most about the conference, namely, traveling with students and visiting with friends from the region. 

Once again, I was in charge of the Student Showcase, and so I had to spend some time this past summer trying to determine how to transition the event from in-person to online. Usually, the Student Showcase starts between sessions, with poster presentations and demos given near the refreshments area, and then the Showcase continues through the next session. During that time, I find volunteer judges, explain the criteria to them, and then later collect and collate their results. 

It is worth noting that the Student Showcase is more than just a research poster presentation. Indeed, it had been the tradition at CCSC:MW—like most conferences—that presentable student work was framed strictly as “research.” When the previous session chair retired from the region and I volunteered to take over the event, I was able to expand the scope. Part of the reason for this is my dissatisfaction with the word “research” that derives from my reading of Boyer's classic, Scholarship Reconsidered. This led to my creation of two tracks in the Showcase, named the discovery track and the applications track after Boyer's identified scholarships of discovery and of application.

I had two major goals in the design of the online student showcase. The primary goal was to give the students an authentic experience in showing their work to others, knowing that it would necessarily be a lesser experience than in-person. The second goal was to keep it as simple as possible so as to avoid failures. This second goal imposed a particular constraint on the format: it led me to abandon any plans for online, synchronous presentations. Multiple teams of students across multiple sites presenting in real-time to a distributed audience just meant too many opportunities for failure. In the absence of synchronous communication, I knew that I would need a way for anyone—not just judges—to give formative feedback to the students. Such discussion is certainly the best part of the in-person event, and I wanted to try to capture that as much as I could.

Based on these goals, I decided to allow the students to submit their work in one of three formats: a plain text abstract not to exceed 750 words; a 3–5-minute video presentation; or a research poster or one-page brochure. Each of these are authentic scholarly artifacts, and I have had to make each at different times in my own career. They are asymmetric in that each mode affords different expression, and it was my hope that students would be able to choose the mode most amenable to their work. A theoretical work may be best expressed in an abstract while a capstone software project might show best in a recorded talk, for example. I specifically excluded informational websites from the list because they can easily combine all the other forms together, and I wanted a semblance of balance between forms. I also excluded traditional research papers because students can already submit such work to the conference; it is rare, but I know of at least one student whose original research results were presented right alongside the faculty research in recent years.

The judging criteria remained the same as in previous years, and these criteria were shared with the students in the call for participation. The six criteria are taken from Glassick's Scholarship Assessed. 

One of the impacts of the pandemic is that I have been spending a lot more time writing policies than in the past, when I could easily explain things in person and answer questions. The CCSC:MW website maintainers tend to reuse the conference URL each year, but I want to archive the presentation of the call for participation. Below, then, is the entire text of the call from the 2020 conference page.

CALL FOR STUDENT SHOWCASE PROJECTS

One of the highlights of the annual CCSC Midwest Conference is the Student Showcase, which is an opportunity for students to demonstrate their best work to faculty and other students from around the region. Moving the conference online necessitates shifting our traditional face-to-face showcase to an online mode.

To qualify for submission to the Student Showcase:

• The entry must be original work of an undergraduate student or team of undergraduates.
• The work was supervised by one or more faculty members.
• The work was completed in the last twelve months at one of the colleges within the CCSC Midwest region.
• The student, or at least one member of the student team, is registered for the CCSC:MW 2020 Conference. (We recommend participating in the Programming Competition as well!)
• The entry must be received by 11:59PM EDT on September 11, 2020.

Interested students will classify their work into one of two categories. The Applications Track is for interesting, novel, or noteworthy applications of computing science. This track is usually selected for work that is best displayed as a demonstration of a working system. The Discovery Track is for projects that contributed new understanding to the body of computing science knowledge. This track is usually selected for work that is displayed on a poster or in a research paper. In either case, projects can be completed individually or as part of a team.

Regardless of track, all work is judged on the same six criteria. These criteria are adapted from Scholarship Assessed (Glassick et al. 1997) and provide guidance for evaluating any form of scholarship. Sample reflective questions are provided for each criterion.

• Clear Goals: What is the goal of this work? What problem are you solving
• Adequate Preparation: How did you get ready to do this work?
• Appropriate Methods: How did you solve your problem? Why did you approach the problem this way?
• Significant Results: What was the result of this work? Who is affected by this work?
• Effective Presentation: How well does this poster, demonstration, or presentation communicate what is important about this work?
• Reflective Critique: What would you do differently? What does this mean for you and your future?

For the CCSC:MW 2020 Online Student Showcase, student entrants can choose one of the following three options for their submission.

• Submit a three- to five-minute video presentation that summarizes the project. This must be submitted as a URL that any conference attendee can access.
• Submit a 750 word abstract that explains the project. This must be submitted in plain text.
• Submit a research poster or brochure that can be downloaded as a PDF and viewed online. This is a conventional way of expressing research findings at an academic conference, but keep in mind that in an online showcase, you cannot control the medium by which people will read your work: smartphone, laptop, large monitor.

This year, every conference attendee will have the opportunity to judge the Student Showcase entries. We value all attendees’ participation, and faculty and professional attendees will be given a 3X weight to their scores to reflect their expertise. The results and feedback will be tallied, and we hope to get you the results soon after the conference.

To submit your work to the CCSC Midwest 2020 Online Student Showcase, go to [form url].

I collected the submissions through Google Forms as I usually do. We had eight student teams submit their work to the Showcase. It is worth recording here that more than half of them categorized themselves into the wrong track, which tells me that (1) Boyer's model has not had the impact I would prefer in higher education, and (2) my description presumed too much prior understanding about the philosophy of scholarship. I need to be more blunt and say that the Discovery Track is what your faculty call "research". Baby steps.

After a short experiment with Google Forms, I decided to present the students' work and collect the judging results using my university's licensed version of Qualtrics. My intention was to have the form available throughout the conference, and so at any time, an attendee could follow the link to the online showcase, read or view the students' work. Furthermore, anyone then could volunteer as a judge. The only demographic information I collected was whether the participant identified as a student or as a faculty or professional, which information I collected for the purpose of weighting scores as described in the call. 

The form itself included the student submission and the following form. Neither portion of the form were required, and Qualtrics made it easy to show all eight students' work in an arbitrary but balanced order. This way, even if some participants only viewed some students' work, everyone's work should be viewed by someone.

Sample Judging Form

There were no technical problems during the conference, although I had to put tighter reins on the availability of the survey because I had not accounted for the need to announce the winner's at Saturday's closing session. We ended up making the Online Showcase available from 9am (the time of the first pre-conference workshop) until 11:59pm on Friday.

We had a total of 22 views to the Student Showcase. In a normal year, almost everybody would at least see that the students were showing their work, and a few might approach those that looked interesting or in their areas of interest. Still, I think 22 out of the roughly 100 conference participants was very good, given that they really had to go out of their way to see what students had to show. In retrospect, I should have asked for the names, universities, and submission titles to be included in the conference schedule along with the Qualtrics link; this might have grabbed more viewers. 

I was surprised at the low number of written comments. I made sure to write comments for each one so that they would at least have some feedback from a researcher/practitioner on their work.

I think that the Showcase went well, and I have already identified some easy steps I could take to improve it for next year. Indeed, improving the description of the tracks is something I can do regardless of the format of the 2021 conference. The question of submission format sticks in my mind, however. I certainly found the video presentations to be the easiest to watch during my own judging, and I found the posters predictably to be the worst. Making a good poster is hard, and most faculty fail at it; students making posters that I have to view on a screen is like double jeopardy. Some of the students who submitted text could have benefited from some kind of visuals—the sort one might include in a recorded presentation, for example. This makes me think that if I had to run an online showcase again, I would simplify the submission process to allow only recorded presentations.

Another aspect that was missing from the online instantiation of the Showcase was that student presenters could not chat with each other. During the in-person event, there's a lull at the end when the judging is done and students are tired, and this leads to a more casual, less stressful social time. I like keeping the online showcase asynchronous to prevent technical problems from ruining the experience, but I wonder if there would be an opportunity for a breakout-room style Q&A session in another year. For example, if everyone who viewed their presentations could drop into something like student office hours. This has a weakness that any synchronous event does: if no one actually comes, then you feel devalued.

Thanks for reading. I hope that this post is useful to you. I am planning on exploring this problem a bit this semester with my HCI students, and so I expect to write about that at least in my end-of-semester reflection post.

Reflecting on my activities at CCSC:MW 2019

This past weekend, I attended the annual conference of the Consortium for Computing Sciences in Colleges, Midwest. I have been involved with this organization for many years, and I wanted to take a moment to share an overview of how I was involved at this conference.

Student Showcase

I am in charge of the Student Showcase. Many years ago, it was a Student Poster Competition, but it felt to me that this format gave inordinate status to the scholarship of discovery. When I took it over, I revised the format to have two tracks inspired by Boyer's Scholarship model: traditional "research" goes into the Discovery track, and interesting applications of computing goes into the Applications track. This year, we had six Applications track presentations and three Discovery track.

There are practically always Ball State students in the Showcase, and so rather than judge the event myself, I organize volunteer judges. The students are told ahead of time that they will be judged on the same six categories recommended by Glassick et al.; specifically, they are told they will be evaluated on the following:

• Clear Goals (“What is the goal of this work? What problem are you solving?”)
• Adequate Preparation (“How did you get ready to do this work?”)
• Appropriate Methods (“How did you solve your problem? Why did you approach the problem in this way?”)
• Significant Results (“What was the result of this work? Who is affected by this work?”)
• Effective Presentation (“How well does this poster or demonstration communicate what is important about this work?”)
• Reflective Critique (“What would you do differently? What does this mean for you and your career?”)

Over the years, I have fiddled with the judging form with varying degrees of success. This year, I asked each judge to rate presenters on a 1-5 scale for each presentation and each category, and there were no hitches. The scale I provided was Poor (1), Below Average (2), Satisfactory (3), Good (4), and Excellent (5). I have also put the actual judging form online, in case anyone would like to see it.

To determine the winners, I simply take the medians across the six categories and sum them. This led to a clear winner in Discovery and Application tracks as well as a clear Honorable Mention (the third highest overall score), so those are the prizes we awarded. I am proud that my own Canning Heroes team presented and won the Applications track award this year.

Tutorial: Unreal Engine 4 for Computer Scientists

Since getting into Unreal Engine 4 a few years ago, I have noticed many interesting manifestations of Computer Science concepts. I decided to run a tutorial session this year at CCSC:MW to show these to the attendees. However, it did not go as well as I had hoped. I did not have machines that could run UE4, nor could I expect attendees to have adequate laptops, so I designed the tutorial as a sort of Show and Tell. Then I was scheduled for 8:30AM on Saturday, which meant that attendees would be tired and very few students would be there. The room we were given was really awkward: it was a lab, which meant everybody was behind monitors where I could not make eye contact. To make it worse, there was no station where I could stand and work at my laptop, so I was seated and, because of HDMI cabling, facing away from the attendees. To top it all off, I was traveling with family, and my son got sick that morning at the hotel, so I was distracted and unfocused.

Suffice it to say, I would not be surprised if the session were poorly reviewed. Heck, I would poorly review it. I would like to do something like this again, but in a more controlled environment. Perhaps I will move forward with writing up an actual paper about some of the interesting manifestations, and then be able to give a shorter show-and-tell in a future year.

On a positive note, preparing for the tutorial gave me several new ideas for video tutorials. In fact, I could take my tutorial outline, chop it up, and have a pretty good series. Now, it's a matter of determining which ideas have sufficient weight to merit the time required to do the video. (For those who don't know, I have a YouTube playlist of game programming tutorial videos. In fact, I have written up this blog post while rendering my latest video in Blender.)

WIP: Mapping Game Design Learning Outcomes to CS2013

I presented in the Works-in-Progress session what I have shared here on my blog about mapping game design learning outcomes to the ACM/IEEE CS Body of Knowledge. I think it was well received. I have made the slides available online for anyone who wishes to see them, but as usual, my slides do not make a lot of sense without the stories to go with them.

I think the audience assembled for the WIP session was happy to hear my story, and they seemed to understand my frustration. I think they appreciated seeing how one becomes more critical of CS2013 as one digs deeper into the recommendations. One mentioned that his institution had given up on CS2013 and simply used instructor consensus. Ball State is really the largest school in the region that regularly participates in CCSC:MW though, and most of the attendees were in very small departments at private liberal arts schools. 

Learning objectives for introductory game design: Continuing comparison against CS2013

I am giving a presentation in the Works-In-Progress session of the upcoming CCSC:Midwest conference. This short talk is connected to my post from April about mapping learning outcomes of game design to learning outcomes of Computer Science. Conversation on Facebook led me to understand that I had not fully explained my goals in that post, in large part because I had skipped the first part of the mapping: the learning objectives of introductory game design.

As far as I know, there is no standards body or recognized formal recommendations for game design curricula as there are with Computer Science, where we have the ACM/IEEE CS2013 curriculum recommendations. However, if you look at all the various introductory texts on game design, you will find significant overlap, although these are by their nature written along the lines of content rather than objectives. Each time I have taught game design as an Honors Colloquium or a Computer Science seminar, I have had to articulate the learning outcomes of the course; what I did this morning is reflect on this process in an attempt to pull out my current thoughts about what a student should learn from an introductory course. Here is my proposal:

After taking my introductory course in game design, a successful student will be able to:

• Apply common analytical tools to existing games, examples being MDA (Hunicke), Formal vs Dramatic elements (Fullerton), and Interactive Forms (Burgun).
• Identify positive and negative feedback loops in a game.
• Critically analyze existing games in order to understand their component parts and describe their interactions, including such components as rules, theme, and narrative
• Describe, apply, and justify iterative prototyping models to produce an original game.
• Describe and apply formal techniques of playtesting.
• Compose and present status reports.

I think this is a good number and granularity of items for how I like to introduce game design. Furthermore, having taught such courses for years, I think it's a good representation of the skills that students develop.

As I reflected on this list, though, I realized that there's another layer to the learning objectives. There are objectives that are implicit—objectives that are hiding in the cracks and in the details. Here's what I came up with as the shadow objectives:

A successful student will be able to

• Read, understand, and critique book chapters, essays, and articles on game design.
• Articulate specific, measurable design goals from ambiguous constraints and inspiration.
• Incorporate feedback to improve a prototype and reflect on that process.
• Incorporate feedback to improve analytical, compositional, and creative skills and reflect on that process.

The first one here is not to be overlooked. When I look at the design and goals of introductory computer science, learning how to read does not come up at all. I believe it also to be completely absent from CS2013. However, this is a crucial skill for someone who wants to be a lifetime learner.

The second is, perhaps, the most important skill that someone can learn in this class, and its transferable to any other creative domain. By "creative domain" here, I mean any that require creativity in the technical sense—the development of new ideas—so we can include all the sciences, engineering, and education along with the arts and humanities. More specifically, it is exactly this skill I want to see strengthened for nascent software developers and computer scientists: every day we have to manage ambiguity, transforming it into measurable goals and progress. Again, however, CS2013 has very little to say about this except for a couple of hours of requirements engineering. It may simply be a case that CS2013 is too deep into the weeds, but that's kind of my point: stepping back from the weeds, managing ambiguity and turning into concrete communications is something that a computer scientist does daily, but it's also something that is often ignored in curriculum and assessement, in my experience.

The third item is a clarification of the objective to apply playtesting techniques, but as above, there is something more potent here than just playtesting: it's about closing feedback loops to improve a product. Reflecting on this led me to compose the fourth bullet point, that it's not just about the feedback loops of the game but the feedback loops of the person. Maybe you might just call this "learning", yet I think there's not enough of this going on across our campus. For example, for a course to be considered a writing intensive course here has required that a student complete at least one draft-feedback-revise cycle. In my department, there are a few places where we practice this explicitly, and all of these are places that I have touched, which would not have this otherwise.

My next step in preparing for the presentation is to take this list, look back at my earlier post, and turn again to CS2013 to make sure that my points are clear. Right now, I have a feeling like the things I want to be emphasized by CS2013 are simply not in it at all, or at least not explicitly. It will be interesting to see the reactions of the generally-conventional conference attendees.

Notes from Meaningful Play 2018

One of many effective learning techniques that my students don't use is transcribing their notes. I was reminded of this when I took many pages of notes at the 2018 Meaningful Play conference. I decided it might be fun to share my list of odds and ends here. Perhaps there will be something here that the interested reader might learn from, or at the very least, it gives me a chance to create a digital archive that I can return to.

Tracy Fullerton mentioned a book called Wanderlust: A History of Walking. That could be interesting.

She also mentioned a Situational Game Design, whose abstract claims that it is about analyzing games as player experiences rather than systems. This could also be good, and it makes me wonder how it compares to Chris Bateman's writings about games as being constituted of player practices and with the systems approaches that I tend to like from folks like Koster, Cook, and Burgun.

Fullerton was making a case that games should be about interesting situations, and that this should include meditative play. She gave an example of a meditation-reward system in Walden: having Thoreau pause and look over a scenic setting rewarded the player with a little narration. To me, this raises the question: who is meditating, the player or the character? She gave several examples of games that included this kind of mechanism, but there weren't any compelling counterexamples given. It left me with a sense that maybe I didn't understand her, or maybe she assumed the listeners were already on the same wavelength. A particular call was made to move "beyond mechanics, beyond systems" and toward "a series of meaningful situations" in game design. The problem I have with this is that having Thoreau narrate his meditative experiences is system design. I didn't have these thoughts fully assembled in time for the Q&A, and I did not have a chance to talk with her later in the conference. I appreciate her sharing her work-in-progress ideas about where to take game design. That is, after all, a lot of what I do here.

Well Played has a CFP about intergenerational gameplay, and the idea is for the intergenerational players also coauthor an article about the experience. I'm thinking of trying this with my sons, probably the oldest one.

Ann Arbor is known as a fairy town, with little hidden fairy doors throughout the city. People geocache with these doors too. I need to tell my colleagues at Minnetrista about this, in case they don't know about it already.

The city of Brussels has a tour based on comic art. I need to send this to Easel Monster.

Is reading Finnegan's Wake worth it? Paul Darvasi says so, and he's one of the coolest scholars I know. He also said you can't "read" it, but rather you "study" it, time and time again. It sounds fascinating, maybe it's like the English literature version of Godel Escher Bach.

I heard someone refer to "gamification" as "punish by reward", referring to how it uses extrinsic motivations to diminish actual learning results. I need to keep that in mind for this workshop I've been asked to run at BSU about game-based learning.

Eric Zimmerman seemed to me to be separating games, systems, and play in his keynote. Part of what gave this away was a slide with those three words on it, and that he talked about them as separate. I had a hard time with this, since I see them as not just interconnected, but essentially the same—at least when done well. I asked Darvasi about this and he pointed me to Zimmerman's Ludic Century manifesto, which covers the same ground and, fortunately, is shorter than Finnegan's Wake. As with my earlier notes, I don't have all my ducks quite in a row here yet, but let me share the gist of it. This is informed, without a doubt, by Cockburn's Heart of Agile philosophy. There are systems in nature that are studied by natural scientists. Other systems arise from human behavior, and the intentional ones are studied by the sciences of the artificial. A good designer must recognize that the system they design fits into a bigger system, and that people have experiences before and after, and sort of parallel to, their systems. These systems are like software: we design them as specifications, but they have dynamic behaviors. We design them statically, we hope for the dynamic behavior we want. To design intentionally for humans requires accounting for human nature, which is playful; put another way, if you design for humans and you don't account for their playfulness, you are not doing good design. I think this is the right idea but I need to work on the articulation. I am very grateful to Andrew Peterson for listening to me as I tried to sort out my thoughts about this, and who constructively disagreed with me, and Mars Ashton, who said essentially, "Yes, it's all just design."

One of the best presentations I saw was by Sandra Danilovic. She organized a game jam for people with a variety of disabilities and welcomed them to make games about their disabilities if they desired, and she conducted a qualitative study around the event. One of her findings she called logopoiesis, which was essentially about the healing power of computational thinking (as I understand it from the presentation). I asked her if this particular factor was tied into the fact that these people were making games versus, say, film or poetry. Turns out she herself had background in a variety of arts, and she said two particular cases in her study dealt with the challenges of problem solving through programming and with the meditative act of arranging pixel art. These things she found in her analysis to be separable from the other characteristics. I thought this was fascinating: there is a lot of rhetoric about the power of computational thinking (more rhetoric than empiricism, but maybe that's unavoidable), and I have said for years that computing is really a new liberal art. This was the first time, though, that I have come across the therapeutic value of it, as related to its poesis (the making of a thing that did not exist before—a definition I had to check because it's not in my usual lexicon).

I learned about the game Night in the Woods, which sounded quite interesting. The more I heard about it, though, it also started sounding more and more nihilistic. I'm not sure if it should go on my to-play list or not. The speaker also recommended Wandersong, which I also didn't know much about. I seem to be behind the times on trendy indie games. Heck, I just finished Bard's Tale IV, a sequel to a game from thirty years ago, so "behind the times" may actually be generous.

Someone recommended I check out Mark Rosewater's Ten Things Every Game Needs. They said it was a good summary of ideas, well articulated though not groundbreaking. I did not write down who made this recommendation and cannot remember now. I just scrolled through, and it looks reasonable.

I had a great conversation with Andrew Peterson during a "dinner break" when neither of us felt like going and getting dinner. He shared with me several interesting ideas he uses in his game design class. First, he has completely "flipped" the class. They do readings and preparation on their own, and class sessions are almost entirely devoted to teams working on their prototypes. The teams are randomly assigned themes from a brainstormed list. His students have a week 12 ship date, at which point they have to have their materials sent off to Game Crafter. The physical prototype that arrives is what he then grades. This front-loads the work into the earlier part of the semester and allows for more slack time for the students in the last three weeks, when their other classes tend to build to fever pitch. Peterson also mentioned that as an instructional designer and game designer, what he likes to do is ask faculty what they don't like to teach from a course: identify the variables, determine which can be turned into a game. This also might be useful for me as I start prepping for my own campus presentation on games in learning.

I met Chris Totten, who said the most perfect and quotable thing over breakfast: Games should be good. He is clearly a like-minded individual.

Kate Edwards supposedly has an excellent talk about imposter syndrome. I think this must be it. I was at a table with some very talented people, all of whom had stories about how they themselves had been touched by imposter syndrome and how they knew some of their heroes also did. This is interesting by itself, but one also mentioned how he teaches his game design students about imposter syndrome and the Dunning-Kruger effect. He has them do a short jam of sorts to show off their skills to their cohort, after which it is easy for people to feel outclassed. He introduces these topics then, and he reminded our table that students generally are unaware of these phenomena. It made me think, I should do something like this in many of my classes as well.

Henry Petroski has a book about the pencil. That sounds amazing.

The Death and Life of Great American Cities sounds like a great book about urban planning. Some colleagues were very excited about this work, enough to make me think it could be worth looking into.

A friend told me about a talk that Jesse Schell gave at GDC about grant funding in which he set a $50 bill ablaze. I was able to reach out to him for the link, and I look forward to watching it later.

A speaker happened to mention that he uses one-page design documents successfully in an upper-division game design and development class for Computer Science majors. This surprised me, since the times I've tried it, I've found the assignment confounded by my students' lack of visual communications and document design skills. It sounds like they are doing the designs on whiteboards, but they're also iterating on them during the semester. I sent out an email asking for more information last night, and this morning—as I continue this lengthy post—I have already received a response. They have their students start with whiteboard designs and make them progressively more refined during the semester. I also see that I think we were using different terms for the same thing. They are drawing upon David Osorio's style, where a "one-page" is more of a pitch document, whereas I use the term drawing upon Stone Librande's, where it replaces a design document. What they're doing in Osorio's style, which incorporates images and two-dimensional design, I have my students do using Tim Ryan's concept document format, which privileges text.

One of the best talks I saw, both in terms of content and form, was by Jessica Hammer. Her work was inspired by research findings that students don't get better at giving feedback during the semester if the skill is not taught. Hammer presented her EOTA model for helping students give feedback to their peers after playtesting. "EOTA" is an acronym that walks through the kinds of feedback that should be given in sequence, the expansion being: Experience (I thought..., I felt...), Observations (I saw...), Theories (Therefore...), and Advice (You might...). She described a learning experience where the designer had to remain silent during the whole feedback process, although she amended that to include that the designer could say "Thank you." I look forward to reading her whole paper once the proceedings are up, because I think this is the kind of thing I can bring into many of my classes to help students learn how to give and receive feedback.

Another wonderful pedagogic structure she described was to have students write down their favorite snacks on a survey at the start of the semester. Then, when a team does really well, she can bring in those students' favorite treats in honor of their accomplishments. This is very clever, since it doesn't put anyone on the spot: the students who did well know it was them, and everyone gets to enjoy the treats. I need to keep this in mind as I'm planning next semester's courses, although maybe this will have to wait until Fall 2019 just due to the teaching I expect to be assigned for Spring.

A keynote speaker mentioned the concept design fixation, which is when one gets caught thinking of an object only for its conventional purpose. I jot this down here just because it's a useful term that I'm not sure I would have thought of, had I been looking for it.

I met Derek Hansen who is teaching cybersecurity and using games, so I just sent him some info about Social Startup Game, which Kaleb Stumbaugh and I created as a research and design project a few years ago. I had to look up where we presented our findings besides the S2ERC Showcase, and it turns out that was Meaningful Play 2016. Unfortunately, when you look for the proceedings of this conference, they are nowhere. I have emailed the conference organizers a few times over the past two years to ask about it, in part because I am so happy with the evaluation that Kaleb and I conducted. Still, however, no paper in the proceedings. A copy of the paper is available on the project site, however, so interested readers can check that out. (I forgot it was there, so in fact I just rebuilt the paper from LaTeX sources to send to Hansen. Oh well.)

A speaker strongly recommended Kurt Vonnegut's Player Piano, which I am sure I've never read. Maybe that would be a good piece of fiction to offset all the non-fiction building up in my reading list.

A keynote was given by Katherine Isbister, whose work on wearable technology was not previously known to me. I am sending her name to some friends studying HCI.

A speaker brought up the idea that karaoke-style games can help people learn to pitch correct. It made me wonder if there are good ones that my kids can use. Some of them clearly love to sing, but I find it hard to teach them to hear where the notes go.

Sissy's Magical Ponycorn Adventure. Gunman Taco Truck. Ian Schreiber has been kind enough to talk to me quite a bit about "fam-jams", or family game-jams, and these two games are great inspirations. I think I already wrote about how my oldest son made two complete games at the last Global Game Jam, whereas mine barely worked. It inspires me to find opportunities to do some more creative collaboration with my kids. Chief among his tips are that when working with your children, you should do what they say—not necessarily what they want, but what they say. Watch this blog for details.

I overheard a friend mention a book called something like "Just Keep Writing," and he said that although the book was about creative writing, it could just as easily be applied to games. I am not sure what the specific book was, but I'm sure he's right. It brings up a thought that came up all during the conference for me: I should make more.

In her keynote address Saturday morning, Diana Hughes from Age of Learning brought up mastery learning. This is the idea that students may not move forward in the curriculum until they have shown that they have mastered antecedent work. It struck me that this is essentially a tech tree or a skill tree in games: you cannot build the next thing until you build its predecessors.

I attended a workshop about The Agile Teacher. It is a game created to help teachers—particularly new teachers—to explore active learning techniques. The game presented each group with a context, ours being a mathematics seminar with 10 or more students. Each group presented their findings to the rest of the room. By design, groups were supposed to have a context in which no one at the table was an expert. The designers explained that they had seen cases where the one person at the table who is from that area would tend to dominate the conversation. That makes sense. However, I also noticed (and shared with the designers) a fascinating phenomenon: without pedagogical content knowledge, every group designed activities that represented stereotypical understandings of the domain. The groups that had drawn Math as their domain reverted to a high-school understanding of math as essentially computation. The groups that drew Science worked on techniques to help students memorize taxonomies. Computation and taxonomies are both parts of math and science, but they are ancillary. It's not exactly a flaw with their game, since it was doing what it was designed to do (namely, foster conversation), but it was an interesting phenomenon nonetheless. It made me think about how I have seen people at my institution pigeonhole other faculty because they think they understand the others' domain. Maybe it's an instance of good old, "You're a Computer Science Professor? Can you help me with my printer?"

The final session I attended was another workshop, this one given by the aforementioned Andrew Peterson about his game, enRolled. The game is designed for new college students, to get them thinking about the impacts of their decisions as students. His motivation was excellent: as an adjunct, he was handed a course where he was supposed to lecture new students about how to succeed, and he realized that lecturing about these topics was of limited value. Peterson then worked with his students to design the first incarnation of a game to convey similar ideas. The difference was that student would engage in meaningful and authentic discussion around how to codify things like drinking and studying as game mechanisms, whereas they were hesitant to do so in a dry lecture. Very clever. He said a few times, "The game sucks," pointing to its dependence on random events and lack of balance. However, he was also clear that it does what it was designed to do: foster important conversations. This is an interesting dovetailing into my previous notes, reflections, and conversations around what design really is. In this case, Peterson has a playful conversation generator that happens to be a game, and so its fitness function is not about balance but about authenticity of emergent conversation. The game can be purchased one-off from The Game Crafter, but he also mentioned that he hopes to run a Kickstarter to print many copies at once and therefore reduce the price.

There was one particular story that Andrew shared with me that I want to capture here, so that I can re-tell it to my game design students. I hope that he doesn't mind my doing so. When he was working with this students on enRolled, the task was to determine the relative negative points of drinking and drugs. Andrew started with the idea that drugs would be worth -10 and alcohol, -2. His students, however, disagreed, justified by the prevalence of alcohol abuse. Very few of them knew someone who dropped out of school because of drugs; they all knew several who had dropped out due to alcohol. What a great example of how game design gave rise to new insights and conversations!

That's the end of my notes. I've shared here almost everything within my pocket notebook, just skipping some of my notes about questions to ask presenters that turned out to be not worth sharing here. Other interesting things happened during the conference as well, but my goal here was not to write a conference report but only to assemble the thoughts that I wrote in my pocket notebook. (Variations on these notebooks have always been in my pocket since reading Pragmatic Thinking and Learning, by the way.) I am tired out from the conference and feeling a little apprehension at the coming week, knowing that I now have to catch up on a backlog of tasks. If you have made it this far through my notes, know that I'm happy to discuss any of these ideas with you in the comments or future communications. Many thanks to the organizers of the conference for such an inspiring event, and thanks to those attendees who shared their knowledge, wisdom, stories, and advice.

SIGCSE 2018 Talk: Design and evaluation of an undergraduate course on software development practices

For many years I have been saying to myself, "I should write a paper about CS222." Well, I did, and I just finished giving the oral presentation at SIGCSE 2018. Check it out, standing room only!

(The empty seat in the front row is my own.)

I was the third speaker in a post-lunch session, and I enjoyed the other two talks. Nicole Herbert from Tanzania was especially interesting, as she was describing a capstone structure very much like where I think we could take ours at Ball State with a bit of curricular and potentially administrative revision.

My paper can be found on the ACM Digital Library. I think the talk was well received, and a few attendees asked for a copy of my slides. I prepared all the slides using Google Slides—which I've never done before—andthat makes it very easy to generate a public link. You can find all the slides here.

Thanks to all the attendees, especially to those who came up to ask questions and share stories afterward. I'm truly heartened to hear how much this message resonates with you. If you are new to the blog and want to read more about the Advanced Programming course, search for the "cs222" label. My most recent post on the topic is in my "What we learned" series, in which I share the outcome of the somewhat unconventional final exam format that I use.

You can find the most recent course site, with assignments and evaluation schemes, at https://www.cs.bsu.edu/homepages/pvg/courses/cs222Fa17. I actually have a break from teaching the course this semester—the first such break in many years!

My CS222 YouTube channel is publicly available. It is a collection of tips and tutorials. Some videos are extensions of in-class activities, particularly the earlier ones; I've tried to transition to more encapsulated presentations for improved modularity.

This is a quick after-presentation post. I will try to share some more about the talk later, but I wanted to get this up for any visitors who come looking for the slides and links. I am happy to share my course materials, instructional videos, and reflections. If you do end up using them in your courses and designs, I'd love to know about it. Consider the material something like CC BY-NC-SA.

If you came for the miniature painting, search for the "painting" label instead :) Cheers!

GLS 2013 Conference Report, Day 3

[Day 1][Day 2]

Constance Steinkuehler gave a fantastic keynote presentation this morning. She gave a reflection primarily on her year at the Office of Science and Technology Policy in Washington DC. She focused her comments in four domains: federal agencies, philanthropic agencies, the games industry, and academic consortia. The talk was recorded for Webcast, although the link doesn't work as of this writing.

One of her major points was that we need more businesses coming out of serious games, particularly to deal with the pernicious problems of maintenance, marketing, and community. This echoes what Schoettler said Wednesday and Flanagan described doing on Thursday: seeing commercialism as a means to an end. I find this compelling as I consider my own corpus of work and current interests. However, I also fear the negotiations with university IP people. I chatted with a colleague who works in a start-up, and he described how university lawyers killed a private-public partnership over a $15,000 grant. That's peanuts for any serious project, and from his story, they wasted much more than the overhead cost of the grant in their conversations around it. The school he was working with was larger—and more successful with external funding—than my own. However, Steinkuehler also made a strong argument during Q&A that to get anything done requires people and resources, but that people are the hardest part: if you want to see something done, you have to be the one to invest the effort to make it happen. By that token, I suppose I should bite the bullet and not shy away from the more ambitious projects. I think she used the term "Own that ulcer for two years," which got a laugh, but I feel I need to seriously consider the value of my own time and happiness as well, and that of my family.

She made a strong call as well for the GLS community to be playing, critiquing, and spreading the word about each others' games. This is a nice idea, but I wonder if the community tries to keep on its kids gloves a bit too much. The last two games I've played in this vein were overproduced garbage. That is, the gameplay did not match the learning objectives at all, and the narrative could very convincingly be argued to teach the opposite of their goals.

Steinkuehler described her frustration in trying to collaborate with AAA publishers on learning games, coming to the conclusion that they just couldn't justify the lesser revenue to their shareholders, and that the megapublishers are necessarily risk averse. By contrast, she made the valuable and justifiable claim that we—the community that designs and develops learning games—are indie developers, and we need to embrace that. In particular, she encouraged everyone to go to GDC and walk through the Indie Arcade. Easy sell for me: that trip's going into my next grant proposal.

The four parting points of the keynote were: collaborate; we need more businesses; stop using the "chocolate-covered broccoli" metaphor (because of what one has to believe in order for this metaphor to be meaningful); and that happiness is a global priority (and so not to back away from "fun").

After the keynote, I ended up in a session that was a bit outside my interest area, but I was curious as to what happened in it. The most entertaining talk was one by Edd Schneider and Tony Betrus who reported on a series of experiments on task completion in GTA. The part I found most interesting was how they took a series of easy, medium, and hard tasks and compared the results between having the tasks ordered by a teacher (starting with easy, then into medium, then into hard) versus allowing the player to choose which ones to do in what order. They found that the teacher's choice resulted in the players completing more tasks, but the player's choice resulted in the them completing more hard tasks. Schneider and Betrus argued that this was a win for both cognitivists and constructivists, highlighting that they produced better results—while acknowledging that constructivists probably care less about learners completing small irrelevant tasks.

After this, I wandered over to the Education Arcade, my first trip there of the conference. I walked through to get a feel for what was there, and I overheard a great question from Roger Travis, who was giving feedback to a designer (I think): "What's the verb of the learning objective?" This is the kind of question I've heard posed regarding course syllabi particularly when I was serving on the College Curriculum Committee and as departmental Undergraduate Program Coordinator. I think it's a useful technique for sharpening the discussion of learning outcomes. Yet, I had never thought to apply it to serious games, which surprised me. I had just presented yesterday on the learning objectives for Morgan's Raid, as articulated by the original design class, and they were pretty raw; they wouldn't pass muster on the College Curriculum Committee, that's for sure. I'll need to keep this in mind when I work with this coming Fall's game design course to see if we can make some hay out of this.

Walking through the Education Arcade, I saw a poster about Monsterismus, a game to teach programming. Well, that's up my alley. Turns out it's by Matthew Berland, who was a discussant in several sessions I attended the previous two days, and whose comments at the time made him seem like a kindred spirit. Berland, Don Davis, and I, along with a chap whose name I did not jot down, ended up talking at length about games to teach programming, and I used the opportunity to bounce some of my latest thoughts off of them, including breaking the C-style programming mold (such as by using stream-based or prototype-based languages) and incorporating programming as player action in board games. I tried writing a lengthy blog post on these ideas at the end of last semester, but I had a very hard time inventing a vocabulary with which I was happy; fortunately, these guys seemed to grok where I was coming from, and I look forward to continued conversations with them as well as seeing where their work takes them. (Incidentally, Berland and Davis are also involved in IPRO, a game for iOS that is played by programming soccer robots, for those of you in the Mac space. I guess you can find it on your AppStore, or something.)

After lunch was a fireside chat with Jim Gee, but it was a bit anticlimactic since he mostly talked about The Anti-Education Era, which I just read. He is clearly something of a superhero to many in this community, and he had some good stories. Gee pointed out that schooling should not be geared toward jobs since 3/5 of the jobs are in the service sector and require only rudimentary training; instead, education has to be about something bigger than merely certification for employment. In a similar vein, he mentioned that education is a complex system, and that it is known that control studies cannot work in complex systems because you cannot fix the input variables. Yet, policymakers don't seem to understand this. I would have had some questions for Gee, in particular from my criticism of aspects of the book, but they were only accepting questions via Twitter, which is not a thing I do. I don't mind not having asked my question, but it was strange to me that they limited questions to this one technology.

Several awards were scheduled to be given next, and I was contemplating skipping this part. I tend not to like award ceremonies, but GLS did it right. They brought up a slide with a bunch of names on it and asked all those people to come to the front. Then, they ran through all the categories, handed out the awards, cheered, and were done in five minutes. Attention everybody: if you need to give awards at a conference, do it this way.

In summary, I found this to be an excellent conference. The mix of attendees was refreshing, including professionals (including non-profits such as museums), academics, and teachers. I was able to get lot of new references to read about, which is always good. I definitely hope to return again in the near future, and given that it's in driving distance, perhaps I can convince some of my students to come along as well.

It was good to see some friends at this conference too, people I have met at other conferences and online. When I started working in serious games about three years ago, I felt very disconnected from any community of practice. Now, I feel like there are people whom I can contact for help and inspiration, and indeed, several of the people I've met through these conferences have been generous in sharing their stories and advice.

For those of you who read this far, I will share my collection of Conference Tips from my trip to GLS. Note that most of these are based on mistakes committed (often unwittingly) by graduate students in attendance, but not all.

• If you come to the microphone to give any kind of formal talk, and you have not been introduced, introduce yourself.
• Don't say you're interested in making a game to "target women." They're half the population. That's not a target.
• If your volunteers have shirts, don't put, "Have questions? I can help!" on the back of the T-shirt. This means the person who can help is already walking away from you. Put this on the front.
• Never put light text on dark slides: it fails with a modicum of ambient light. There's certainly no excuse for this if you're from the host institution and know the rooms that will be used.
• Don't put thing on the slides that people cannot read, and then tell them it's not important to be able to read it. Design your slides differently.
• When giving a presentation, don't use a friend as a remote control. Get an actual remote control, or manipulate the slides yourself. It's less distracting than your trying to tell the other person when to move forward or backward.
• Don't use 3D bar charts for any purpose. They're just bad visualizations. Exceptions can be made for irony.
• Don't use pie charts either, 2D or 3D. They're ineffective and misleading. Exceptions can be made for pie charts describing their own similarity to Pac-Man.
• Be aware of the British and American hand signs that involve sticking fingers up at people, and to be safe, don't use any of them. That is, if you need to count, do it with your palm to the audience.

Ice cream on the terrance? Yes, please.
See you next time, Madison.

GLS 2013 Conference Report, Day 2

This morning's keynote was from Mary Flanagan about her work at TiltFactor. I have been a vocal fan of her lab's work since meeting some of them at Meaningful Play. The keynote did not disappoint—except the part where she used non-horizontally-aligned 3D bar charts to display some data. Ack. In any case, bias was a theme of the presentation, and she referenced the Why So Few? report, which was not previously familiar to me. Flanagan shared an interesting study that showed how Apples to Apples and Awkward Moment, even though they share the same game mechanisms, produce significantly different psychological impact on players. In another study, her lab showed that females' performance in Blokus reduced dramatically depending only on whether it was introduced as a game, a strategy game, or a spatial game. Included in her presentation was a strong recommendation to read Timothy Wilson's Redirect, so I've put it on my summer reading list.

As the previous morning's keynote had done, Flanagan also addressed the issue of sustainable models. This makes me reflect on the choices I give to my own student teams. For example, last Spring's game development studio consciously and explicitly chose to design a game for a very specialized context: educational outreach programs at the Indiana State Museum. This is a very narrow niche! I wonder if perhaps I should be sharing more of the advice I'm hearing from recognized leaders in serious games, and if as a result we should be targeting wider markets. Of course, then university IP rules and commercialism raise their collective head, and that's just not a beast I've felt up to taming.

I attended the morning session on assessment, and the first paper was presented by Jodi Asbell-Clark, director of the Educational Gaming Environments group at TERC. She described work evaluating Impulse, a physics education game. Notably, the game is designed for implicit learning, something I am interested in. She described the game as "situating game mechanics in scientific behaviors." They heavily instrumented the game to produce rich player activity data and combined this with both interviews and video recordings of players. This is yet more encouragement for me to push my teams to include meaningful logging data with all of our educational projects. An interesting anecdote from the presentation was that kids who liked science tended to also like science-like puzzle games and to also see their relevance to their lives, whereas kids who reported as not liking science did neither. I need to make time to read the full paper, as the presentation was full of great ideas that I would like to explore in my own work.

The next presentation was a study connected to Projenitor X. The big takeaway for me here was a reference to Kevin Dunbar's work on the critical and inspirational role of failure in scientific discovery. It's certainly something for me to look into after the conference.

Dan Hickey's presentation was my primary reason for attending the assessment session, since yesterday he promised to talk about why we should only be assessing reflections, not the other artifacts students are making. He presented a new assessment model based upon Engle and Conant's 2002 Cognition and Instruction article on productive disciplinary engagement, which I have not yet read but look forward to doing. He also strongly recommended Torrance's 2007 Assessment in Education piece—similarly not yet read but bookmarked.

During his talk, Hickey admitted to being a convert from individual differences psychometrics to sociocultural psychology. After the three talks, the discussant framed all three from an explicitly cognitivist perspective, Andrea diSessa's in particular. I'm still learning bits of both traditions, but from what I know, I suspected that this framing was not true to Hickey's core theories; so, I posed the question of why Hickey chose the sociocultural route. I have to admit, I had a hard time following the answer, but I think that's because it was based in a lot of the references that he had previously shared. I think this is a rich area for me to study, to help me to better understand assessment as well as sociocultural theories. Then, I'll ask Hickey again, and perhaps be better prepared to understand the answer!

Later in the day, Jim Bower shared some stories from his career, the most relevant of which to me was his articulation of the partnership model rather than the client model for educational software. This is something that we occasionally struggle with in the immersive learning model at Ball State, and the message sent by the university does not always match either the reality or the intention. Bower made a point that it is challenging to work with subject-matter experts who assume kids should love their content as much as they do, and when asked to elaborate on this, he described how educating partners as well as the end users was part of his goal.

I have a note in my notebook, "I should make a tower defense game." I have never made one. It would probably be a fun project. Maybe next GGJ or Ludum Dare.

A presenter referenced the term "intrinsic integration," which I have not heard before. It seems to reference designing games in which the player activity aligns with what you want them to learn. I talk about this a lot in my papers and teaching, but I never thought it needed a name. I suppose I should do a bit of legwork and figure out what communities are using this term so I can be sure to use it to help frame my theories of effective game design.

I went on a tour of Filament Games in the afternoon, mostly because I wanted to see how they used space and managed their talent. I verified that we had permission to take photos, which I assume means I can share them here. Not the best documentary evidence, I admit, but enough for me to share a few comments below.

Filament organizes into worker-units called "pods" that consist of two programmers and one UX designer. These pods then are connected with a producer, game designer, audio designer, etc., to tackle development tasks. The space in their studio leveraged this structure, with pods tightly collocated and separated from each other by dividers. Illustrators sat together and game designers sat together, but the space was clearly open enough that face-to-face communication was easy to achieve. Indeed, as we visited and ate lunch, I saw such informal meetings occur.

The tour was led by Maurice Cheeks, who mentioned that they follow agile practices, and so I was surprised when I did not see conventional artifacts such as Kanban boards or Scrum boards. I was able to talk with Gregg Sanderson, a project director, about how they track time and effort. They make significant use of Jira supplemented with Google Docs. He was kind enough to show some live data from Jira about stories, sprints, and estimates, which was great to see. Two particular pieces that stood out to me was that it looked like they had people assigned to fine-grained stories, not tasks, and that each of the disciplinary specialists (programmers, designers, etc.) provided their own estimates to complete their parts. After seeing it, I think I will likely stick with my familiar sticky-note approach in the short term, but depending on what kind of team I can recruit for next Spring, it's good to know what options are out there. (Did I mention I have a cool new project next academic year? More on that another time.)

Later in the afternoon, I heard someone make the claim that a game in which a player can click on animated animals for hints counts as helping the player to learn social-emotional skills. I find this hard to swallow, but I did not feel like challenging them, since I was getting myself psyched up for the next session.

My coffee, about 4oz left for the presentation.

I was the last speaker in my session. The first described an analysis of some of the iCivics games (coincidentally developed by Filament) and the second provided a theoretical model for how game design and civic engagement are intrinsically connected. My talk was a bit different, not dealing with civics or civics education, but rather with what real people actually learn by playing Morgan's Raid.

The Friendly Audience at GLS 2013

I'm happy to share the paper with anyone who's interested, but the short version is that this was Lyle Franklin's undergraduate honors thesis collaboratively transformed into a conference paper, with some additional input from my colleague Ronald Morris. He conducted a semistructured interview and thinkaloud gameplay with fifth graders in order to answer the broad question, "What does a student learn when playing Morgan's Raid?" We discovered three themes in the data. First, the students did learn about some of the important facts of the raid: who Morgan was, what he did, that it was part of the Civil War. Second, students demonstrated increased empathy after playing the game, both for the people of Indiana in 1863 as well as for Morgan and his raiders. Third, the student's youth-culture interpretations of parts of the game—particularly the words "chaos" and "reputation"—led them toward historically counterfactual conclusions. Fascinating stuff!

This was the end of the business day, so I enjoyed a beer and a nice dinner on the Terrace before returning to see how my family's day was in lovely Madison, Wisconsin. But that's the topic for another post.

[Day 1][Day 3]

GLS 2013 Conference Report, Day 1

I'm in beautiful Madison, Wisconsin, for the 9th Games+Learning+Society conference. My wife and I decided to make it a family trip, so we arrived Monday and took in some of the sights yesterday. We particularly enjoyed waking around the capitol building and the Henry Villas Zoo.

This is my first time at GLS, and I wasn't completely sure what to expect. I had heard that the conference would have a healthy mix of educators and researchers. My plans to attend last year's did not pan out. This year, I am presenting a paper entitled "Empirical Research on the Effectiveness of Morgan's Raid," which was coauthored with Lyle Franklin and Ronald Morris. It is based on Lyle's work on his undergraduate honors thesis, and I'm proud to be presenting it tomorrow.

There are two seasons on university campuses: winter and construction

The morning started with a few comments from Steve Schoettler, who is most famously a co-founder of Zynga and is now involved in the start-up Junyo. He started by sharing some stories from his background. My favorite was the story of Zynga's first game, a poker game that became popular enough for there to be a black market for chips. Schoettler and the rest of the team recognized this as an opportunity, a statement from the players that there was a valuable feature to be had if only they could create it—and, of course, monetize it. The talk was generally interesting, though he meandered into making broad claims about education reform that did not sit well with the audience. In particular, he had a direct assault against teachers' unions that drew the ire of many in the crowd. I suspect he will be remembered more for this than for anything else, which is a shame, but also a good cautionary tale to know your audience.

I want to highlight two calls for action that Schoettler included in his keynote. First, he called for a concerted quantification of learning. Given the broad and successful use of play analytics used at Zynga, it follows that if one could similarly quantify learning, one could apply similar data mining techniques. He did not mention the role that badges-for-lifetime-learning play here, and I remember learning at MeaningfulPlay that one of the goals of an open badge framework is to permit academic study of human learning activities. Later, however, he made a call to answer the question of what students actually learn. This strikes me as a complementary but contrasting point to the first, and it dovetails into what I want to bring up in my talk tomorrow. Quantitative methods can only ever answer the question "how much?" In fact, all the statistical techniques I know can only answer "probably how much?" By contrast, "What did they actually learn" is necessarily a qualitative question: each person learns based on his or her unique situation and history. I was hoping that he might recognize and comment on this difference, but by this point, the crowd was too busy cheering for unions and teachers. (And, in truth, it was probably not the right venue for this kind of discussion.)

I attended research-oriented talks all day, and I enjoyed it. The general quality of work was good, in my opinion. In the first session I attended, Douglas Clark presented a meta-analysis of a large body of learning games. I like to see this kind of work, and I was hoping to catch him and ask about how many of the studies address validity and reliability of their instruments, but I didn't have the chance. I was disappointed when an attendee brought up some qualitative studies that he referred to them, offhandedly, as not methodologically rigorous enough to fit into his meta-analysis. It's a shade of this same problem that's been bothering me, of a blind trust in statistics for statistics' sake, as if the quest for statistical significance is really the heart of the problem. Of course, if you're reading my blog, you've probably already read "The Earth is Round (p<0.05)", but I find it doesn't hurt to re-read it and occasionally re-recommend it.

An intriguing part of Clark's talk was an observation that most of the papers they studied either (a) spent a lot of time explaining and justifying their methodology, with almost no discussion of the actual game being used, or (b) described the game design in detail, and only lightly addressed research methodology. He encouraged attendees to consider which of the two groups they tended towards, and to consider the other audience as well when writing papers. This seems like sound advice.

Another speaker in the same session was Michael Lavine who discussed the Games and Learning Publishing Council. Looks like they're not quite ready from prime time yet, but I look forward to following the group.

The sessions are structured so that there are three speakers, and their three talks are given in sequence. Then, the moderator ("discussant") makes some general comments—which seem to often run much longer than necessary—before the audience is invited to ask questions of any of the presenters. I do not care for this model, since it seems that the attempt to put all the talks under one tent rarely works, and usually, the best presentation simply gets all the question. I prefer the approach taken at most of the conferences I frequent, which is 15 minutes to talk, five minutes of Q&A, and then out. One interesting tidbit from the Q&A this morning, though, was from a teacher who suggested that YouTube videos would be the best way for her to consume lesson plan and curriculum ideas for a game. This was interesting and surprising—I am not sure I would have thought of it on my own—and so I wanted to make a note of it here.

In the next session, Scott Nicholson gave a great talk about his playful courses, which is the term he prefers over "gamified." A few good ideas here that I need to think about as I revise my courses for Fall: don't use the word "optional" to describe assignments; multilevel badges (bronze, silver, gold) with unlimited attempts at improvement; and individualized learning contracts were useful.

I was glad to hear a talk by Dan Hickey later in that session. I had previously only known him as the author of one of my favorite education blogs, so it was good to see him in person. (I have not yet walked up and introduced myself as a fan of his blog, but that's on the agenda for tomorrow.) He was reporting on work related to Quest Atlantis, but I think it was his comments on the side that I found most interesting. Here's the best: "All this blah blah about '21st-century skills' is just that: blah blah. The 21st-century skill is writing." He also called for an end to studies that ask whether giving students autonomy increases their motivation: we know it does, so we don't need to keep checking in different contexts. He referenced another scholar, whose work is not familiar to me, in a claim that competition in education is not bad per se, but rather, it's the lack of feedback in competitive environments that hinders learning. His study showed that with good feedback—that is, feedback that is meaningful and that is used—competition can enhance learning. The key thing, it seems, is the feedback. I will have to read the paper to get a better understanding of the supporting research, which seems up my alley.

In a presentation, someone referred to Lord of the Rings Online as a "generic fantasy MMORPG." What does that mean? It's based on the property on which practically all other fantasy settings have their roots. Minor thing, but odd to me that someone who takes games seriously would belittle a game like this as "generic."

The provided lunch was a tasty Indian buffet, an unexpected treat. Dinner included a range of hors d'oeuvres, some pasta stations, very nice desserts, and wine and beer, all around a poster session. I talked to a few poster presenters, and I noticed a high concentration of non-student-presenters. The most interesting conversation I had was around a poster describing a restructuring of Bloom's Taxonomy for game-based learning, a collaboration between Second Avenue and University of Rochester. Sadly, my photos came out all screwy, otherwise I would share them as well as the names of the researchers. We had a good discussion, and I learned that Bloom's Taxonomy is taught quite rigidly as "true" in conventional teacher education. They use teacher's familiarity with this model in order to help them contemplate how game-based learning shakes up the status quo. Our discussion made me realize that my interpretation of Bloom's Taxonomy may not match with many others—particularly those trained in the teacher education tradition. This is good to know as I continue to frame my own ideas.

The conversations, posters, and free booze were all good,  but it was about this time that I got an email from my wife that she was feeling poorly, so I decided to call it a night. Turns out, I got back to the hotel right as a storm was breaking, so the timing was good. Now my wife is resting, the boys are in bed for the night, the blog post is written while ideas are still fresh, and it's time for me to turn in. My talk tomorrow is the last one in the 4:00PM session, so I suspect it will be a coffee-achiever day.

[Day 2][Day 3]

SIGCSE 2013 Conference Report, Day 3

(Day 1) (Day 2)

I had toyed with the idea of skipping the morning session to have breakfast with a friend, but it turns out he had a coauthored paper being presented. I didn't go to his session, but I ended up at the "Computational Thinking" session. Am I glad I went! The session was well-attended considering that Nifty was going on at the same time, and that's always a big draw. What was most exciting about this session was that each presentation was given by a senior researcher. I don't mind grad students giving papers—in fact, it's critical that they learn to do it. It just happened that these papers were all given by very bright researchers, and they had both good insights and good responses to questions.

Alexander Repenning talked about the sustainability of K-12 interventions; that is, how well do they continue once the researchers leave the scene? His results were promising, and his research group is trying to tackle a very difficult problem of scale. As part of his presentation, he included an infographic that took Csikszentmihalyi's Flow diagram and injected Zone of Proximal Development into it, wedging ZPD between "Flow" and "Anxiety." I asked for a clarification, which prompted him to explain fundamental definitions of Flow and ZPD—probably helpful for others to frame the question, but not really addressing my concern. I am not sure that these two theories live in a planar embedding on the skill/challenge axes. If one considers points moving through the Flow space over time (adding time as third dimension), I suspect ZPD would embed into upper-right arcs. I need to think about this some more. Repenning suggested that his group has some methods for tracking student projects through this two-dimensional space, and that it is useful; I should follow up with him to see how they're doing it. I had never really thought of combining Vygotsky and Csikszentmihalyi in this way, but I still have some doubts about the simplicity of the proposed overlay.

The next presentation in this session was given by Yasmin Kafai, who spoke about the social aspects of computing. I got several references from her talk that I should investigate, including Samba Schools and Scratch, Jr. During Q&A, I asked her about her thoughts on the relationship between social learning and individual grading a la GPA and NCLB. She thought for a moment and suggested that badges-for-learning is a valuable effort in this direction. I like this model of badges as a wedge, a way of getting leverage into an otherwise closed system.

The third presenter was David Touretzky, who reported on an effort to teach kids computational thinking with a fast-pased introduction to Kodu, Alice, and Lego NXT. One interesting result was that, in this intensive one-week introduction, the robots took the cake. Touretzky is a robotics researcher and so certainly is not unbiased, but knowing kids... yeah, programming robots would be pretty cool. An interesting aside: Touretzky mentioned that if he could only teach two computational tools to someone, they would be state machines and trees. A beautiful choice, I say—not sure I could do any better.

My talk was in the next session. Although it was called "Capstones," only the first presenter talked about capstones. I am sad to report that my presentation was very sparsely attended. There were less than ten people in the audience besides the other presenters, maybe less than five. At some point, you just try not to count. I find this sad, because I really loved this work and writing this paper, and I know that hearing papers presented is often an inspiration to read them and incorporate their ideas.

On the positive side, I think it was well-received. During Q&A, all the questions were really about the VBC experience, not the assessment model I presented. Someone asked about how well the arts and humanities students adopted the agile methodology, and I explained how it helped to give them an early win, so that they could understand how and why Scrum operated. Another asked about how the relationship with the Children's Museum related to the initiation and execution of the project, and afterwards, he and I had a good discussion about the pros and cons of no-cost student projects with non-profits. In the coding of sprint retrospectives, I used the terms Collaboration, Community, Support, and Learning; one attendee explained after the talk that she thought "Support" meant something completely different until I talked through it. This was great feedback, since the terms only really made sense to me, and her recommendation—"Resources"—really was better than mine.

The lunch was good aside from a rotten tomato in my salad. Since I was the only person out of 1300 attendees who had a sour face and left the room, I think I had the only one, which is not a bad rate. Jane Margolis talked about diversity, it being the ten-year anniversary of the publication of Unlocking the Clubhouse. She mentioned that she had expected to give a rather somber presentation until about six weeks ago, but that now we have exciting news such as President Obama's advocacy of teaching computing to everyone and the phenomenal success of the Code.org video. The numbers for diversity are still low, and she pointed to the dangers of growing too fast in the wrong ways: if we don't take this opportunity to grow Computer Science for all, we risk staying in the shallow end of diversity. In particular, she pointed out how several of the Code.org video celebrities came from privileged backgrounds, which could enforce the idea that programming is some kind of inborn skill as opposed to learn. Also, she pointed out that the Code.org video talked mostly about programming as if it were the goal, ignoring the years of work by the CS Education community in getting people to understand that computing and computational thinking are not only realized through programming.

I typed this day's report from the airport, where I await my on-time departure. Many flights have been cancelled and postponed due to inclement weather, but I think these are mostly smaller planes. Now it's time for me to enjoy a few more minutes on the ground before packing up. Thanks for reading! I already have a few ideas for papers for SIGCSE 2014 in Atlanta...

SIGCSE 2013 Conference Report, Day 2

(Here's a link to day 1.)

I went to the speaker's breakfast this morning and was able to talk at length with Stephen Edwards of Virginia Tech about their use of test-driven development in CS1. Part of his discussion that resonated with me was that students with prior programming experience tend to be the most vocal opponents of TDD, often willing to defend their implementations sans tests as correct. It is noteworthy that Virginia Tech uses Web-CAT for automatic grading of programming assignments, and that this grading includes unit tests and test coverage. I experience this phenomenon in CS222, where the students push back on TDD, instead assuming that the ad hoc methods that got them through CS1 and CS2 are somehow good and universally correct—and no one pushes harder than the ones who had prior experience and easily aced those prerequisites. I notice, however, that their scale is a weakness. At Virginia Tech, they cannot automatically detect whether tests were written first or not, but I have found in CS222, with manual inspection, I can. There are traces that are left in proper TDD, and while I'm not sure I can build a taxonomy of them at this time, I do get a definite sense when reading implementations as to whether TDD was followed. In particular, with an emphasis on "Refactor" as a critical part of "Red-Green-Refactor," code that was tested post-creation tends to have much worse semantic connection between test and production code, and the tests tend to be poorly constructed. When writing just enough code to make a test pass, it's generally obvious. There's probably an empirical research project in here somewhere involving machine learning and automated grading.

This morning's plenary talk by Michael Kölling was excellent. In part, I felt vindicated in his embracing his status as a tinkerer, not a researcher. This was important to me, since there's a sense in which we were intellectual competitors in the early 2000's, not that he would have known it. I developed a visual operation semantics for Java as part of my doctoral work, and I had several pages in my dissertation comparing my notation to his for Blue and BlueJ—of course, with the claim that mine was a superior notation. However, my contour-based approach never had enough traction to make a tool like BlueJ, let alone a textbook, workshops, staff, federal funding, etc. Still, it was good to hear him happily say that he just made something up and it caught on, which I think situates my work well in a historical context as being more driven by issues of soundness and completeness—programming language issues, not pedagogic ones.

Kölling made a fascinating analogy to Haeckel's Recapitulation Theory, summarized as "ontogeny recapitulates phylogyny." Of course, that's meaningless to mere mortals, but he explained it as the idea that human embryos grow through all the stages of evolutionary heredity, which is why they look sequentially like fish, frog, or chicken embryos. This theory is disproven, but he described how well-meaning professors treat Computer Science learning—and programming in particular—as if Recapitulation Theory were true: I learned programming by learning pointer arithmetic, so you must have to learn programming by pointer arithmetic. His bigger point, then, was that in the layers of abstraction in computing, we all have to draw a line somewhere and treat the stuff underneath as a black box. This resonated with me, and I appreciated drawing upon falsified science for the analogy.

My presentation was the first in my session, and I feel it went well. I was able to embellish the presentation with some personal stories that don't show up in the paper, but work really well to "sell" the ideas in the research. I hope that some people are inspired to take a closer look at the paper.

The other two presenters in my session had interesting work, and it was fun to talk to them a little about it. Henry Etlinger drew explicitly upon Contributing Student Pedagogy (as defined by Hamer et al.), which had slipped off of my radar, but which is probably a useful frame for some of my research. Etlinger also described how he organized students into "coalitions": not quite teams, not quite groups. When asked to clarify, he described his taxonomy. To him, a team had a shared mission and a leader, while a group was just an unorganized collection. He used the term "coalition" to frame to students that they had to work together and had to negotiate with each other, but that they didn't necessarily have a holistic team identity. I thought this choice of words and explanation were interesting, though I'm not sure what I'll do with it.

I was able to meet up with Bonnie MacKellar at lunchtime to talk about my CS222 course and compare it to her course, which is designed to meet similar constraints. This inspires me to write something a bit more academic about the design, execution, and evaluation of CS222; watch for this in a future CCSC or SIGCSE submission.

I went to the Rediscovering Passion, Beauty, Joy, & Awe panel after lunch, which I didn't realize was one in a series. It seems it started with Grady Booch's SIGCSE keynote several years ago, which I still remember as one of the best keynotes I have seen. This time around, the theme was innovative and engaging approaches to introductory computer science. The presentations were good, although I regret a bit my question to Mark Guzdial about paradigms for media computing. I am very interested in how learners are impacted by procedural vs. object-oriented introductions to programming, although my one experiment in this area yielded nothing publishable; unfortunately, I think he mistook my question about paradigm for an attempt to start a language war. Media Computing is available in Java and Python, and what I really wanted to know was whether the procedural implementation (Python) offered different learning outcomes from the OO implementation (Java). From his response, I think the answer is no, but I feel bad that I didn't articulate the question clearly, or maybe should have just emailed it to him.

Regardless, I think it is noteworthy that this panel was packed, with hundreds of people in the room, and meanwhile, there were four paper sessions that collectively probably had fewer attendees (or at least close). I think this is symptomatic of what I wrote about yesterday. I would rather have the best and the brightest attending paper sessions and asking hard questions of the presenters. In this way, they can model the perceptions and insights that allowed them to rise to the top of the field. Also, if they get sick of sitting through low-quality research, they have the clout to change the direction of the conference.

After the panel, I spent some time in the exhibit hall, talking to some friends and a few vendors. I spent quite a bit of time at the Broadening Participation Alliance, trying to figure out which of the partnering organizations might be interested in a pet idea I've been kicking around. One of the gentleman I met was Manuel Pérez-Quiñones, who is involved with the Coalition to Diversify Computing. Also, it turns out he is a Ball State alumnus! We're going to try to get him to come and give a talk some time next year.

I attended two more paper presentations after this, and the best story here is that the last presenters were hoping to run slides from their iPad. Even with the proper dongle, the projector would not recognize the device. They tried pulling up a backup presentation from Google Drive, but it was in Keynote, and the presentation laptop was a Windows 7 PC. They were a bit flustered, understandably, and proceeded to give their presentation without any visual aids. It was excellent. I shook their hands afterwards and encouraged them to be confident and always deliver that talk without slides. After seeing bullet points for the last two days, it was refreshing to have a pair of presenters just talk to the audience.

It turns out that their presentation drew a little bit upon Vygotsky's Zone of Proximal Development. This reference in their paper title was my main reason for attending the session, but I was a little disappointed. I had hoped for a more formal theoretical framing of the work in terms of internalization of the instructors' guidance, but from the discussion, it seemed that they simply used ZPD to rationalize scaffolding. I have nothing against good scaffolding, but as I've been reading more about activity theory, I find myself wanting to tie this line of work with CS education in a more formal way.

I returned to my hotel room and brought up my slides for tomorrow's talk, expecting to just do a little mental rehearsal. I was dismayed to see that I had never actually finished them. Whoops. I think I did a good first pass, and then devoted all my attention to the first talk—the one I gave this morning. Revising these will keep me off the street, at least. I visited a few friends who were setting up a workshop for MIT App Inventor, got some seafood chowder and a beer, and now I'm back in the hotel room. Next up: slide revision.