Rubric-based self- and peer-evaluation on team programming projects

I have written before about grading rubrics, and I am taking this opportunity to share some of my recent developments here. In my game programming class (CS315), I have eleven students working together on one project. We're using Scrum, and it's still going well. The early part of the semester was ungraded, formative, small-project work, and I found myself needing a way to assign grades to the students. It may be worth mentioning that I don't actually want to assign grades to them, but it is a requirement of the course, and so I must do it responsibly.

I happened to stumble across a rubric I created in 2008, before my blogging days, while I was cleaning up my Web space. As I recall, this was an amalgam of several other rubrics I had found on the Web with an ounce of invention. This was my inspiration to create the following rubric:

	3	2	1	0
Commitment	Attended all scheduled team meetings or notified the team of absence.	Missed team meetings, with notifications, with enough regularity to be problematic.	Missed one or more team meetings without notifying the team.	Regularly missed team meetings without notifying the team.
Participation	Contributed to sprint planning, sprint review, and sprint retrospective meetings.	Did not contribute to one of the following: sprint planning, sprint review, or sprint retrospective meetings.	Did not contribute to two of sprint planning, sprint review, and sprint retrospective meetings.	Did not contribute to any of sprint planning, sprint review, and sprint retrospective meetings.
Communication	Clear standup reports on what was accomplished, what is planned, and identification of impediments, to allow the team to continue to make progress.	Some unclear standup reports that inhibit team progress.	Regularly unclear standup reports that inhibit team progress.	Did not give standup reports.
Technical contributions	High quality technical contributions that facilitate success of the team.	High quality technical contributions that do not directly facilitate the team's success.	Low quality technical contributions that frequently require redress by other team members.	Low quality technical contributions that inhibit success.
Attitude and Leadership	Listens to, shares with, and supports efforts of others, and actively tries to keep the team together.	Listens to, shares with, and supports the efforts of others.	Frequently fails to listen, share, or support teammates.	Displays an antagonism that inhibits team success.

I created this after the team's first sprint, and parts of it are reactions to problems the team was having. For example, in the team's first sprint retrospective, they identified that many had spent inordinate time on aesthetic issues that were unrelated to the user stories. Hence, there is a distinction between those technical contributions that facilitate success and those that do not. Note that the numbers across the top relate to my triage grading approach; some day I will write a blog post about my experiences with it, but for the time being, suffice it to say A=3, C=2, D=1, F=0.

I shared this rubric with the team during Sprint 2, and at the end of the Sprint, I asked each person to do a personal evaluation. This was designed as a formative evaluation for the student as well as a trial run of the rubric. From reading the students' self-evaluations, I think the instrument worked as intended, and we're going to do self- and peer-evaluations at the end of this sprint.

In my other course (CS222), students are working in teams of four on six-week projects of their own design. They just finished their mid-project milestone presentations, and I've modified and distributed the rubric to this other group as well. I've highlighted those cells that have changed.

	3	2	1	0
Commitment	Attended all scheduled team meetings or notified the team of absence.	Missed team meetings, with notifications, with enough regularity to be problematic.	Missed one or more team meetings without notifying the team.	Regularly missed team meetings without notifying the team.
Participation	Contributed to project planning, implementation, testing, and milestone presentations.	Did not contribute to one of the following: project planning, implementation, testing, and milestone presentations.	Did not contribute to two of the following: planning, implementation, testing, presentation.	Did not contribute to three or more of the following: planning, implementation, testing, presentation.
Communication	Clear reports on what has been accomplished, what is in progress, and what stands in the way, thereby facilitating progress.	Sometimes is unclear about what has been done, what is in progress, and what stands in the way, creating minor impediments to progress.	Is regularly unclear about what has been done, what is in progress, and what stands in the way, creating significant impediments to progress.	Communication patterns directly disrupt team progress.
Technical contributions	High quality technical contributions that facilitate success of the team.	High quality technical contributions that do not directly facilitate the team's success.	Low quality technical contributions that frequently require redress by other team members.	Low quality technical contributions that inhibit success.
Attitude and Leadership	Listens to, shares with, and supports efforts of others, and actively tries to keep the team together.	Listens to, shares with, and supports the efforts of others.	Frequently fails to listen, share, or support teammates.	Displays an antagonism that inhibits team success.

These students are not using and are not familiar with Scrum, and so the Scrum-specific references needed to change. I am not that happy with the articulation of participation above, in part because we invested a lot of time on test-driven development, and I fear the students will be confused about separation of testing and implementation. The "communication" entry is clearly influenced by the idea of the Daily Scrum, but I have not previously suggested this particular reporting strategy to the students.  My hope is that by asking them to conduct rubric-based self- and peer-assessments, they will gain some insight into better ways to organize themselves. As with the game programming students, I am using this first pass as a formative assessment, and also as an instrument to help me identify teams that are having problems with team members—an unfortunate but oft-inevitable situation.

The students in CS222 had their first group experience with rubric-based evaluations this week when they evaluated each others' milestone presentations using this instrument:

Presentation requirement	Satisfactory (3)	Probationary (2)	Unsatisfactory (1)	Unacceptable (0)
Executable release	Stable and appropriate demo of executable release	Unclear demonstration of stable executable release	Unstable executable release	No demo
Milestone #1 Estimate	Clearly met milestone estimates	Clearly met a reasonable subset of milestone estimates	Did not meet an adequate subset of milestone estimates	Progress on initial estimates was not addressed
Milestone #2 Estimate	Clear presentation of well-articulated estimates for next milestone	Clear presentation of poorly-articulated estimates for next milestone	Unclear presentation of poorly-articulated estimates for next milestone	Milestone #2 estimates not addressed
Software architecture	Good presentation of reasonable software architecture	Decent presentation of reasonable software architecture	Unclear or awkward software architecture	No presentation of software architecture

I printed these on small slips of paper. On Tuesday, we had four presentations, so I instructed the students to pick up four slips as I talked through the structure and purpose of the rubric. On Thursday, we had five presentations, so I left the slips in the same place in the back of the room and I put on some mood music.

I was a little disappointed that none of them stopped to tell me how incredibly clever this was.