My Notes on "Make It Stick: The Science of Successful Learning"

Several weeks ago, I finished reading Make It Stick by Brown, Roediger, and McDaniel. It was recommended to me by a good friend with a heart for improving education. The book aims to explain what we know about learning from cognitive science and how this can impact the practices of teaching and learning. I found the book to be inspirational, and I mentioned the book in several recent essays and presentations. I happened to meet local cognitive science and student motivation expert Serena Shim this semester, and she affirmed the findings and value of the text as well.

One of the most important findings that came up throughout the book is that spaced practice is better than massed practice. I think we all recognize that it is true: of course studying throughout the semester is more effective than cramming. However, the science is more nuanced. Massed practice is actually better for short-term recall than spaced practice, but spaced practice is better for long-term recall. This has a fascinating corollary: if our courses contain high-stakes tests, then it is a good tactical decision for students to cram.

This implies to me then that we instructors have to make a real choice between I want students to pass this test and I want students to remember this a year from now. I have a rule of thumb that I have only recently had to articulate, which is that I only want to teach content that I think students should know in five years. My general pedagogic approach favors spaced practice, but perhaps I can do more to support this. However, recent conversations made me realize that this perspective is not universal. I was involved in a somewhat heated discussion about a master syllabus revision with a colleague. The particular syllabus had, in my opinion, too many low-level learning outcomes. I argued that students don't learn these items, and he argued that they do. As evidence, I cited that they could not repeat their achievements a year after taking the class, and as evidence, he cited that they passed the final exam. Here are the loggerheads of higher education: we both believed the other to not just be wrong, but to be holding the wrong value system.

I was reminded by the text of the value of testing as retrieval practice. I had read this before but tried to dismiss it; however, the presentation by Brown et al. makes it hard to ignore. Learning improves through retrieval practice, and testing is perhaps the simplest way to practice retrieval. I mostly gave up on using tests many years ago, favoring instead continuous authentic work. However, I also see my students not remembering to apply fundamental lessons early in the semester into their work later in the semester. I need to review my use of quizzes and tests, as well as how I prepare students to do their own self-testing.

Another theme of the book that knocked my proverbial socks off was that immediate feedback is not always better than delayed feedback. I think that in the educational games community, it is taken for granted that feedback is simply good, and that quicker feedback is better feedback. As the argument goes, if feedback is good for learning, and games are feedback machines, then games can be good for learning. This is not wrong, but it is also superficial. Not all feedback is created equal. The authors cite studies that show that delayed feedback can lead to better learning. As I understand it, the actual reason for this is not understood, but the prevailing hypothesis is that immediate feedback makes the feedback indistinguishable from the task itself; this leads to a result where when the feedback is not present, knowledge of the task breaks down. This sounds an awful lot like "I can do it in the game, but I cannot do it outside the game." I wonder how many empirical educational game research projects have investigated feedback delay as a dependent variable, and if not, how one would construct such a study. After all, a player expects that if they press 'A', Mario should jump right away.

Reading the section on delayed vs. immediate feedback made me think of two other salient examples where immediate feedback may be causing problems. The endemic one is automatic spellcheck and grammar check: we all know that students do not learn to spell or write by letting their word processor do the work, it just builds a reliance on the word processor. The other, related example is IDE for novice programmers. As with automatic spellcheck, the IDE will add red squiggles to invalid code, and students can right-click on it and change it to whatever the IDE wants—often without regard for whether it is what they should want.

Chapter 8 of the book provides a series of helpful summaries that are organized for different reader demographics. It's a valuable chapter, and so I will spend a bit of time on it here describing what caught my attention and where I think it should take me. In the section for teachers, they recommend explaining to students how learning works. The following quotation is a good overview:

• Some kinds of difficulties during learning help to make the learning stronger and better remembered
• When learning is easy, it is often superficial and soon forgotten
• Not all of our intellectual abilities are hardwired. In fact, when learning is effortful, it changes the brain, making new connections and increasing intellectual ability
• You learn better when you wrestle with new problems before being shown the solution, rather than the other way around
• To achieve excellence in any sphere, you must strive to surpass your current level of ability
• Striving, by its nature, often results in setbacks, and setbacks are often what provide the essential information needed to adjust strategies to achieve mastery

Another tip for teachers is to teach students how to study. This has been on my mind quite a bit, along with the question, "Where does the buck stop?" I teach primarily junior and senior undergraduates, and I estimate that 5% of them have any real study tools. Indeed, I think a good description of the Ball State demographic is, "Students who are smart enough to have gotten this far without having developed study skills." Including direct instruction on study habits is an investment in their future learning, but I doubt I would be able to reap it in my own courses, so it's taking away from time on topic. More frustratingly, I have seen for years that I can teach good processes for learning and software development in a course like CS222, only to see that a year later, the students have never touched any of those techniques because other faculty do not expect them to. For example, I can teach the value of pair programming or test-driven development, present the students with research evidence that these increase productivity, and require them to deploy these techniques; but a year later, when I ask them to do these in a follow-up course, they say that they have not used these since CS222. Why should I be more optimistic about study skills, when inertia is powerful and habits are so hard to override?

The section of tips for teachers returns to the theme of "desirable difficulties" that came up throughout the book. Here are some specific desirable difficulties that they recommend:

• Frequent quizzing. Students find it more acceptable when it is predictable and the individual stakes are low.
• Study tools to incorporate retrieval practice: exercises with new kinds of problems before solutions are taught, practice tests, writing exercises reflecting on past material and related to the aspects of their lives; exercises generating short statements that summarize key ideas of recent material from text or lecture.
• Quizzing and practice count toward course grade.
• Quizzing and exercises reach back to concepts and learning covered earlier in the term.

Again, this is a valuable summary. Each of these items is covered in the text with explanation and citation. It's clear what actions can come from this list as well, and it makes me look at opportunities in my upcoming HCI class in a new way. I also recognize in it the value of several things I already do in the class, such as having students connect readings to their experience and writing reflections of development experiences. Given that I tend to divide semesters into a content-oriented first half and project-oriented back half, I need to be more conscientious about designing assignments and quizzes that reach back to the early part of the first half; this should help students deploy these ideas more readily in the second half.

The final bit of advice in Chapter 8 is to be transparent with students about incorporating desirable difficulties into the class. I have always been a fan of white-box pedagogy, although it's not every semester that I see students take interest in why I am teaching the course the way that I am. Student teaching evaluations often reveal quite mistaken models about my intentions as well. Sometimes I get these excellent reflection sessions as I described in Fall's HCI class, but the irony here is that they generally come after students have completed their evaluations.

I highly recommend Make It Stick. It is written clearly and precisely and organized in a way that emphasized the important points. Crucially, it avoids educational fads in favor of empirical research. Chapter 8, as I have said, provides a great synopsis that turns the ideas of the book into potential action items for practice.