What they think they learned, Spring 2011 edition

After two weeks of frantic activity and a week of relative relaxation, I arrive at Memorial Day with an interest in wrapping up some reflections about the Spring semester.

I ran the final meeting of Spring's CS222: Advanced Programming class similarly to Fall's, which I wrote about back in December. First, the students brainstormed a list of the 80 things they thought they learned through the course. Then, each student was given three votes, and we voted on the most significant items. It should be noted that the students had freedom to choose how they determined what it meant for learning to be related to the class, and so not all of the items fall directly from in-class interactions; similarly, the students could choose any heuristic for voting. The top earners with their total vote tally are shown below.

Item	Votes
Object-oriented design	6
Test-driven development	5
Professional tips	4
Time management	3
Model-view separation	3
Refactoring	3
Mercurial	3

These are all self-explanatory except perhaps "professional tips." We spent a good deal of time in the course practicing and reflecting on how we learn the best practices for a language and environment. I used Java as a language of instruction, and everyone had a copy of Joshua Bloch's Effective Java, second edition. Early in the semester, I gave carefully-constructed assignments in which the students would read portions of Effective Java and apply them. As the semester went on, I loosed the reins, and by the end of the semester, I had the students searching through the book to find tips to apply in their six-week projects. I hope and believe that this kind of inversion of control was useful in helping the students learn how to be better learners.

"Time management" may be worth another mention, since my approach to this was pragmatic. There were few directed exercises that were related to time management, the most prominent being requiring projects to be completed in multiple deliverables. Students, as a subset of humans, procrastinate. Making someone take an incremental and iterative approach can have a huge impact in how they think about their own work, especially when it's the first time having to do this in a software development environment.

After this convergent thinking activity, we returned to divergent thinking as I asked the students to brainstorm on how they learned these things. As in Fall's class, the students seemed to have a harder time answering this question, and I don't think it's just because they were tired. I suspect that, despite all my efforts to foster reflective practice, the students spend most of their cognitive energy on content rather than learning processes. I need to try to find more ways to help students make the leap to metacognition, although I acknowledge that I am limited in what I can do in this sophomore-level course. Regardless, here are the only three of the 22 that earned any votes when each student was given one vote:

Item	Votes
Practice / Doing	7
Working in a group	5
Failing usefully / learning from mistakes	2

Clearly, all three of these are closely related, and they all deal with learning-by-doing, as I expected. Of course, the course was framed around constructionist and studio-based learning principles, and so it's no surprise that students would say they learned from these activities—it's practically all they did!

Here's the part that surprised me: in the list of 22 items, reflective writing did not come up. The students did a lot of reflective writing. The most prominent example was the learning objectives document, which I wrote about in April. In a nutshell, this collaboratively-written document was the medium by which the students' learning was to be communicated with me, and it was explicitly designed for reflective practice. Yet, even at the end of the semester when thinking about their own learning processes, and even when I was trying to steer their thinking towards this while moderating the brainstorming session, no one even mentioned this reflective writing. Far from meaning that this writing was unnecessary, I think this echoes my earlier point, that (generally) the students were not savvy at moving between multiple levels of cognition, from the content, to the thinking about the content, to metacognition.

The good news is that this was an experimental class, part of a research study on how collaborative writing impacts learning in STEM. My co-PI and I have a large amount of data to analyze, and hopefully we can gain insight into what the students experienced and how they made sense out of it.

A Day with Autobeans (or, JSON vs XML for data representation in GWT, Part 2)

I spent much of the day yesterday tinkering with AutoBeans, a technology that I stumbled across while trying to find a solution to the problems described in yesterday's post. (Thanks, StackOverflow!) The tagline on the AutoBeans describes exactly what I desire: structure, not boilerplate. The problem I always have with XML is that it takes so much plumbing to get the data into the domain model. Keep in mind that I am going to have content experts creating the data files manually, and so ORM tools, Hibernate, etc. won't solve my problem.

Structure

AutoBeans provide a mechanism for automagically reading JSON data into domain objects. I put together a functionally-equivalent demo akin to the XML demo. Specifically, I have a data file that contains the definition for an image and a "hotspot," a region of the image that generates a message when moused over. The data file looks like this:

{
    'image':'Jellyfish.jpg',
    'hotspots': [
        {
            'bounds': { 'left':10, 'right':50, 'top':10, 'bottom':50 },
             'message': 'You found it!'
        }
    ]
}

As before, this file sits right alongside my client source code and is referenced in a clientBundle, to wit:

public interface JsonResources extends ClientBundle {
    
    static final JsonResources INSTANCE = GWT.create(JsonResources.class);

    @Source("demo.json")
    TextResource place();
}

Setting up domain objects using AutoBeans is as simple as it should be: create interfaces using the traditional Java beans naming conventions. My three domain interfaces are shown below.

Compare these to the JSON above, and you can see that there's a strong parallelism: they are essentially two representations of the same semantic model.

The Hotspot and Place are "simple" interfaces, using the nomenclature of AutoBeans. They only contain bean-style accessor and mutator methods, and so they can immediately be deserialized from JSON. The Rectangle class is not simple since it has the contains method. The AutoBean documentation describes how to deal with this: create a "category" that provides an implementation of the interface's instance method as a static method with an extra parameter, as demonstrated below.

public class RectangleCategory {
    public static boolean contains(AutoBean<Rectangle> instance, int x, int y) {
        Rectangle r = instance.as();
        return r.getLeft() <= x //
                && r.getRight() >= x //
                && r.getTop() <= y //
                && r.getBottom() >= y;
    }
}

The last piece of the architecture is the AutoBean factory that wires Rectangle objects into their category. The critical piece for Rectangle is the @Category annotation.

@Category(RectangleCategory.class)
public interface MyAutoBeanFactory extends AutoBeanFactory {
    AutoBean<Rectangle> rectangle();

    AutoBean<Place> place();

    AutoBean<Hotspot> hotspot();
}

This completes the model and serialization framework. The only other critical piece is the UI, which is done with a custom Composite that uses constructor injection to receive its model, a Place. Hence, the whole system is contained in the following class diagram.

I modified my EntryPoint class to use this rather than my SimpleMessage approach, and it is short and sweet:

public class JSTest implements EntryPoint {

    private MyAutoBeanFactory factory = GWT.create(MyAutoBeanFactory.class);

    @Override
    public void onModuleLoad() {
        AutoBean<Place> placeBean = AutoBeanCodex.decode(factory, Place.class,
                JsonResources.INSTANCE.place().getText());
        PlaceWidget widget = new PlaceWidget(placeBean.as());
        RootPanel.get().add(widget);
    }
}

Analysis

One of the problems of dealing with GWT is that there can be a lot of silent errors. There's a sense in which mixing statically- and dynamically-typed code gives you the worst of both worlds. Then again, it might be my own blinders from dealing almost exclusively with statically-typed systems. For example, when I first wrote out the JSON, I called the bounding area 'rect', but when I wrote the domain object, I decided to be more explicit with my semantics and called the accessor getBounds. The AutoBeans system happily chugged along and gave me no warning that I was trying to access a field that was not there, because Javascript doesn't care about that.

The more aggravating error arose from a misconception about JSON. I was in a situation where my rectangles were being loaded, but with the wrong values: (0,0,0,0) instead of my JSON-specified values. I set it aside and played some Witcher 2, which you should buy DRM-free from the good people at gog.com, and then I decided to try writing unit tests to capture the error. The test is very simple, but I'll post it here for the sake of discussion:

public class RectangleTest extends GWTTestCase {

    @Override
    public String getModuleName() {
        return "com.example.archaeology_adventures";
    }

    public void testDeserialize() {
        MyAutoBeanFactory factory = GWT.create(MyAutoBeanFactory.class);
        
        AutoBean<Rectangle> rectBean = AutoBeanCodex.decode(factory,
                Rectangle.class,
                "{'left':10,'right':11,'top':12,'bottom':13}");
        Rectangle r = rectBean.as();
        assertNotNull(r);
        assertEquals(10, r.getLeft());
        assertEquals(11, r.getRight());
        assertEquals(12, r.getTop());
        assertEquals(13, r.getBottom());
    }

}

This test passes, but origially I was specifying the JSON literal thus:

"{'left':'10','right':'11','top':'12','bottom':'13'}"

I thought that all JSON objects were string:string pairs, just because that was my recollection from last Summer's Wave API experimentation. It turns out that if you deserialize that second JSON literal using AutoBeans, it will interpret strings as having the numeric value zero, and hence, give rectangle (0,0,0,0). Again, because this is dynamically typed, there is no warning or error about this conversion: you, the developer, just have to know that this is part of the language semantics. I assumed that there would be some kind of implicit conversion that doesn't happen. Removing the erroneous apostrophes solved this unit test's problem, and removing them from my experimental code fixed the problem there as well.

Conclusions

I am pleased that this technology provides a nigh-seamless way to go from a data representation to domain objects. Because there is little boilerplate code, it should permit rapid iteration much more than hand-parsed XML. It's possible that there's an analogous tool for XML of course, and if you know of one, please share it.

Coming back to the original problem, these data files will be created by non-programmer content-experts, although they will be collocated with the development team. Hence, one of the outstanding problems in the JSON/XML consideration is which of the following is more sensible to non-programmers:

{
    'image':'Jellyfish.jpg',
    'hotspots': [
        {
            'bounds': { 'left':10, 'right':50, 'top':10, 'bottom':50 },
             'message': 'You found it!'
        }
    ]
}

or

<place>
    <img src="Jellyfish.jpg"/>
    <hotspot>
        <rect x1='10' y1='10' x2='50' y2='50'/>
        <text>You found it!</text>
    </hotspot>
</place>

As always, I welcome your comments.

JSON vs XML for data representation in GWT

(This is Part 1 of a series. See also Part 2.)

In a few weeks, I will be working with my Morgan's Raid collaborator Ron Morris (History), Mark Groover from Anthropology, and about ten undergraduate students to develop a prototypical digital archaeology simulation. The intent of the project is to create a technological tool to teach 4th-graders about historical archaeology, that it is a scientific process and more than just a dig. I will not dwell on the project design here, but rather I would like to share some of my experimentations with JSON, XML, and GWT.

We want to have the widest possible adoption, and since the expected interactions are fairly simple, making this a Web application seemed best. GWT stands out as an excellent candidate technology since it handles cross-browser issues better than any of our team can. Flash would be a contender, of course, but I ruled it out early since I don't know enough Flash/Actionscript to be confident in leading the team in an appropriate software architecture.

We will only have five weeks to develop the prototype, and the team will include a technical team (mostly Computer Science majors and minors) and a content team (mostly Anthropology and History majors). Unlike Morgan's Raid, in which we could create content directly in Java since the designers were also developers, this project would benefit from an intermediary domain-specific language that the designers can use. An architecture I had kicked around in my head involved using Ruby to define an internal DSL that would be processed in GWT. However, it's not possible to put JRuby into a GWT project since the GWT compiler would try to transform the whole kit and kaboodle into Javascript, and that's not possible or even sensible. I'll just have to find another project in which to experiment with Ruby DSLs; for now it's off the table.

Javascript

Considering a data representation layer, I started by considering how Javascript could be used directly for configuration-based programming. That is, the technical team could write an interpreter on top of Javascript data created by the content team. I played with JSON last Summer when working with the Wave API, and it seemed like a nice way to represent domain objects. Overlay types provide a well-documented approach for using JSON to represent domain objects in GWT. As a proof of concept, I whipped up a simple message wrapper. The interface looks like this:

@SingleJsoImpl(LittleMessageImpl.class)
public interface LittleMessage {

    /**
     * Get the text of this message.
     * 
     * @return message text
     */
    public String text();
}

The annotation references another class that is shown below. Note that Google's documentation, as of this writing, does not make this relationship clear, but a little tinkering with the annotation value revealed the proper approach.

public class LittleMessageImpl extends JavaScriptObject { 

    /**
     * Create a {@link LittleMessageImpl} from JSON text.
     * @param json the JSON string
     * @return new message object
     */
    public static final native LittleMessageImpl buildMessageFromJSON(String json) /*-{
        return eval('(' + json + ')');
    }-*/;
    
    // Required for the GWT compilation process
    protected LittleMessageImpl() {}
    
    /**
     * @return the text of this message
     */
    public final native String text() /*-{
        return $wnd.checkNotNull(this.text);
    }-*/;

}

The two important bits are the factory method, which uses native Javascript to evaluate a JSON string into an object, and the Javascript native implementation of the text() method.


In debugging this program, I encountered a problem where I had called the field by two different names, and of course because Javascript is dynamically typed, I had no compiler support to detect this. To ease debugging, I wrote a nigh-trivial Javascript method, checkNotNull, that behaves like Preconditions.checkNotNull from Guava: if the argument is not null, it is returned, and if it is null, it bombs out. I put this method into a file called preconditions.js that lives in the "public" folder of my project. Making this work took some trial and error, but the crux of it is this: if your application's root package is com.example, then you can put a folder under com/example/public and put resources there, and these resources will be loaded prior to your GWT applications' execution. This is defined in the documentation, although I had a hard time extracting the previous sentence's meaning from what was given.

Coming back to LittleMessageImpl, the json itself sits right in the application: it is not the result of a request to a server, as JSON is frequently used. I made a file message.json that sits in my com.example.client package, and it is referenced as a text resource in my JsonResources class:

public interface JsonResources extends ClientBundle {
    
    static final JsonResources INSTANCE = GWT.create(JsonResources.class);
    
    @Source("message.json")
    TextResource message();

}

Now, to create my LittleMessage, I need only do this:

LittleMessageImpl message = LittleMessageImpl
                .buildMessageFromJSON(JsonResources.INSTANCE.message()
                        .getText());

Like many things, it's easy once you know how.


This was a helpful process for learning how GWT works, but I encountered a problem soon afterwards that perhaps I should have foreseen. One of the actual problems I want to solve with GWT is to have an image that has "hotspots" where, when they are moused over, something happens. This is trivially done in GWT using Image and MouseMoveHandler. The problem, however, is granularity of representation. I would like to be able to represent the image as a composite of hotspots that define geometric regions and actions. I could structure the whole thing as a JSON object, but in Java, I want it broken down into pieces, an InteractiveImage holding zero or more Hotspots, for example. This is where my knowledge of Javascript breaks down. It's not clear to me how I could take a JSON string and, with the same kind of elegance as simply evaling it, end up with a beautiful composite object. Two-pass parsing is of course an option, where the first pass makes a "dumb" Java representation that is then converted into a better domain model, but then you lose the elegance of the overlay types.


From here, I considered diving into Javascript, JSON, and GWT a bit more deeply, but I decided instead to go on a tangent. It's all experimentation, after all. Why not try that nasty old de facto standard for Web application data representation, XML?

XML

GWT has good XML support. It has to. Plus, the XML parser looks exactly like Java's XML parser that I've used a good many times. I had built some confidence from my Javascript experiments and decided to try to solve a real problem this time. My data representation in XML looks like this:

<place>
    <img src="Jellyfish.jpg" />
    <hotspot>
        <rect x1="10" x2="50" y1="10" y2="50">
        <text>You found it!</text>
    </rect></hotspot>
</place>
</code>

I happened to be on Windows when writing this, so I opened up my sample pictures folder and found a nice jellyfish image to use. (Why was I in Windows? To minimize time between experiments and Witcher 2.) Like message.json, I put this file—demo.xml—right in my com.example.client folder, and I load it as a text resource analogously.


Without overlay types, I need to do my own XML parsing. I took what I consider to be a standard approach, making a domain object called MouseablePlace that extends Composite and gave it a public static parseXML(Document) method. This then does rather mundane building of a MouseablePlace from the configuration data, but the result is a beautiful MouseablePlace which contains a series of Hotspot objects, each of which contains a Rect, and these are pulled from the XML via ad hoc recursive descent parsing. To the MouseablePlace is attached a MouseMoveHandler that checks whether the mouse is in any of the Hotspots, and if so, the Hotspot's message is shown in a status label on the screen.

Next steps

Both XML and JSON are contenders to represent the domain objects in this application. For that matter, we could also use an internal DSL in Java, but I would rather keep the learning curve as low as possible for the content team. Whether XML or JSON is more sensible to non-programmers, I have no idea. If we had nore time, we would wrap the whole thing in an editor, but we do not have that luxury.

If you have any experience or suggestions, please feel free to share them in the comments.

[ADDENDUM]
Go to Part 2, which covers my experimentation with AutoBeans.

Inquiry

"Conventional wisdom holds that different people learn in different ways. Something is missing from that idea, however, so we offer a corollary: Different people, when presented with exactly the same information in exactly the same way, will learn different things. Most models of education and learning have almost no tolerance for this kind of thing. As a result, teaching tends to focus on eliminating the source of the problem: the student's imagination."

Douglas Thomas and John Seely Brown, A New Culture of Learning, 2011, p79.

Education and Industry

Developing software is the process of learning how to develop that piece of software. As in any craft, there are similarities among projects, yet each one is different and each one enriches the creator.

If learning is an explicit goal of software development as well as the means to its end, then what is the difference between industry and education?

The best hypothesis I have is that there's a value-oriented ordering of goals. In business, shipping a product has preference over improving the team; in the academy, improving the individual has preference over completing a project.

As we transition higher education toward a more enlightened, learning-centric model, I think that it will become increasingly important for us to address these issues—especially considering decreasing public support and the administrative desire for additional revenue streams. Are my students more or less than my apprentices?

No answers, just more questions.

Teaching Assessment Culture to Undergraduates

My CS222 students are in the middle of their six-week projects, in which they just finished the first of two required iterations. There are three teams, with 4–5 members per team. Last time I taught the course (which was also the first time the course was taught, by me or anyone else), I continued to give rigorous assignments throughout the six week project. This time, I decided to try turning the accountability structure on its head and making the students think about how they would achieve the learning objectives of the project. This inversion of control was designed to foster reflective practice and metacognition.

At the beginning of the project, the students were told that they would have to display competency in the following concepts and technologies through their six-week projects:

• Requirements analysis
• Unit testing
• Acceptance testing
• UML: sequence diagrams, class diagrams
• Design patterns
• Architecture: model-view separation with MVC or Holub-style model-view-presenter
• Logging
• OOP: encapsulation, polymorphism
• Iterative and incremental development
• Distributed version control with Mercurial
• User-centered design
• Programming conventions
• Application of expert tips (e.g. Effective Java, Pragmatic Programmer)
• Estimation of time to complete tasks, including feedback for improvement
• Effective team communication
• Technical presentation
• Reflective practice

Several of these had been discussed in the preceding eight weeks of the semester, but many had not.

Each team had to create a "learning objectives document" and share it with me via Google Docs. The document was required to express how the students intended to show competency in each of these areas. They were instructed that this would be a living document: throughout the semester, the teams should come back to the document and consider how they would demonstrate competency.

For the first three-week iteration, the documents were rarely updated by the teams. I was using in-class meetings to introduce topics from the list that the students had not yet seen, scaffolding these with some light readings while the students focused on getting their first milestone delivered. Immediately following the milestone deliverable, the students were given the assignment to update their learning objectives document, ensuring that all items have clear and explicit plans for how the team intended to demonstrate competence.

As I looked over their submissions, at first I was disappointed that there was so little of substance. For example, under the topic of "effective team communication," most of the teams said something like, "we get together at regular times to work as a team." In my comments on their work, I suggested that they had missed the target: working together is a requirement, but learning to work together effectively is a learning objective. To show the latter requires evidence.

I believe it was that particular case—though certainly not in isolation—that made me realize a connection between what I was asking the students to do and what I myself have to do in terms of assessment. I will give some background in order to explain the case.

It was about three years ago, while serving as chair of my department's Promotion and Tenure Committee and therefore also representing the department to the College of Sciences and Humanities Promotion and Tenure Committee, that I read Scholarship Reconsidered and Scholarship Assessed. I loved these books, finding in them clear articulations of my intuition. I have used the framework from Scholarship Assessed within much of my work with undergraduates, following the principle that if the point of higher education is to be exposed to scholars and scholarship, then having students become scholars themselves is the best way for them to understand it. These books also made me very cold on the devious connotations of "research," as used throughout the institution of higher education.

Since last year, I have been departmental assessment coordinator. My university, like most others, is attempting a cultural shift towards more regular, formal, and fruitful assessment. In fact, the Future of Education Task Force suggested that we become obsessed with assessment and continuous self-improvement. This is a good thing, and I wholeheartedly agree with the mission. The downside is that I have had to attend umpteen workshops and meetings that re-introduce the fundamentals of assessment, presented to dubious or antagonistic faculty (who, I can only presume, have good intentions but see no incentive to change decades of practice). I have also served on the College of Sciences and Humanities Curriculum Committee, which means I have read tens of assessment plans for new and revised courses. Suffice it to say that I've been bumping into assessment pretty much everywhere I go.

Now, when I look at the task I gave my students, it was clearly an assessment task—yet, as it was in the guise of teaching rather than service or research, I had not consciously connected it to my past experience. Looking at how the students tried to approach the task, I see the same kinds of confusion that is common among faculty and staff who have not adopted a mindset of incremental and iterative development, a perspective that views problem-solving as a form of model building that requires positive feedback loops.

This insight made it easier for me to evaluate their work as novice assessors. I do not think that I adequately addressed how assessment works—that is, how initiating and fostering a feedback loop of self-improvement will pay off in pragmatic terms. The students' learning objective documents did not adequately provide evidence, but without seeing the bigger picture, they had no reason to believe that evidence was worth gathering. This might be best exemplified in a student's response to a note I left in the document asking for more evidence: the student modified the corresponding part of the document and then asked, "Does this satisfy your requirements?" My Socratic response was, "Does it help you demonstrate whether you have met the learning objective?"

While there have been some stumbling blocks in this approach, I am happy with the overall shape of it. Part of my current research is an investigation into the role of writing in STEM classrooms, and this CS222 happens to be part of the experiment. I will be teaching the class again in the Fall, and I am now considering how such a learning objectives document might be introduced earlier in the semester. This should help students get the idea that they are responsible for their learning, putting them on the path of reflective practice as early as possible.

"Object-oriented"

Reviewing papers for an upcoming conference reminded me how difficult it is to communicate when technical terms lose their meaning. "Object-oriented" is frequently thrown around without so much as a wink or a nod—much less a citation to indicate what someone actually means.

In CS222: Advanced Programming, I have my students read a segment from the first chapter of Holub on Patterns: Learning Design Patterns by Looking at Code. This is one of my favorite books on design patterns and on design in general. Holub adopts a pedagogically sound perspective: rather than present the patterns in isolation as in the Gang of Four book (a manner in which they never occur "in the wild"), he demonstrates the patterns as collaborating parts of a system design. Additionally, the first two chapters of the book provide an excellent introduction to the field of patterns and object-oriented design. On pages 13-14, Holub provides a set of heuristics that can be used to determine if a system is, in fact, object-oriented at all. Directly quoting:

• Objects are defined by "contract." They don't violate their contracts.
• All data is private. Period. (This rule applies to all implementation details, not just the data.)
• It must be possible to make any change to the way an object is implemented, no matter how significant that change, by modifying the single class that defines that object.
• "Get" and "set" functions are evil when used blindly (when they're just elaborate ways to make the data public).

Note that he does not say that these are sufficient for good object-oriented design, but rather that a design that does not satisfy all of these is not an object-oriented design. Holub asserts very clearly that this is not a value judgement: OO is not better than procedural. His point is that when he says "object-oriented," he wants to make it very clear what he is talking about. By his definition, almost all ostensibly object-oriented systems that I have seen are actually hybrid designs.

I find Holub's zeal appealing because he leaves no room for guesswork. You can agree with him or disagree with him, but there is no ambiguity in his statement, and it is sufficient foundation for him to build the remainder of the book. Let's not forget the lesson of Humpty Dumpty in Through the Looking Glass:

'When I use a word,' Humpty Dumpty said, in rather a scornful tone, 'it means just what I choose it to mean — neither more nor less.'