Or so I hypothesize. and I certainly hope so.)

My guess, based on the years I've spent as a TA in college engineering classes (and far too much time grading first-year math and physics problem sets): anything that can be done to reduce one's evaluation workload is extremely tempting to a teacher. And maybe there's a better word for this, but the one I know for "things that let you quickly monitor how somebody or something's doing" is a dashboard.

The intent of this is not to automate grading, but to let teachers draw on a richer pool of data on which to base your evaluations by making that pool of data easier to access and manipulate to draw your own conclusions. Show me all the patches a student has made this term; how many were accepted? Show me her blog posts. Are they thoughtful and articulate? Show me his mailing list conversations, his IRC logs - is he teaching others and being constructive? Show me their wiki edits. Show me a visualization of their code. Show me, show me, show me.

I don't know what this would look like; I think it would be different for each class and for each teacher. I would be fascinated to see what teachers and professors would do if we made better APIs to help them mine the data sources of the open source projects their students are working in.

Bonus: this makes life easier for researchers studying open source participation and teaching and learning in open source communities. Actually, an initial prototype (8 lines of quickly-written and deliberately horrid python written in the hopes that it would make another programmer twitch enough to pick it up and do it properly) of something like this was inspired by Diana Martin, an anthropology grad student

For instance, some Python-esque examples for a variant of this I could imagine for Fedora-specific services:

import fedorascraper

# returns a list of strings containing the last 10 edits ianweller made on Fedora's mediawiki instance
fedorascraper.mediawiki.lastedits(10, 'ianweller')

# returns the number of Planet posts rrix wrote since Jan 4 of 2010
fedorascraper.planet.num_posts_since('2010-01-04','rrix')

That way I can start looking for interesting correlations and asking questions like...

  • Do people who maintain packages tend to post a lot on mailing lists right after a release?
  • Of the people who have posted at least 3 times to design-list in the past 2 months, who maintains the most packages?
  • These two students seem to build off each other's work a lot - when did they start having a lot of back-and-forth conversations on IRC, and has this increased over time, and is it steady-state?

Interesting cross-correlation questions like that should be Easy To Ask. Easy enough that they can be assigned for homework questions like this - imagine students doing this not just for US Census data, but Fedora data, or data from - well, any open source project you might implement this sort of thing for.

Teachers: would this be useful? What sorts of tools and libraries would make it easier for you to grade student work in open source - or what do you already use for it - and what lessons can we learn from this in terms of making a "open source community health" dashboard?