On MOOCs

I just completed Keith Devlin's course "Introduction into Mathematical Thinking" on Stanford's coursera platform. It's a massive open online course or MOOC. It wasn't the first one I did. But this one was, based on my limited experience, so remarkable that I wanted to write about it here.

The course may not seem overly exciting at first glance and maybe even after looking at the syllabus: logical combinators, quantifiers, elementary number theory and proofs. The primary audience are first-year college students. What makes Devlin's course different from the two excellent courses on more advanced topics I have attended so far (one on machine learning, one on functional programming) is that he really made the most of the format and I learned a lot on the way.

I will try to explain my point by quickly summarising and comparing the different components of the MOOCs I have seen so far.

The Lecture Videos

The MOOCs I have signed up for consist usually of more or less well edited lecture videos accompanied by some form of assignment or homework.

I have seen quite elaborate technical solutions in regards to how the videos are presented. Martin Odersky used a quite fancy overlay technique during his functional programming course, which seemingly allowed him to "write" onto the slides he was using to support his presentation as if it was paper. You could even see his hands.

Keith Devlin's approach was more low tech: He used almost no slides just a piece of paper while the camera was looking from a 90 ° angle onto the table. This may seem like a lot of slow handwriting, but some clever editing makes for an impressively simple and effective presentation format: sequences where he has to write longer bits of text or mathematical formulae are sped up ever so slightly during editing to avoid the effect that the students have to "wait" until he finishes writing.

One big advantage of the MOOC format is that you can adjust the speed of the videos to your level of comprehension. If the teacher is very verbose and overly detailed in his explanations you speed up the video. I found that I can still follow when watching at 1.5 times the normal speed. On the other hand, if you do not understand something you can pause or watch it again until you get it.

The Assignments

Computer science lends itself well to the MOOC format because its "output" can in most cases be graded by a machine (of course not without some considerable effort to set this up). And indeed the assignments I submitted in the programming related courses were graded by running some form of automated testing on my code.

Mathematics, you might think, are even better: the output of your computation is typically highly formalised and either true or false. But this course was not about learning how to solve equations or learn new "recipes" how to compute things. It was focused on teaching a different way of thinking about mathematical problems and a way of communication that is highly subjective in its form of expression: the mathematical proof.

Devlin and his team had to come up with a way to evaluate the quality of proofs submitted by thousands of students. There is no easy way to do that without an army of human teaching assistants looking at every single submission. They came up with the following idea: instead of having the students submit proofs for grading by teachers let them grade the proofs themselves.

This is not only a brilliant solution to the technical problem but has a nice educational benefit. It turns out you learn a substantial amount by looking at other people's work and trying to grade them.

The proofs we got to grade as part of an assignment were carefully prepared by Devlin for the best educational effect: he added typical mistakes and imperfections based on his experience.

Exam And Peer Review

This course had a final exam (as opposed to the other courses). The exam was the only occasion where we had to submit proofs we actually did ourselves. All the grading was done via peer review. We had exercised our grading skills during the assignments and this was the first attempt to apply them to real world data.

I found this to be the most exciting part of the course. I graded the exams of three of my peers. First of all, it was a shock: real exams looked quite different compared to the artificial training proofs. Sometimes I reached the limits of my mathematical abilities while trying to judge whether the slightly unorthodox proof my fellow student had submitted was valid and if yes how good it was. The final step was to grade your own exam after grading your peers. I looked at my work with different eyes after grading the others. It turned out that the grade I gave myself came reasonably close to what my peers thought of my work.

And of course there were quite a few students complaining in the forums that they felt their peers had done a poor job at grading them. Anecdotal evidence in form of exam proofs posted by those students seemed to support their case. It is obvious that this format is still experimental and very much work in progress. But there is not much to lose: you don't get any credit towards a possible degree at any university I know of by doing Coursera's courses.

Final Thoughts

There are many questions unanswered about MOOCs: Who is the target audience (I do it just for fun, but what if you want to obtain a degree)? How does the business model work (Coursera has started to offer a so called "signature track" where you have to pay and get a verified certificate)? What is the hidden agenda (some courses can be seen as a form of clever advertisement for a specific technology e.g. a programming language)? How do you replace the missing personal contact and tutoring (discussion forums are an imperfect but valuable substitute)? Keith Devlin's course might point to a possible answer to one of the most fundamental problems of the format: how to make qualitative judgements about the work of tens of thousands of students. His answer might not be perfect but it is certainly very instructive.