The move towards massively open on-line courseware (MOOCs) has the potential fundamentally to change the practice and experience of university. But what does it feel like to study on a MOOC-delivered course? No-one in academia can have been indifferent to the influence the internet has had on their teaching and research over the past decade, but MOOCs are a significant development nonetheless. They represent an attempt to “unbundle” education from the traditional providers, reducing barriers to entry and costs to students while facilitating alternative styles of learning and attendance not always supported in existing institutions. It’s impossible to generalise about how universities deliver their material, but it’s fair to say that the ability to study a bespoke programme at one’s own speed and from whatever location is convenient is a significant departure from the usual three-to-four-year resident degree programme. (See, for example, Salman Khan. What college could be like. Communications of the ACM 56(1). January 2013.) Like most other research-led institutions, St Andrews is exploring on-line learning and MOOCs — in our case as part of the FutureLearn initiative. But planning a radical style-change in teaching doesn’t really work if it’s purely “from the top”, as it’s the experience “from the bottom”, as a student, that will dictate whether an approach succeeds or fails. So I signed up to take a MOOC module. We’re only in Week 3 (of 12) at present, but it’s worth making some early observations. To avoid embarrassing the providers, I won’t identify the lecturer or module. Suffice to say it’s being delivered on one of the three major platforms for MOOCs; is delivered by a tenured academic at one of the world’s top universities; is a science subject, not a humanity; and is not in computer science or mathematics, to avoid the risk of my subject expertise getting in the way. The experience of signing up is as simple and straightforward as one would expect from a modern web provider, with each module being given its own home page containing links to all the videoed lecture material, practicals, class tests, and a discussion forum. Clearly a lot of work has gone into preparing all the material: around 30 hours of video (with closed captioning) plus materials for formative and summative assessment: it’s a substantial investment of time by all concerned. Interestingly, this particular module is being run by the originating university as an experiment, to see how best they can deploy on-line learning. The student cohort includes both on-line students and students resident on campus for a “traditional” programme — with the former outnumbering the latter by two orders of magnitude. Both sets are taking the same classes and assessments, but the resident students can also avail of a lecture each week “on demand” on any subject they feel needs more attention. This could be seen as a safety net in case the on-line experience is less than satisfactory, but also as a feedback loop to refine the module for next time: without this, it’d be harder to work out which parts needed more explanation, as always happens. The main course material is divided into units with about two hours of video material each, split into convenient sections which can be watched on-line or downloaded. The partitioning is convenient both for students and (I suspect) for the lecturer, who can record them in shorter “takes” to simplify correction and re-recording. The videos as typically either the lecturer as a “talking head” against a plain background, or a slide set onto which the lecturer writes additional notes as he talks. (Sometimes a single video piece includes both styles.) Having handwriting appear works better than might be expected: it keeps the human connection, is less mechanical and more engaging than might be the case with purely printed slides. Some sections also include in-line multiple-choice questions for students to answer to check understanding during the lecture. Having a lecture on video has a number of advantages, not least the ability to pause and repeat arbitrarily. I found this useful for note-taking, in that I could listen to a section, pause it to write summary notes, and not miss what the lecturer said next. A more experienced student note-taker might be able to multi-task enough for this not to matter. Note-taking is important for its own sake, of course, as it activates other parts of your brain than listening alone and makes the material much more engaging and likely to stick. Repetition has been less useful, which may just be a reflection of the level of the material so far, and a non-native English-speaker would probably find it invaluable. The module assessment includes some labs, which to get the whole class involved had to occur at set times: 9pm UK time was bad enough, but of course Indian and Chinese students had it much worse! However, the lab software didn’t work: I think it simply couldn’t handle the demand of the student numbers, although it was hard to tell for sure from my outsider’s viewpoint. The lecturer ended up dropping the entire lab component and changing the rubric, which seemed substantially easier than the same task would have been in St Andrews, where it would have raised questions about the fairness of assessment. In this case it probably reflects teething troubles, but it does highlight some of the concerns that have been raised about the standard of examination (and consequently the value of certification) from MOOC-delivered courses: does the module actually still make sense without the labs with which it was designed? The remaining assessment consists of problems sets and an examination. All of these have deadlines, but with a sliding scale of lateness penalty (up to a fortnight) within which students can still submit. This seems reasonable for an on-line module, especially when one has no visibility of, or expectation about, students’ personal circumstances. The marking overhead for assessment will be substantial, however, as the problem sets aren’t all multiple-choice — although they do show signs of being designed for large-scale grading. Coming from a university with small classes (less than a hundred is the norm, less than twenty not exceptional), this would require a significant change in my style of assessment — although to be fair, no university is set-up for class sizes in the tens of thousands that could easily attend a MOOC. This remains a major academic and technical challenge. The discussion forum was a particularly interesting experience. There are a number of teaching assistants as well as the lecturer monitoring the discussions, answering a guiding questions, and students get credit for their engagement in discussions. Some students were noticeably more active, offering a range of different experiences and levels of expertise. Some clearly know enormously more about the subject matter than has been taught, possibly students of this area trying to supplement their experience with modules from another — perhaps more prestigious — university. Others, it rapidly became clear, are actually academics doing exactly what I’m doing: attending the class for interest, but really studying the MOOC and its experience. As I said, MOOCs are not an area any institution is willing to let pass by. Overall the experience is broadly positive, and I’m not concerned about any of the technological challenges posed: they’re simply those of any large, distributed computer system. But I think at least four questions arise about how to do this sort of learning well. Firstly, it’s clear that there’s an immense amount of work involved at the back end of any MOOC, in terms of its design and preparation but also in terms of timetabling, support and — especially — assessment. Unless one is willing to accept multiple-choice questions or other approaches that can be mechanised it’s hard to see how this can be avoided, and it’s at a scale that few if any first-tier institutions have experience with. Secondly, the feeling of a class community is notably absent, and discussion boards can’t make up for face-to-face interaction and live discussion. Especially in a small institution like St Andrews, the student experience — of a small, closely-knit class in daily close contact with lecturers, demonstrators, and the wider student body — is a major attraction, one that is highlighted by our teaching fellows and in all our student surveys, and one that would be hard to match in a distance setting. Perhaps this is simply academic snobbery: the cost of such an experience may be prohibitive for a progressively larger fraction of potential students. On the other hand, its pedagogical advantages are unquestionable. If we simply accept the difference, we run the risk of consigning a large group of students to a study experience we know to be sub-standard — and indeed have designed to be such. Are there ways to provide an equivalent, if not better, experience? This has to be a major topic for future research, and has technological and sociological dimensions. Thirdly, distance interrupts the feedback loops that usually exist between a lecturer and their material, and between lecturer and class. In a local setting, feedback is generally immediate and clear: you can tell pretty much instantly if a class isn’t engaged with or understanding the material, and adjust accordingly. In a distance setting, feedback is indirect and time-delayed, moderated by students’ willingnesses to criticise or complain about their lecturers, which I would expect to complicate the refinement of a module that usually happens. Again, designing a better experience is probably not an insoluble problem, but it is an urgent one. Finally, there’s the issue of coherence in a system in which one can make arbitrary module choices. I’m a great believer in broad curricula and plenty of options, but one often wants to get a degree in something, and this requires structuring either for professional or academic reasons. Without such structure, certification of what the student has learned, and how well, become a lot more difficult. But even beyond this, there’s a problem of the embarrassment of choice, and of not knowing what one should study to accomplish whatever learning goal one has. This is a function often provided by advisers, counsellors and other staff within a university, who can help students understand what to study and why they’re going to university in the first place. Their function is easily forgotten in distance learning, but seems to me to be vital for successful learning outcomes and eventual student satisfaction. It certainly suggests that the notion of distance learning needs to be broadened significantly beyond the content if it’s to offer a real alternative to a “normal” degree: indeed, the content sometimes feels almost irrelevant, given the avalanche of material available on the web outside the MOOC universe. Curation, direction and socialisation feel like far more dominant challenges. When I decided to conduct this experiment, I thought it would draw me into the technology of MOOCs. It didn’t: the technology is well done, but unexceptional. The content is great, but again no more impressive than any number of other sources on the internet. (Admittedly a sample size of one is hardly conclusive….) The sociology, however, is another matter: there’s clearly a need to study the ways in which people experience learning through computers, at a distance, and in distributed groups across which one wants to establish a cohort effect, and how the universities can leverage their expertise in this into the on-line world. It’ll be interesting to see what happens next.