Simon Dobson (Posts about strategy)

Me and my research

Simon Dobson — Mon, 28 Nov 2022 19:45:42 GMT

Me and my research

I've recently been thinking about my research directions, making use of some advice I came across years ago.

It's the tooling, not the terabytes

Simon Dobson — Mon, 17 Mar 2014 08:30:06 GMT

With all the hype about big data it's sometimes hard to realise that it's about more than just data. In fact, it's real interest doesn't come from big-ness at all. The term big data is deceptively hard to parse. It's a relative term, for a start: when you get down to it, all it really means is data that's large relative to the available storage and processing capacity. From that perspective, big data has always been with us -- and always will be. It's also curiously technology-specific for something that's garnering such broad interest. A volume of data may be "big" for one platform (a laptop, for example) and not for others (a computing cluster with a large network-attached store). Getting away from the data, what researchers typically mean by big data is a set of computational techniques that can be applied broadly to data somewhat independent of its origins and subject. I'm not a fan of "data science", but the term does at least focus on techniques and not on data size alone. This is much more interesting, and poses a set of challenges to disciplines -- and to computer science -- to identify and expand the set of techniques and tools researchers can make use of. What often frightens people about big data (or makes it an attractive career niche, depending on your point of view) is that there is a step-change in how you interact with data beyond a certain size -- and it's this change in tooling requirements that I think is a more significant indicator of a big-data problem than simply the data size. Suppose you collect a dataset consisting of a few hundred samples, each with maybe ten variables -- something that could come from a small survey, for example. This size of data can easily be turned into a spreadsheet model, and a large class of researchers have become completely comfortable with building such models. (This didn't used to be the case: I had a summer job in the late 1980's building spreadsheet models for a group that didn't have the necessary experience. Research skills evolve.) Even a survey with thousands of rows would be possible. But what if the survey has several million rows? -- for example because it came from an online survey, or because it was sensed automatically in some way. No spreadsheet program can ingest that much data. A few million rows does not constitute what many people would regard as "big data": it'll be at most several megabytes. But that's not the point: rather, the point is that, in order to deal with it, the researchers have to change tools -- and change them quite radically. Rather than using a spreadsheet, they have to become programmers; not just that, they have to become programmers who are familiar with languages like Python (to get the libraries), and Hadoop and cloud computing (to be able to scale-out the solutions). They could employ programmers, of course, but that removes them from the immediate and intimate contact with their data that many researchers find extremely valuable, and that personal computers have given us. To many people, hiring a programmer and running calculations in the cloud is suspiciously like a return to the mainframe computing we thought was rendered obsolete decades ago. It's not the terabytes, it's the tooling. Big data starts when you cross the threshold from your familiar world of interactive tools and into a more specialised, programmers world. This transition is becoming common in humanities and social sciences, as well as in science and medicine, and to my mind it's the major challenge of big data. The basic problem is simple: changing tools takes time, expertise, and mental effort, that is taken away from what's a researcher's actual research interest. A further disincentive is that the effort may not be rewarded: after all, if this really is research, one is often not sure whether there actually is anything valuable on the other side of a data analysis. In fields where competition is really competitive, like medicine, this feels like a lot of risk for an uncertain reward. There's evidence to suggest -- and I can't prove this contention -- that people are steering clear of doing the experiments they know they should do because they know they'll generate data volumes they're uncomfortable with. This, then, is where the big data challenge actually is: minimising the cost of transition from tools that are familiar to tools that are effective on the larger data volumes that are now becoming commonplace. This is a programming and user interface challenge, to make the complex infrastructure appear easy and straightforward to people who want to accomplish tasks on it. A large challenge, but not an unprecedented one: I'm writing this just after the 25th birthday of the World Wide Web that took a complex infrastructure (the internet) and made it usable by everyone. Let's just not get too hung-up on the idea that data needs to be really big to be interesting in this new data-driven world.

Being a student with open courseware

Simon Dobson — Wed, 23 Jan 2013 08:00:19 GMT

The move towards massively open on-line courseware (MOOCs) has the potential fundamentally to change the practice and experience of university. But what does it feel like to study on a MOOC-delivered course? No-one in academia can have been indifferent to the influence the internet has had on their teaching and research over the past decade, but MOOCs are a significant development nonetheless. They represent an attempt to "unbundle" education from the traditional providers, reducing barriers to entry and costs to students while facilitating alternative styles of learning and attendance not always supported in existing institutions. It's impossible to generalise about how universities deliver their material, but it's fair to say that the ability to study a bespoke programme at one's own speed and from whatever location is convenient is a significant departure from the usual three-to-four-year resident degree programme. (See, for example, Salman Khan. What college could be like. Communications of the ACM 56(1). January 2013.) Like most other research-led institutions, St Andrews is exploring on-line learning and MOOCs -- in our case as part of the FutureLearn initiative. But planning a radical style-change in teaching doesn't really work if it's purely "from the top", as it's the experience "from the bottom", as a student, that will dictate whether an approach succeeds or fails. So I signed up to take a MOOC module. We're only in Week 3 (of 12) at present, but it's worth making some early observations. To avoid embarrassing the providers, I won't identify the lecturer or module. Suffice to say it's being delivered on one of the three major platforms for MOOCs; is delivered by a tenured academic at one of the world's top universities; is a science subject, not a humanity; and is not in computer science or mathematics, to avoid the risk of my subject expertise getting in the way. The experience of signing up is as simple and straightforward as one would expect from a modern web provider, with each module being given its own home page containing links to all the videoed lecture material, practicals, class tests, and a discussion forum. Clearly a lot of work has gone into preparing all the material: around 30 hours of video (with closed captioning) plus materials for formative and summative assessment: it's a substantial investment of time by all concerned. Interestingly, this particular module is being run by the originating university as an experiment, to see how best they can deploy on-line learning. The student cohort includes both on-line students and students resident on campus for a "traditional" programme -- with the former outnumbering the latter by two orders of magnitude. Both sets are taking the same classes and assessments, but the resident students can also avail of a lecture each week "on demand" on any subject they feel needs more attention. This could be seen as a safety net in case the on-line experience is less than satisfactory, but also as a feedback loop to refine the module for next time: without this, it'd be harder to work out which parts needed more explanation, as always happens. The main course material is divided into units with about two hours of video material each, split into convenient sections which can be watched on-line or downloaded. The partitioning is convenient both for students and (I suspect) for the lecturer, who can record them in shorter "takes" to simplify correction and re-recording. The videos as typically either the lecturer as a "talking head" against a plain background, or a slide set onto which the lecturer writes additional notes as he talks. (Sometimes a single video piece includes both styles.) Having handwriting appear works better than might be expected: it keeps the human connection, is less mechanical and more engaging than might be the case with purely printed slides. Some sections also include in-line multiple-choice questions for students to answer to check understanding during the lecture. Having a lecture on video has a number of advantages, not least the ability to pause and repeat arbitrarily. I found this useful for note-taking, in that I could listen to a section, pause it to write summary notes, and not miss what the lecturer said next. A more experienced student note-taker might be able to multi-task enough for this not to matter. Note-taking is important for its own sake, of course, as it activates other parts of your brain than listening alone and makes the material much more engaging and likely to stick. Repetition has been less useful, which may just be a reflection of the level of the material so far, and a non-native English-speaker would probably find it invaluable. The module assessment includes some labs, which to get the whole class involved had to occur at set times: 9pm UK time was bad enough, but of course Indian and Chinese students had it much worse! However, the lab software didn't work: I think it simply couldn't handle the demand of the student numbers, although it was hard to tell for sure from my outsider's viewpoint. The lecturer ended up dropping the entire lab component and changing the rubric, which seemed substantially easier than the same task would have been in St Andrews, where it would have raised questions about the fairness of assessment. In this case it probably reflects teething troubles, but it does highlight some of the concerns that have been raised about the standard of examination (and consequently the value of certification) from MOOC-delivered courses: does the module actually still make sense without the labs with which it was designed? The remaining assessment consists of problems sets and an examination. All of these have deadlines, but with a sliding scale of lateness penalty (up to a fortnight) within which students can still submit. This seems reasonable for an on-line module, especially when one has no visibility of, or expectation about, students' personal circumstances. The marking overhead for assessment will be substantial, however, as the problem sets aren't all multiple-choice -- although they do show signs of being designed for large-scale grading. Coming from a university with small classes (less than a hundred is the norm, less than twenty not exceptional), this would require a significant change in my style of assessment -- although to be fair, no university is set-up for class sizes in the tens of thousands that could easily attend a MOOC. This remains a major academic and technical challenge. The discussion forum was a particularly interesting experience. There are a number of teaching assistants as well as the lecturer monitoring the discussions, answering a guiding questions, and students get credit for their engagement in discussions. Some students were noticeably more active, offering a range of different experiences and levels of expertise. Some clearly know enormously more about the subject matter than has been taught, possibly students of this area trying to supplement their experience with modules from another -- perhaps more prestigious -- university. Others, it rapidly became clear, are actually academics doing exactly what I'm doing: attending the class for interest, but really studying the MOOC and its experience. As I said, MOOCs are not an area any institution is willing to let pass by. Overall the experience is broadly positive, and I'm not concerned about any of the technological challenges posed: they're simply those of any large, distributed computer system. But I think at least four questions arise about how to do this sort of learning well. Firstly, it's clear that there's an immense amount of work involved at the back end of any MOOC, in terms of its design and preparation but also in terms of timetabling, support and -- especially -- assessment. Unless one is willing to accept multiple-choice questions or other approaches that can be mechanised it's hard to see how this can be avoided, and it's at a scale that few if any first-tier institutions have experience with. Secondly, the feeling of a class community is notably absent, and discussion boards can't make up for face-to-face interaction and live discussion. Especially in a small institution like St Andrews, the student experience -- of a small, closely-knit class in daily close contact with lecturers, demonstrators, and the wider student body -- is a major attraction, one that is highlighted by our teaching fellows and in all our student surveys, and one that would be hard to match in a distance setting. Perhaps this is simply academic snobbery: the cost of such an experience may be prohibitive for a progressively larger fraction of potential students. On the other hand, its pedagogical advantages are unquestionable. If we simply accept the difference, we run the risk of consigning a large group of students to a study experience we know to be sub-standard -- and indeed have designed to be such. Are there ways to provide an equivalent, if not better, experience? This has to be a major topic for future research, and has technological and sociological dimensions. Thirdly, distance interrupts the feedback loops that usually exist between a lecturer and their material, and between lecturer and class. In a local setting, feedback is generally immediate and clear: you can tell pretty much instantly if a class isn't engaged with or understanding the material, and adjust accordingly. In a distance setting, feedback is indirect and time-delayed, moderated by students' willingnesses to criticise or complain about their lecturers, which I would expect to complicate the refinement of a module that usually happens. Again, designing a better experience is probably not an insoluble problem, but it is an urgent one. Finally, there's the issue of coherence in a system in which one can make arbitrary module choices. I'm a great believer in broad curricula and plenty of options, but one often wants to get a degree in something, and this requires structuring either for professional or academic reasons. Without such structure, certification of what the student has learned, and how well, become a lot more difficult. But even beyond this, there's a problem of the embarrassment of choice, and of not knowing what one should study to accomplish whatever learning goal one has. This is a function often provided by advisers, counsellors and other staff within a university, who can help students understand what to study and why they're going to university in the first place. Their function is easily forgotten in distance learning, but seems to me to be vital for successful learning outcomes and eventual student satisfaction. It certainly suggests that the notion of distance learning needs to be broadened significantly beyond the content if it's to offer a real alternative to a "normal" degree: indeed, the content sometimes feels almost irrelevant, given the avalanche of material available on the web outside the MOOC universe. Curation, direction and socialisation feel like far more dominant challenges. When I decided to conduct this experiment, I thought it would draw me into the technology of MOOCs. It didn't: the technology is well done, but unexceptional. The content is great, but again no more impressive than any number of other sources on the internet. (Admittedly a sample size of one is hardly conclusive….) The sociology, however, is another matter: there's clearly a need to study the ways in which people experience learning through computers, at a distance, and in distributed groups across which one wants to establish a cohort effect, and how the universities can leverage their expertise in this into the on-line world. It'll be interesting to see what happens next.

Chief scientific advisor for Ireland

Simon Dobson — Sat, 17 Nov 2012 08:00:25 GMT

I'm a little perplexed by the direction of the current discussion over Ireland's chief scientific advisor. The need for scientific advice, as a driver of (and sanity check upon) evidence-based policy formation, has arguably never been greater. Certainly many of the challenges we face living in the 21st century are, directly or indirectly, influenced by a sound understanding of both the science and its limitations. This is why it's attractive for governments to have dedicated, independent scientific advisors. We need first to be clear what a good chief scientific advisor isn't, and that is an oracle. The chief scientist will presumably be a trained, nationally and internationally respected practitioner who brings to the job an experience in the practice of science but also in its wider analysis and impact. In many respects these are the qualities one looks for in a senior academic -- a full professor -- so it's unsurprising that chief scientists are often current or former (emeritus) professors at leading institutions. They will not be expert on all the areas of science required to address any particular policy question -- indeed, they might never be expert in the details pertaining to any question asked -- but they can act as a gateway to those who are expert, and collate and contrast the possibly conflicting views of different experts who might be consulted in each case. They provide a bridge in this sense between the world of the research and the world of policy, and will need to be able to explain clearly the basis and consequences of what the science is saying. Ireland has recently abandoned having a chief scientist as an independent role, and has instead elected to combine it with the role of Director of Science Foundation Ireland, the main advanced research funding agency. There are several stated reasons for this, most centring on the current resource constraints facing government spending. I don't think this structure is really consistent with the role of chief scientist, nor with the principles of good governance. To avoid any misunderstandings, let's be clear that this has nothing to do with the individuals involved: the current or former directors of SFI would be eminently suited to be a chief scientist in their own right. However, having the SFI director fill this additional role ex officio seems not to be best practice. The concern must be that the combined role cannot be independent by its very nature, in that the scientific direction of SFI may wholly or in part be involved in the policy decisions being made as a consequence of advice received. If this occurs, the chief scientist is then recommending actions for which the director must take responsibility, and the perception of a confusion of interest is inevitable if these roles are filled by the same individual. To repeat, the integrity of the office-holders is not the issue, but rather a governance structure that conflates the two roles of execution and oversight. If resource constraints are really the issue, one might say that the chief scientist does not need to be independently employed to be independent in the appropriate sense. The chief scientific advisory roles in the UK, for example, are typically filled by academics on part-time release from their host institutions. They collate and offer scientific advice, possibly made better by the fact that they remain active researchers and so remain current on both the science and its practice, rather than being entirely re-located into the public service. (The chief scientific advisor for Scotland, for example, remains a computer science researcher at the University of Glasgow in addition to her advisory role.) The risk of confusion is significantly less in this structure, because a single academic in a single institution does not exert executive control or influence over wider funding decisions. Moreover the individual remains employed by (and in the career and pension structure of) their host university and is bought-out for part of their time, which reduces the costs significantly. It also means that one can adjust the time commitment as required. I thought when the post was first created that it was unusual that the Irish chief scientist's post was full-time: requiring the (full-time) SFI director to find time in addition for the (presumably also full-time) duties of chief scientific advisor is expecting a lot. It is of course vital for the government to be getting good science advice, so it's good that the chief scientist role is being kept in some form. But I think it would be preferable to think about the governance structure a little more, to avoid any possible perception of confusion whether or not such confusion exists in practice.

Response to Science Foundation Ireland's consultation on its strategic plan

Simon Dobson — Wed, 26 Sep 2012 07:00:04 GMT

Science Foundation Ireland (SFI) recently launched its strategic plan for the coming years. This is my contribution to the consultation process. It's quite unusual for there to be a consultation process, of course: many such documents are drafted by governments and their agencies without reference to the opinions of outside stakeholders, so it's gratifying that SFI is confident enough to put its thoughts out for public comment. It's also gratifying that the aims and aspirations embodied by the document are so forward-looking and ambitious, given the parlous state of the country's finances: the straightened times only make government investment in science more important, as a route to improving the country's position.

There are however some issues that I think merit further consideration. These include:

the orientation of basic research;
the demands on staff from different phases of the research lifecycle;
the career trajectories of PhD students;
the significance of capacity-building, especially in the area of spin-outs and other indirect benefits; and
the possible extended leveraging of the expertise present in Irish science as part of the organisation.

What follows is abstracted from the letter I submitted to the consultation. I've removed some of the extraneous detail and generalised slightly to make the content less Ireland-specific, as a lot of the issues will be faced (or indeed are being faced) by funding agencies elsewhere.

Orientation of basic research. The orientation of research suggests that one can create a clear vision of what is going to be successful and impactful. This is clearly not the case: many areas that have changed the world in the long term had little or no short-term applicability when they were first investigated (for example lasers and hypermedia). The notion of government funding as “sowing seeds” therefore needs to be regarded with caution, and the history of (for example) Silicon Valley is notable mostly for the lack of directed and co-ordinated government investment and a focus instead on capacity-building (see below).

To maximise the chances of breakthroughs, one must allow experimentation that cannot be justified in terms of its known or predicted impact. One hears a lot about the “impact” and “spillover” of “basic” research into more “applied” areas. It is worth noting that such a hard distinction between “basic” and “applied” research is now largely discredited: a more accurate characterisation might be between “applied” and “not applied (yet)”. This is important, as it implies that any calculation of the value for money of any piece of research is often more a matter of timescale than of any intrinsic property of the work or field. Much of the mathematics that now underlies on-line cryptography, for example, was developed decades before its was practically applied, without such number theory having any obvious applications.

The basic difficulty with orientating basic research is that it is almost always a backward-facing activity, in the sense that one can only generate evidence in support of areas that have already demonstrated relevance and/or which already have a significant local industrial presence. Unless care is taken this can exclude promising areas for which there is currently no market but which have the capacity for enormous impact going forward. (3-D printing is the technology that springs most to mind here.) Setting a limit on the funding that will go to non-prioritised areas seems unlikely to provide for a broad exploration of speculative areas.

Phases and skills. It is important to recognise that excellent research scientists are often not the appropriate individuals to commercialise their own work. When one is speaking of the research lifecycle, it is clearly true that the translation phase is as creative and exciting as the discovery phase. However, it must be recognised that the skills required in these two phases are very different. While some exceptional individuals are able to muster excellence in both discovery and translation, this is extremely rare, as evidenced by the tendency of the founders of research-led start-ups to leave (or be eased out) as their companies grow: most scientists function better in one regime or the other. Put another way, excellent research scientists will be more productive overall if they are not forced into an inappropriate role. It would therefore be appropriate to generate structures whereby research can be ”handed off” between groups, and that recruitment and funding structures be introduced to ensure that scientists in each phase are treated equally and fairly – although not necessarily identically, to reflect their different motivations.

PhD careers. The decision whether to go into industry or academia is a complex one, driven by an individual's temperament and interests. I believe that care is needed in aspiring to move some given percentage of PhD graduates into industry. It would be a mistake to attempt to direct such career decisions, since trained researchers wanting to pursue academic careers that are not available locally will not generally take up industrial posts as an alternative: they will simply move abroad. This cohort of researchers is highly mobile and motivated, and only by providing matching opportunities will their skills be retained.

Capacity-building. While there is clearly enormous potential value in direct commercialisation of research products, there is far more value in the ICT space from simply building capacity. I have been struck by the number of start-up companies in Dublin formed by former PhD students (including several of my own) – but I have been further struck by the work these companies are doing, which often does not relate to the research topics of their founders. Indeed, in most cases the companies' work could not have led to a PhD.

This I think underlines the importance of intellectual capacity-building, and a corollary is that what is important is that the system generate researchers, rather than being solely concerned about the actual research done in training these individuals. Brilliant, educated minds will go on to do good work: if attracting the best minds is best accomplished by supporting them in basic research for their PhDs, this will be a good investment. It is noticeable that many staff in Silicon Valley companies have PhDs from local universities in very foundational topics.

Another aspect of capacity-building that often goes unmentioned is the progression of staff in post: the recognition that the excellent researchers need to have their career aspirations met and respected. There is ample evidence that this function is not properly dealt with by many institutions within their current structures: the exclusive focus on importing “iconic” and “prize-winning” staff can be demoralising to local staff, who can then become demotivated or induced to emigrate.

I believe the evidence supports the notion that staff in post will overall be more motivated, and more committed, by promotion than many high-flying individuals who may regard their appointment as a temporary base or a prelude to retirement, and may not continue to do the world-leading work that underpinned their recruitment.

Integrating expertise. SFI aspires to be a “model” department in terms of supporting scientific activity. One approach that might be beneficial is that pursued by the NSF, to second scientists into the organisation as programme officers. This approach – which I believe is currently unique to NSF – seems to deliver very well-managed programmes, gives the organisation access to a range of scientific talent, ensures that the staff in charge of programmes are up-to-date with the latest science, and also immensely benefits the management skills of the scientists involved. It is true that it can be challenging to manage conflicts of interest, but the research community in the US is also “a small country” in this sense, so I am sure that mechanisms can be found. Providing seconded individuals with a funded postdoc ex officio (as we do in St Andrews for Heads of School) might allow their own research to proceed in their absence.

It'll be interesting to see what happens to the strategic plan as a result of the consultation, but whatever the result it's a creative and constructive exercise to test the plan against an outside audience. I'd like to think this can only improve the process of governance for State-supported science.

The shifting balance of university power

Simon Dobson — Wed, 04 May 2011 10:00:17 GMT

The shifts in economic power are being mirrored in the university sector, both in education and research. It's happened before. The global financial crisis has exposed a lot of unfunded holes in different parts of the economy, and the resulting cuts and re-prioritisations are affecting the ways in which a lot of organisations operate. Universities find themselves uncharacteristically in the front line of this process. In terms of teaching, the sudden enormous increase in fees in England is currently being resisted -- futilely, I think -- in Scotland. The shifting of burden onto students will have long-ranging impact because, as isn't often realised, the increase in fees is being coupled with a projected decrease, applied differentially across subjects, in core State funding for teaching. This means that the huge influx of money from fees will be largely offset by a decrease in other funding: the universities will be no better off. In research, there is already a shift in the amounts of money available from funding agencies as well as in the ways that money is distributed. Crudely put, in future we'll see a smaller number of larger grants awarded to larger institutions who already have significant research funding from these same funding sources: the funding bodies will follow their own money to reduce risk. We have no idea what impact these changes will have on the quality of education, research, innovation or scholarship, or on the rankings that (very imperfectly) track these features. What we do know is that they're all intertwined, and that major shifts in the global balance of quality in education and research are not just possible, but likely. People looking at the university rankings tend to think that they reflect a long-standing, established assessment that changes only peripherally as "new" universities improve. This is actually very far from being the case. To see why, we need to consider the history of universities and their evolving quality relative to each other over the past six to seven hundred years. To simplify I'll focus on what we might regard as the modern, Western model of universities and ignore the university-like institutions in the Islamic caliphate, the House of Wisdom and the like -- although I really shouldn't, and it'd be good to see how they fit into the story. The designation of "best university in the world," whatever that may mean, has shifted several times. Initially it went to the University of Bologna as the first modern, Western university. But it soon shifted in the eleventh century to be the University of Paris, largely through the controversial fame of Peter Abelard -- an uncharacteristically scandal-prone academic. Over the course of the next centuries the centre of the academic world moved again, to Oxford and Cambridge. So far so familiar -- except that the dynamism that pushed these institutions forward didn't sustain itself. By the late nineteenth century the centre of research and teaching in physics and mathematics had shifted to Germany -- to the extent that a research career almost required a stint at a German institution. Oxford and Cambridge were largely reduced to teaching the sons of rich men. That's not to say that the Cavendish Laboratory and the like weren't doing excellent work: it's simply to recognise that Germany was "where it's at" for the ambitious and talented academic. When people think of Einstein, they mostly know that he worked for a larger part of his career in the Institute for Advanced Study at Princeton. What isn't often appreciated is that this wasn't the pinnacle of his career -- which was in fact when he was awarded a chair at the University of Berlin. In the early twentieth century the US Ivy League was doing what Oxford and Cambridge were doing fifty years earlier: acting as bastions of privilege. It took the Second World War, the Cold War and the enormous improvements in funding, governance and access to elevate the American institutions to their current levels of excellence. All this is to simplfy enormously, of course, but the location of the pre-eminent universities has shifted enormously, far more and far faster than is generally appreciated: Italy, France, England, Germany, the US. It isn't in any sense fixed. Many people would expect China to be next. It's not so long ago that Chinese universities were starved of investment and talent, as the best minds came to the West. This is still pretty much the case, but probably won't be for much longer. There are now some extremely impressive universities in China, both entirely indigenous and joint ventures with foreign institutions. (I'm involved in a project with Xi'an Jiaotong Liverpool University, a joint venture between China and the UK.) It's only a matter of time before some of these institutions are recognised as being world-class. Whether these institutions become paramount or not depends on a lot of factors: funding, obviously, of which there is currently a glut, for facilities, faculty and bursaries. But there's more to it than that. They have to become places where people want to live, can feel valued, and can rise to the top on their merits. You will only attract the best people if those people know their careers are open-ended and can grow as they do. The pitfalls include appealing solely to a small and privileged demographic, one selected by its ability to pay and to act as patrons to otherwise weak and under-funded institutions, and of focusing on pre-selected areas to the exclusion of others. Both these are actually symptoms of the same problem: a desire to "pick winners," avoid risk, and score well against metrics that can never capture the subtleties involved in building world-class institutions of learning.

Hiring academics

Simon Dobson — Fri, 25 Feb 2011 08:00:35 GMT

Hiring anyone is always a risk, and hiring a new academic especially so given the nature of our contracts. So what's the best way to approach it? What's brought this to mind is a recent discussion about the need to -- or indeed wisdom of -- interviewing new staff. The argument against interviewing is actually not all that uncommon. People tend to like and identify with -- and therefore hire -- people like themselves, and this can harm the diversity of a School when everyone shares a similar mindset. Another version of the same argument (that was applied in a place I used to work) says that you appoint the person who interviews best on the day, having shortlisted the five or so best CVs submitted regardless of research area. I can't say I buy either version. In fact I would go the other way: you need a recruitment strategy that decides what strengths you want in your School -- whether that's new skills or building-up existing areas -- and then interview those people with the right academic fit and quality with a view to deciding who'll fit in with the School's culture and intentions. My reasons for this are fairly simple. The best person academically isn't necessarily the best appointment. The argument that you employ the best researchers, in line with the need to generate as much world-class research as possible, is belied by the need to also provide great teaching (to attract the best students) and to engage in the impact that research has in the wider world. The idea that one would employ the best person on the day regardless of area strikes me as a non-strategy that would lead to fragmentation of expertise and an inability to collaborate internally. (It actually does the academic themselves no favours if they end up hired into a School with no-one to collaborate with.) Interviewing weeds-out the unsociable (and indeed the asocial) and lets one assess people on their personal as well as academic qualities. It's important to remember that academics typically have some form of tenure -- or at the very least are hard to fire -- and so one can't underestimate the damage that hiring a twisted nay-sayer can do. In case this isn't convincing, let's look at it another way. Suppose we recruit a new full professor. Suppose that that they're about 45, and so have around 20 years to retirement. Assume further that they stay for that entire time and don't retire early or leave for other reasons. The average pre-tax salary for a full professor in the UK is around £70,000. So the direct salary cost of the appointment is of the order of £1,500,000. After that, the individual will retire and draw (for example) 1/3rd of salary for another 15 years. (Although this is paid for from an externally-administered pension fund, we can assume for our purposes that the costs of this fund come at least partially from university funds.) So the direct cost of that appointment doesn't leave much change out of £1,800,000. (And that's just the direct costs, of course. There are also the opportunity costs of employing the wrong person, in terms of grants not won, students not motivated, reputations damaged and so forth. I have no idea how to calculate these, but I'm willing to believe they're of a similar order to the direct costs.) So in appointing this individual, the interview panel is making a decision whose value to the university is of the order of £2,000,000, and probably substantially more. How much time and care would you take before you spent that much? My experience has been mixed. A couple of places I interviewed had candidates in front of the interview committee (of four people, in one case) for a fifteen-minute presentation and a one-hour interview: one hundred minutes of face time to make what was quite literally a million-pound decision. By contrast I was in St Andrews for three days and met what felt like half the university including staff, teaching fellows, students, postdocs, administrators and others. I think the idea that a CV is all that matters is based on the fallacy that the future will necessarily be like the past. I'm a contrarian in these things: if I interview someone for a job I don't care what they've done in the past, except to the extent that it's a guide to what they're going to do in the future. What you're trying to decide in making a hiring decision is someone's future value. Taken to its logical conclusion, what you ideally want to do is to identify people early who are going to be professors early -- and hire them, now! What you shouldn't do is only consider people with great pasts, because you get little or no value from that if it isn't carried forward. You want to catch the good guys early, and then you get all the value of the work put on their CVs going forward. You also get to benefit from the motivational power of promotion, which for many people will spur them to prove themselves. Clearly there's a degree of unacceptable risk inherent in this, which we basically mitigate by employing people as junior academics. But this only works for the young guns if the institution's internal promotion scheme is efficient and will reward people quickly for their successes. Otherwise the young guns will look elsewhere, for an institution playing the long game and willing to take a chance on them -- and will do so with a better CV, that you've helped them build by hiring them in the first place. In the current climate institutions can't afford this, so optimising hiring and promotions is becoming increasingly critical for a university's continued success.

Who invented meringue?

Simon Dobson — Thu, 17 Feb 2011 08:00:15 GMT

What the invention of complicated foods tells us about discovery, innovation, and university research funding. Over lunch earlier this week we got talking about how different foods get discovered -- or invented, whichever's the most appropriate model. The point of the discussion was how unlikely a lot of foods are to have actually been created in the first place. The lineage of some quite complicated foods is fairly easy to discern, of course. Bread: leave out some wet flour overnight and watch it rise to form sourdough. Do the same for malt and you get beer (actually the kind that of beer that in Flanders is called lambic). Put milk into a barrel, load it onto the back of a donkey and transport it to the next town, and you'll have naturally-churned butter. It's fairly easy to see how someone with an interest in food would refine the technique and diversify it, once they knew that the basic operation worked in some way and to some degree. But for other foods, it's exactly this initial step that's so problematic. I think the best example is meringue. Consider the steps you need to go through to discover that meringues exist. First, you have to separate an egg -- which is obvious now, but not so obvious if you don't know that there's a point to it. Then you need to beat the white for a long time, in just the right way to introduce air into it. If you get this wrong, or don't do it for long enough, or do it too enthusiastically (or not enthusiastically enough) you just get slightly whiter egg white: it's only if you do it properly that you get the phase change you need. Of course you're probably doing this with a wholly inappropriate instrument -- like a spoon -- rather than a fork or a balloon whisk (which you don't have, because nobody knows there are things that need air beating into them yet). Then you need to determine, counter-intuitively, that making the egg white heavier (with sugar) will improve the final result when cooked. Then you have to work out that cooking this liquid -- which has actually to be a process of drying, not cooking -- is actually quite a good idea despite appearances. It's hard enough to make a decent meringue now we know they exist: I find it hard to imagine how one would do it if one didn't even know they existed, and furthermore didn't know that beating egg whites in a particular way will generate the phase change from liquid to foam. (Or even know that there are things called "phase changes" at all for that matter.) Thinking a little harder, I actually can imagine how meringues got invented. In the Middle Ages a lot of very rich aristocrats competed with their peers either by knocking each other off horses at a joust or by exhibiting ever-more-complex dishes at feasts. These dishes -- called subtleties -- were intended to demonstrate the artistry of the chef and hence the wealth and taste of his patron, the aristocrat. Pies filled with birds, exact scale models of castles, working water-wheels made out of pastry, that kind of thing. In order to do this sort of thing you need both a high degree of cooking skill and a lot of unusual food-based materials to work in. You can find these as part of your normal cooking, but it's probably also worth some experimentation to find new and unusual effects that will advance this calorific arms race a little in your favour. So maybe meringue was invented by some medieval cook just doing random things with foodstuffs to see what happens. The time spent on things that don't work -- leaving pork fat outside to see if it ferments into vodka, perhaps? -- will be amortised out by the discovery of something that's really useful in making really state-of-the-art food. Contrary to popular belief the Middle Ages was a time of enormous technological advance, and it's easy to think of this happening in food too. So food evolves under the combined effects of random chance operations shaped by survival pressures. Which is exactly what happens in biology. A new combination gets tried by chance, without any anticipation of any particular result, and the combinations that happen to lead to decent outcomes get maintained. At that point the biological analogy breaks down somewhat, because the decent outcomes are then subjected to teleological refinement by intelligent beings -- cooks -- with a goal in mind. It's no longer random. But the initial undirected exploration is absolutely essential to the process of discovery. Bizarrely enough, this tells us something more general about the processes of discovery and innovation. They can't be goal-directed: or, more precisely, they can't be goal-directed until we've established that there's a nugget of promise in a particular technique, and that initial discovery will only be performed because of someone's curiosity and desire to solve a larger problem. "Blue-skies" research is the starting point, and you by definition can't know -- or ever expect to know -- what benefits it might confer. You have to kiss an awful lot of frogs to have a reasonable expectation of finding a prince, and blue-skies, curiosity-driven research is the process of identifying these proto-princes amongst the horde of equally unattractive alternatives. But someone's got to do it.

The (new) idea of a (21st century) university

Simon Dobson — Tue, 18 Jan 2011 08:00:01 GMT

What should the university of the 21st century look like? What are we preparing our students for, and how? And how should we decide what is the appropriate vision for modern universities? There's a tendency to think of universities as static organisations whose actions and traditions remain fixed -- and looking at some ceremonial events it's easy to see where that idea might come from. Looking in from the outside one might imagine that some of the teaching of "old material" (such as my teaching various kinds of sorting algorithms) is just a lack of agility in responding to the real world: who needs to know? Why not just focus on the modern stuff? This view is largely mistaken. The point of teaching a core of material is to show how subjects evolve, and to get students used to thinking in terms of the core concepts rather than in terms of ephemera that'll soon pass on. Modern stuff doesn't stay modern, and that is a feature of the study of history or geography as much as of computer science or physics. Universities by their nature have a ringside view of the changes that will affect the world in the future, and also contain more than their fair share of maverick "young guns" who want to mix things up. It's natural for academics to be thinking about what future work and social spaces will be like, and to reflect on how best to tweak the student experience with these changes in mind. What brought this to my mind is Prof Colm Kenny's analysis piece in the Irish Independent this weekend, a response to the recently-published Hunt report ("National strategy for higher education: draft report of the strategy group." 9 August 2010) that tries to set out a vision for Irish 3rd-level (undergraduate degree) education. Although specific to Ireland, the report raises questions for other countries' relationships with their universities too, and so is worth considering broadly. A casual read of even the executive summary reveals a managerial tone. There's a lot of talk of productivity, broadening access, and governance that ensures that institutions meet performance targets aligned with national priorities. There's very little on encouraging free inquiry, fostering creativity, or equipping students for the 21st century. The report -- and similar noises that have emerged from other quarters, in Ireland and the UK -- feel very ... well ... 20th century. Life and higher education used to be very easy: you learned your trade, either as an apprentice or at university; you spent forty years practising it, using essentially those techniques you'd been imparted with plus minor modifications; you retired, had a few years off, and then died. But that's past life: future, and indeed current, life aren't going to be like that. For a start, it's not clear when if ever we'll actually get to retire. Most people won't stay in the same job for their entire careers: indeed, a large percentage of jobs that one could do at the start of a career won't even exist forty years later, just as many of those jobs haven't been thought of now. When I did my PhD 20 years ago there was no such thing as a web designer, and music videos were huge projects that no-one without access to a fully-equipped multi-million-pound studio could take on. Many people change career because they want to rather than through the influence of outside forces, such as leaving healthcare to take up professional photography. What implications does this have for higher education? Kenny rightly points out that, while distance education and on-line courses are important, they're examples of mechanism, not of vision. What they have in common, and what drives their attractiveness, is that they lower the barriers to participation in learning. They actually do this in several ways. They allow people to take programmes without re-locating and potentially concurrently with their existing lives and jobs. They also potentially allow people to "dip-in" to programmes rather than take them to their full extent, to mash-up elements from different areas, institutions and providers, and to democratise the generation and consumption of learning materials. Some students, on first coming to university, are culture-shocked by the sudden freedom they encounter. It can take time to work out that universities aren't schools, and academics aren't teachers. In fact they're dual concepts: a school is an institution of teaching, where knowledge is pushed at students in a structured manner; a university is an institution of learning, which exists to help students to find and interact with knowledge. The latter requires one to learn skills that aren't all that important in the former. The new world of education will require a further set of skills. Lifelong learning is now a reality as people re-train as a matter of course. Even if they stay in the same career, the elements, techniques and technologies applied will change constantly. It's this fact of constant change and constant learning that's core to the skills people will need in the future. (Ten years or so ago, an eminent though still only middle-aged academic came up to me in the senior common room of the university I taught in at the time and asked me when this "internet thing" was going to finish, so that we could start to understand what it had meant. I tried to explain that the past ten years were only the start of the prologue to what the internet would do to the world, but I remember his acute discomfort at the idea that things would never settle down.) How does one prepare someone for lifelong learning? Actually many of the skills needed are already being acquired by people who engage with the web intensively. Anyone who reads a wide variety of material needs to be able to sift the wheat from the chaff, to recognise hidden agendas and be conscious of the context in which material is being presented. Similarly, people wanting to learn a new field need to be able to determine what they need to learn, to place it in a sensible order, locate it, and have the self-discipline to be able to stick through the necessary background. It's probably true, though, that most people can't be successful autodidacts. There's a tendency to skip the hard parts, or the background material that (however essential) might be perceived as old and unnecessary. Universities can provide the road maps to avoid this: the curricula for programmes, the skills training, the support, examination, quality assurance and access to the world's foremost experts in the fields, while being only one possible provider of the material being explored. In other words, they can separate the learning material from the learning process -- two aspects that are currently conflated. I disagree with Colm Kenny on one point. He believes that only government can provide the necessary vision for the future of higher education. I don't think that's necessary at all. A system of autonomous universities can set their own visions of the future, and can design processes, execute them, assess them, measure their success and refine their offerings -- all without centralised direction. I would actually go further, and argue that the time spent planning a centralised national strategy would be better spent decentralising control of the university system and fostering a more experimental approach to learning. That's what the world's like now, and academia's no different.

The computer is the new microscope

Simon Dobson — Thu, 01 Apr 2010 15:49:46 GMT

What contributions can computer scientists uniquely make to the latest scientific challenges? The answer may require us to step back and look at how instruments affect science, because the computer is the key instrument for the scientific future. In the late seventeenth century, science was advancing at an extraordinary rate -- perhaps the greatest rate until modern times. The scientists of this era were attempting to re-write the conceptual landscape through which they viewed the universe, and so in many ways were attempting something far harder than we typically address today, when we have a more settled set of concepts that are re-visited only periodically. This also took place in a world with far less of a support infrastructure, in which the scientists were also forced to be tool-makers manufacturing the instruments they needed for their research. It's revealing to look at a list of scientists of this era who were also gifted instrument-makers: Newton, Galileo, Hooke, Huygens and so on. Antonie van Leeuwenhoek is a classic example. He revolutionised our understanding of the world by discovering single-celled micro-organisms, and by documenting his findings with detailed drawings in letters to the Royal Society. The key instrument in his research was, of course, the microscope, of which he manufactured an enormous number. Whilst microscopes were already known, van Leeuwenhoek developed (and kept secret) new techniques for the manufacture of lenses which allowed him significantly to advance both the practice of optics and the science of what we would now term microbiology. The important here is not that early scientists were polymaths, although that's also a fascinating topic. What's far more important is the effect that tooling has on science. New instruments not only provide tools with which to conduct science; they also open-up new avenues for science by revealing phenomena that haven't been glimpsed before, or by providing measurements and observations of details that conflict with the existing norms. The point is that tools and science progress together, and therefore that advances in instrument-making are valuable both in their own right and in the wider science they facilitate. Not all experimental scientists see things this way. It's fairly common for those conducting the science to look down on the instrument-makers as mere technicians, whose efforts are of a different order to those involved in "doing" the science. It's fair to say that the scientists of the seventeenth century wouldn't share (or indeed understand) this view, since they were in a sense much closer to the instrument as a contributor to their research. Looked at another way, new experiments then typically required new instruments, rather than as now generally being conducted with a standard set of tools the researcher has to hand. What are the instruments today whose advance will affect the wider scientific world? "Traditional" instrument-making is still vitally important, of course, and we can even regard the LHC as a big instrument to used in support of particular experiments. But beyond this we have "non-traditional" instruments, of which computers are by far the most commonplace and potentially influential. I've talked previously about exabyte-scale science and the ways in which new computing techniques will affect it. Some experimenters overlook the significance of computational techniques -- or, if they do see them, regard them as making technician-level rather than science-level contributions to knowledge. Even more experimenters overlook the impact that more rarefied computer science concerns such as programming languages, meta-data and search have on the advancement of knowledge. These views are restricted, restricting, and (in the worst case) stifling. They are also short-sighted and incorrect. At the large scale, computational techniques often offer the only way of "experimenting" with large-scale data. They can be used to confirm hypotheses in the normal sense, but there are also examples where they have served to help derive new hypotheses by illuminating factors and phenomena in the data that were previously unsuspected, and furthermore could not have been discovered by any other means. The science is advanced by the application of large-scale computing to large-scale data, possibly collected for completely different purposes. In that sense the computer is behaving as an instrument that opens-up new opportunities in science: as the new microscope, in fact. This is not simply a technical contribution to improving the way in which traditional science is done: coupled with simulation, it changes both what science is done and how it is done, and also opens-up new avenues for both traditional and non-traditional experiments and data collection. A good example is in climate change, where large-scale simulations of the atmosphere can confirm hypotheses, suggest new ones, and direct the search for real-world mechanisms that can confirm or refute them. At the other end of the scale, we have sensor networks. Sensor networks will allow experimental scientists directly to collect data "in the wild", at high resolution and over long periods -- things that're difficult or impossible with other approaches. This is the computer as the new microscope again: providing a view of things that were previously hidden. This sort of data collection will become much more important as we try to address (for example) climate change, for which high-resolution long-term data collected on land and in water nicely complement larger-scale space-based sensing. Making such networks function correctly and appropriately is a significant challenge that can't be handled as an after-thought. At both scales, much of the richness in the data comes from the ways it's linked and marked-up so as to be searched, traversed and reasoned-with. While some experimental scientists develop strong computational techniques, very few are expert in metadata, the semantic web, machine learning and automated reasoning -- although these computer science techniques are all key to the long-term value of large-scale data. As with the earliest microscopes, the instrument-maker may also be the scientist, but that causes problems perhaps more severe today than in past centuries. Like it or not, we live in an era of specialisation, and in an era where it's impossible to be really expert in one field let alone the several one might need in order to make proper contributions. But the development of new instruments -- computational techniques, sensing, reasoning, matadata cataloguing -- is nevertheless key to the development of science. In the years after van Leeuwenhoek, several microbiologists formed close collaborations with opticians who helped refine and develop the tools and techniques available -- allowing the microbiologists to focus on their science while the opticians focused on their instruments. (Isn't it interesting how "focused" really is the appropriate metaphor here?) Looked at broadly, it's hard to say which group's contribution was more influential, and in some senses that's the wrong question: both focused on what they were interested in, solving hard conceptual, experimental and technological problems along the way, and influencing and encouraging each other to perfect their crafts. It's good to see this level of co-operation between computer scientists and biologists, engineers, sustainable-development researchers and the rest beginning to flower again, at both ends of the scale (and at all points in between). It's easy to think of instruments as technical devices devoid of scientific content, but it's better to think of them both as having a core contribution to make to the concepts and practice of science, and as having a fascination in their own right that gives rise to a collection of challenges that, if met, will feed-back and open-up new scientific possibilities. The microscope is a key example of this co-evolution and increased joint value from the past and present: the computer is the new microscope for the present and future.