Skip to main content

Posts about strategy (old posts, page 1)

Hiring academics

Hiring anyone is always a risk, and hiring a new academic especially so given the nature of our contracts. So what's the best way to approach it?

What's brought this to mind is a recent discussion about the need to -- or indeed wisdom of -- interviewing new staff. The argument against interviewing is actually not all that uncommon. People tend to like and identify with -- and therefore hire -- people like themselves, and this can harm the diversity of a School when everyone shares a similar mindset. Another version of the same argument (that was applied in a place I used to work) says that you appoint the person who interviews best on the day, having shortlisted the five or so best CVs submitted regardless of research area.

I can't say I buy either version. In fact I would go the other way: you need a recruitment strategy that decides what strengths you want in your School -- whether that's new skills or building-up existing areas -- and then interview those people with the right academic fit and quality with a view to deciding who'll fit in with the School's culture and intentions.

My reasons for this are fairly simple. The best person academically isn't necessarily the best appointment. The argument that you employ the best researchers, in line with the need to generate as much world-class research as possible, is belied by the need to also provide great teaching (to attract the best students) and to engage in the impact that research has in the wider world. The idea that one would employ the best person on the day regardless of area strikes me as a non-strategy that would lead to fragmentation of expertise and an inability to collaborate internally. (It actually does the academic themselves no favours if they end up hired into a School with no-one to collaborate with.) Interviewing weeds-out the unsociable (and indeed the asocial) and lets one assess people on their personal as well as academic qualities. It's important to remember that academics typically have some form of tenure -- or at the very least are hard to fire -- and so one can't underestimate the damage that hiring a twisted nay-sayer can do.

In case this isn't convincing, let's look at it another way. Suppose we recruit a new full professor. Suppose that that they're about 45, and so have around 20 years to retirement. Assume further that they stay for that entire time and don't retire early or leave for other reasons. The average pre-tax salary for a full professor in the UK is around £70,000. So the direct salary cost of the appointment is of the order of £1,500,000. After that, the individual will retire and draw (for example) 1/3rd of salary for another 15 years. (Although this is paid for from an externally-administered pension fund, we can assume for our purposes that the costs of this fund come at least partially from university funds.) So the direct cost of that appointment doesn't leave much change out of £1,800,000.

(And that's just the direct costs, of course. There are also the opportunity costs of employing the wrong person, in terms of grants not won, students not motivated, reputations damaged and so forth. I have no idea how to calculate these, but I'm willing to believe they're of a similar order to the direct costs.)

So in appointing this individual, the interview panel is making a decision whose value to the university is of the order of £2,000,000, and probably substantially more. How much time and care would you take before you spent that much?

My experience has been mixed. A couple of places I interviewed had candidates in front of the interview committee (of four people, in one case) for a fifteen-minute presentation and a one-hour interview: one hundred minutes of face time to make what was quite literally a million-pound decision. By contrast I was in St Andrews for three days and met what felt like half the university including staff, teaching fellows, students, postdocs, administrators and others.

I think the idea that a CV is all that matters is based on the fallacy that the future will necessarily be like the past. I'm a contrarian in these things: if I interview someone for a job I don't care what they've done in the past, except to the extent that it's a guide to what they're going to do in the future. What you're trying to decide in making a hiring decision is someone's future value. Taken to its logical conclusion, what you ideally want to do is to identify people early who are going to be professors early -- and hire them, now! What you shouldn't do is only consider people with great pasts, because you get little or no value from that if it isn't carried forward. You want to catch the good guys early, and then you get all the value of the work put on their CVs going forward. You also get to benefit from the motivational power of promotion, which for many people will spur them to prove themselves.

Clearly there's a degree of unacceptable risk inherent in this, which we basically mitigate by employing people as junior academics. But this only works for the young guns if the institution's internal promotion scheme is efficient and will reward people quickly for their successes. Otherwise the young guns will look elsewhere, for an institution playing the long game and willing to take a chance on them -- and will do so with a better CV, that you've helped them build by hiring them in the first place. In the current climate institutions can't afford this, so optimising hiring and promotions is becoming increasingly critical for a university's continued success.

Who invented meringue?

What the invention of complicated foods tells us about discovery, innovation, and university research funding.

Over lunch earlier this week we got talking about how different foods get discovered -- or invented, whichever's the most appropriate model. The point of the discussion was how unlikely a lot of foods are to have actually been created in the first place.

The lineage of some quite complicated foods is fairly easy to discern, of course. Bread: leave out some wet flour overnight and watch it rise to form sourdough. Do the same for malt and you get beer (actually the kind that of beer that in Flanders is called lambic). Put milk into a barrel, load it onto the back of a donkey and transport it to the next town, and you'll have naturally-churned butter. It's fairly easy to see how someone with an interest in food would refine the technique and diversify it, once they knew that the basic operation worked in some way and to some degree.

But for other foods, it's exactly this initial step that's so problematic.

I think the best example is meringue. Consider the steps you need to go through to discover that meringues exist. First, you have to separate an egg -- which is obvious now, but not so obvious if you don't know that there's a point to it. Then you need to beat the white for a long time, in just the right way to introduce air into it. If you get this wrong, or don't do it for long enough, or do it too enthusiastically (or not enthusiastically enough) you just get slightly whiter egg white: it's only if you do it properly that you get the phase change you need. Of course you're probably doing this with a wholly inappropriate instrument -- like a spoon -- rather than a fork or a balloon whisk (which you don't have, because nobody knows there are things that need air beating into them yet). Then you need to determine, counter-intuitively, that making the egg white heavier (with sugar) will improve the final result when cooked. Then you have to work out that cooking this liquid -- which has actually to be a process of drying, not cooking -- is actually quite a good idea despite appearances.

It's hard enough to make a decent meringue now we know they exist: I find it hard to imagine how one would do it if one didn't even know they existed, and furthermore didn't know that beating egg whites in a particular way will generate the phase change from liquid to foam. (Or even know that there are things called "phase changes" at all for that matter.)

Thinking a little harder, I actually can imagine how meringues got invented. In the Middle Ages a lot of very rich aristocrats competed with their peers either by knocking each other off horses at a joust or by exhibiting ever-more-complex dishes at feasts. These dishes -- called subtleties -- were intended to demonstrate the artistry of the chef and hence the wealth and taste of his patron, the aristocrat. Pies filled with birds, exact scale models of castles, working water-wheels made out of pastry, that kind of thing. In order to do this sort of thing you need both a high degree of cooking skill and a lot of unusual food-based materials to work in. You can find these as part of your normal cooking, but it's probably also worth some experimentation to find new and unusual effects that will advance this calorific arms race a little in your favour.

So maybe meringue was invented by some medieval cook just doing random things with foodstuffs to see what happens. The time spent on things that don't work -- leaving pork fat outside to see if it ferments into vodka, perhaps? -- will be amortised out by the discovery of something that's really useful in making really state-of-the-art food. Contrary to popular belief the Middle Ages was a time of enormous technological advance, and it's easy to think of this happening in food too.

So food evolves under the combined effects of random chance operations shaped by survival pressures. Which is exactly what happens in biology. A new combination gets tried by chance, without any anticipation of any particular result, and the combinations that happen to lead to decent outcomes get maintained. At that point the biological analogy breaks down somewhat, because the decent outcomes are then subjected to teleological refinement by intelligent beings -- cooks -- with a goal in mind. It's no longer random. But the initial undirected exploration is absolutely essential to the process of discovery.

Bizarrely enough, this tells us something more general about the processes of discovery and innovation. They can't be goal-directed: or, more precisely, they can't be goal-directed until we've established that there's a nugget of promise in a particular technique, and that initial discovery will only be performed because of someone's curiosity and desire to solve a larger problem. "Blue-skies" research is the starting point, and you by definition can't know -- or ever expect to know -- what benefits it might confer. You have to kiss an awful lot of frogs to have a reasonable expectation of finding a prince, and blue-skies, curiosity-driven research is the process of identifying these proto-princes amongst the horde of equally unattractive alternatives. But someone's got to do it.

The (new) idea of a (21st century) university

What should the university of the 21st century look like? What are we preparing our students for, and how? And how should we decide what is the appropriate vision for modern universities?

There's a tendency to think of universities as static organisations whose actions and traditions remain fixed -- and looking at some ceremonial events it's easy to see where that idea might come from. Looking in from the outside one might imagine that some of the teaching of "old material" (such as my teaching various kinds of sorting algorithms) is just a lack of agility in responding to the real world: who needs to know? Why not just focus on the modern stuff?

This view is largely mistaken. The point of teaching a core of material is to show how subjects evolve, and to get students used to thinking in terms of the core concepts rather than in terms of ephemera that'll soon pass on. Modern stuff doesn't stay modern, and that is a feature of the study of history or geography as much as of computer science or physics. Universities by their nature have a ringside view of the changes that will affect the world in the future, and also contain more than their fair share of maverick "young guns" who want to mix things up. It's natural for academics to be thinking about what future work and social spaces will be like, and to reflect on how best to tweak the student experience with these changes in mind.

What brought this to my mind is Prof Colm Kenny's analysis piece in the Irish Independent this weekend, a response to the recently-published Hunt report ("National strategy for higher education: draft report of the strategy group." 9 August 2010) that tries to set out a vision for Irish 3rd-level (undergraduate degree) education. Although specific to Ireland, the report raises questions for other countries' relationships with their universities too, and so is worth considering broadly.

A casual read of even the executive summary reveals a managerial tone. There's a lot of talk of productivity, broadening access, and governance that ensures that institutions meet performance targets aligned with national priorities. There's very little on encouraging free inquiry, fostering creativity, or equipping students for the 21st century. The report -- and similar noises that have emerged from other quarters, in Ireland and the UK -- feel very ... well ... 20th century.

Life and higher education used to be very easy: you learned your trade, either as an apprentice or at university; you spent forty years practising it, using essentially those techniques you'd been imparted with plus minor modifications; you retired, had a few years off, and then died. But that's past life: future, and indeed current, life aren't going to be like that. For a start, it's not clear when if ever we'll actually get to retire. Most people won't stay in the same job for their entire careers: indeed, a large percentage of jobs that one could do at the start of a career won't even exist forty years later, just as many of those jobs haven't been thought of now. When I did my PhD 20 years ago there was no such thing as a web designer, and music videos were huge projects that no-one without access to a fully-equipped multi-million-pound studio could take on. Many people change career because they want to rather than through the influence of outside forces, such as leaving healthcare to take up professional photography.

What implications does this have for higher education? Kenny rightly points out that, while distance education and on-line courses are important, they're examples of mechanism, not of vision. What they have in common, and what drives their attractiveness, is that they lower the barriers to participation in learning. They actually do this in several ways. They allow people to take programmes without re-locating and potentially concurrently with their existing lives and jobs. They also potentially allow people to "dip-in" to programmes rather than take them to their full extent, to mash-up elements from different areas, institutions and providers, and to democratise the generation and consumption of learning materials.

Some students, on first coming to university, are culture-shocked by the sudden freedom they encounter. It can take time to work out that universities aren't schools, and academics aren't teachers. In fact they're dual concepts: a school is an institution of teaching, where knowledge is pushed at students in a structured manner; a university is an institution of learning, which exists to help students to find and interact with knowledge. The latter requires one to learn skills that aren't all that important in the former.

The new world of education will require a further set of skills. Lifelong learning is now a reality as people re-train as a matter of course. Even if they stay in the same career, the elements, techniques and technologies applied will change constantly. It's this fact of constant change and constant learning that's core to the skills people will need in the future.

(Ten years or so ago, an eminent though still only middle-aged academic came up to me in the senior common room of the university I taught in at the time and asked me when this "internet thing" was going to finish, so that we could start to understand what it had meant. I tried to explain that the past ten years were only the start of the prologue to what the internet would do to the world, but I remember his acute discomfort at the idea that things would never settle down.)

How does one prepare someone for lifelong learning? Actually many of the skills needed are already being acquired by people who engage with the web intensively. Anyone who reads a wide variety of material needs to be able to sift the wheat from the chaff, to recognise hidden agendas and be conscious of the context in which material is being presented. Similarly, people wanting to learn a new field need to be able to determine what they need to learn, to place it in a sensible order, locate it, and have the self-discipline to be able to stick through the necessary background.

It's probably true, though, that most people can't be successful autodidacts. There's a tendency to skip the hard parts, or the background material that (however essential) might be perceived as old and unnecessary. Universities can provide the road maps to avoid this: the curricula for programmes, the skills training, the support, examination, quality assurance and access to the world's foremost experts in the fields, while being only one possible provider of the material being explored. In other words, they can separate the learning material from the learning process -- two aspects that are currently conflated.

I disagree with Colm Kenny on one point. He believes that only government can provide the necessary vision for the future of higher education. I don't think that's necessary at all. A system of autonomous universities can set their own visions of the future, and can design processes, execute them, assess them, measure their success and refine their offerings -- all without centralised direction. I would actually go further, and argue that the time spent planning a centralised national strategy would be better spent decentralising control of the university system and fostering a more experimental approach to learning. That's what the world's like now, and academia's no different.

The computer is the new microscope

What contributions can computer scientists uniquely make to the latest scientific challenges? The answer may require us to step back and look at how instruments affect science, because the computer is the key instrument for the scientific future.

In the late seventeenth century, science was advancing at an extraordinary rate -- perhaps the greatest rate until modern times. The scientists of this era were attempting to re-write the conceptual landscape through which they viewed the universe, and so in many ways were attempting something far harder than we typically address today, when we have a more settled set of concepts that are re-visited only periodically. This also took place in a world with far less of a support infrastructure, in which the scientists were also forced to be tool-makers manufacturing the instruments they needed for their research. It's revealing to look at a list of scientists of this era who were also gifted instrument-makers: Newton, Galileo, Hooke, Huygens and so on.

Antonie van Leeuwenhoek is a classic example. He revolutionised our understanding of the world by discovering single-celled micro-organisms, and by documenting his findings with detailed drawings in letters to the Royal Society. The key instrument in his research was, of course, the microscope, of which he manufactured an enormous number. Whilst microscopes were already known, van Leeuwenhoek developed (and kept secret) new techniques for the manufacture of lenses which allowed him significantly to advance both the practice of optics and the science of what we would now term microbiology.

The important here is not that early scientists were polymaths, although that's also a fascinating topic. What's far more important is the effect that tooling has on science. New instruments not only provide tools with which to conduct science; they also open-up new avenues for science by revealing phenomena that haven't been glimpsed before, or by providing measurements and observations of details that conflict with the existing norms. The point is that tools and science progress together, and therefore that advances in instrument-making are valuable both in their own right and in the wider science they facilitate.

Not all experimental scientists see things this way. It's fairly common for those conducting the science to look down on the instrument-makers as mere technicians, whose efforts are of a different order to those involved in "doing" the science. It's fair to say that the scientists of the seventeenth century wouldn't share (or indeed understand) this view, since they were in a sense much closer to the instrument as a contributor to their research. Looked at another way, new experiments then typically required new instruments, rather than as now generally being conducted with a standard set of tools the researcher has to hand.

What are the instruments today whose advance will affect the wider scientific world? "Traditional" instrument-making is still vitally important, of course, and we can even regard the LHC as a big instrument to used in support of particular experiments. But beyond this we have "non-traditional" instruments, of which computers are by far the most commonplace and potentially influential.

I've talked previously about exabyte-scale science and the ways in which new computing techniques will affect it. Some experimenters overlook the significance of computational techniques -- or, if they do see them, regard them as making technician-level rather than science-level contributions to knowledge. Even more experimenters overlook the impact that more rarefied computer science concerns such as programming languages, meta-data and search have on the advancement of knowledge. These views are restricted, restricting, and (in the worst case) stifling. They are also short-sighted and incorrect.

At the large scale, computational techniques often offer the only way of "experimenting" with large-scale data. They can be used to confirm hypotheses in the normal sense, but there are also examples where they have served to help derive new hypotheses by illuminating factors and phenomena in the data that were previously unsuspected, and furthermore could not have been discovered by any other means. The science is advanced by the application of large-scale computing to large-scale data, possibly collected for completely different purposes.

In that sense the computer is behaving as an instrument that opens-up new opportunities in science: as the new microscope, in fact. This is not simply a technical contribution to improving the way in which traditional science is done: coupled with simulation, it changes both what science is done and how it is done, and also opens-up new avenues for both traditional and non-traditional experiments and data collection. A good example is in climate change, where large-scale simulations of the atmosphere can confirm hypotheses, suggest new ones, and direct the search for real-world mechanisms that can confirm or refute them.

At the other end of the scale, we have sensor networks. Sensor networks will allow experimental scientists directly to collect data "in the wild", at high resolution and over long periods -- things that're difficult or impossible with other approaches. This is the computer as the new microscope again: providing a view of things that were previously hidden. This sort of data collection will become much more important as we try to address (for example) climate change, for which high-resolution long-term data collected on land and in water nicely complement larger-scale space-based sensing. Making such networks function correctly and appropriately is a significant challenge that can't be handled as an after-thought.

At both scales, much of the richness in the data comes from the ways it's linked and marked-up so as to be searched, traversed and reasoned-with. While some experimental scientists develop strong computational techniques, very few are expert in metadata, the semantic web, machine learning and automated reasoning -- although these computer science techniques are all key to the long-term value of large-scale data.

As with the earliest microscopes, the instrument-maker may also be the scientist, but that causes problems perhaps more severe today than in past centuries. Like it or not, we live in an era of specialisation, and in an era where it's impossible to be really expert in one field let alone the several one might need in order to make proper contributions. But the development of new instruments -- computational techniques, sensing, reasoning, matadata cataloguing  -- is nevertheless key to the development of science. In the years after van Leeuwenhoek, several microbiologists formed close collaborations with opticians who helped refine and develop the tools and techniques available -- allowing the microbiologists to focus on their science while the opticians focused on their instruments. (Isn't it interesting how "focused" really is the appropriate metaphor here?) Looked at broadly, it's hard to say which group's contribution was more influential, and in some senses that's the wrong question: both focused on what they were interested in, solving hard conceptual, experimental and technological problems along the way, and influencing and encouraging each other to perfect their crafts.

It's good to see this level of co-operation between computer scientists and biologists, engineers, sustainable-development researchers and the rest beginning to flower again, at both ends of the scale (and at all points in between). It's easy to think of instruments as technical devices devoid of scientific content, but it's better to think of them both as having a core contribution to make to the concepts and practice of science, and as having a fascination in their own right that gives rise to a collection of challenges that, if met, will feed-back and open-up new scientific possibilities. The microscope is a key example of this co-evolution and increased joint value from the past and present: the computer is the new microscope for the present and future.

Five big questions

If you try to do everything, you always end up doing nothing. Which is why Gray's laws suggest searching for the twenty "big questions" in a field and then focusing-in the first five as the ones that'll generate the biggest return on the effort invested. So what are the five biggest open issues in programming for sensorised systems?

Of course we should start with a big fat disclaimer: these are my five biggest open issues, which probably don't relate well to anyone else's -- but that's what blogs are for, right? :-) So here goes: five questions, with an associated suggestion for a research programme.

1. Programming with uncertainty. This is definitely the one I feel is most important. I've mentioned before that there's a mismatch between traditional computer science and what we have to deal with for sensor systems: the input is uncertain and often of very poor quality, but the output  behaviour has to be a "best attempt" based on what's available and has to be robust against small perturbations due to noise and the like. But uncertainty is something that computers (and computer scientists) are quite bad at handling, so there's a major change that has to happen.

To deal with this we need to re-think the programming models we use, and the ways in which we express behaviour. For example we could look at how programs respond to perturbation, or design languages in which perturbations have a small impact by design. A calculus of stable functions might be a good starting-point, where perturbation tends to die-out over time and space, but long-term changes are propagated. We might also look at how to program more effectively with Bayesian statistics, or how to program with machine leaning: turn things that are currently either libraries or applications into core constructs from which to build programs.

2. Modeling adaptive systems as a whole. We've had a huge problem getting systems to behave according to specification: now we propose that they adapt in response to changing circumstances. Clearly the space of possible stimuli and responses are too large for exhaustive testing, or for formal model-checking, so correctness becomes a major issue. What we're really interested in, of course, isn't so much specifying what happens as much as how what happens changes over time and with context.

Holistic models are common in physics but uncommon in computer science, where more discrete approaches (like model checking) have been more popular. It's easy to see why this is the case, but a small-scale, pointwise formal method doesn't feel appropriate to the scale of the problem. Reasoning about a system as a whole means re-thinking how we express both specifications and programs. But the difference is target is important too: we don't need to capture all the detail of a program's behaviour, just those aspects that relate to properties like stability, response time, accuracy and the like -- a macro method for reasoning about macro properties, not something that gets lost in the details. Dynamical systems might be a good model, at least at a conceptual level, with adaptation being seen as a "trajectory" through the "space" of acceptable parameter values. At the very least this makes adaptation an object of study in its own right, rather than being something that happens within another, less well-targeted model.

3. Modeling complex space- and time-dependent behaviours. Reasoning systems and classifiers generally only deal with instants: things that are decided by the state of the system now, or as what immediately follows from now. In many cases what happens is far richer than this, and one can make predictions (or at least calculate probabilities) about the future based on classifying a person or entity as being engaged in a particular process. In pervasive computing this manifests itself as the ways in which people move around a space, the services they access preferentially in some locations rather than others, and so forth. These behaviours are closely tied-up with the way people move and the way their days progress, as it were: complex spatio-temporal processes giving rise to complex behaviours. The complexities come from how we divide-up people's actions, and how the possibilities branch to give a huge combinatorial range of possibilities -- not all of which are equally likely, and so can be leveraged.

A first step at addressing this would be to look at how we represent real-world spatio-temporal processes with computers. Of course we represent such processes all the time as programs, but (linking back to point 1 above) the uncertainties involved are such that we need to think about these things in new ways. We have a probabilistic definition of the potential future evolutions, against which we need to be able to express behaviours synthesising the "best guesses" we can make and simultaneously use the data we actually observe to validate or refute our predictions and refine our models. The link between programming and the modelingthat underlies it looks surprisingly intimate.

4. Rich representations of linked data. Sensors generate a lot of data. Much of it has long-term value, if only for verification and later re-study. Keeping track of all this data is going to become a major challenge. It's not something that the scientists for whom it's collected are generally very good at -- and why should they be, given that their interests are in the science and not in data management? But the data has to be kept, has to be retrievable, and has to be associated with enough metadata to make its properly and validly interpretable in the future.

Sensor mark-up languages like SensorML are a first step, but only a first step. There's also the issue of the methodology by which the data was collected, and especially (returning to point 2) were the behaviours of the sensors consistent with gaining a valid view of the phenomena of interest? That means linking data to process descriptions, or to code, so that we can track-back through the provenance to ensure integrity. Then we can start applying reasoners to classify and derive information automatically from the data, secure in the knowledge that we have an audit trail for the science.

5. Making it easier to build domain-specific languages for real. A lot has been said about DSLs, much of it negative: if someone's learned C (or Java, or Fortran, or Matlab, or Perl, or...) they won't want to then learn something else just to work in a particular domain. This argument holds that it's therefore more appropriate to provide advanced functions as libraries accessed from a common host language (or a range of languages). The counter-argument is that libraries only work around the edges of a language and can't provide the strength of optimisation, type-checking and new constructs needed. I suspect that there's truth on both sides, and I also suspect that many power users would gladly learn a new language if it really matched their domain and really gave them leverage.

Building DSLs is too complicated, though, especially for real-world systems that need to run with reasonable performance on low-grade hardware. A good starting-point might be a system that would allow libraries to be wrapped-up with language-like features -- like Tcl was intended for, but with more generality in terms of language constructs and types. A simpler language-design framework would facilitate work on new languages (as per point 1 above), and would allow us to search for modes of expression closer to the semantic constructs we think are needed (per points 2 and 3): starting from semantics and deriving a language rather than vice versa.