Recently there’s been an exchange in the Irish media about the decline of intellectuals in universities and calling into question whether universities are still fit for purpose given their funding and management structures. The fundamental question seems to me to be far deeper, and impacts on the UK and elsewhere as much as Ireland: what is  — or should be — the relationship between academics and their funding sources? To what extent does the person paying the academic piper call the research tune? The opening shot in this discussion was published in the Irish Times as an op-ed piece by Tom Garvin, an emeritus (retired but still recognised) professor of politics at UCD, Ireland’s largest university, although perhaps his intervention was triggered by a rather more supportive editorial in the same paper a few days before. The piece is something of a polemic, but revolves around the lack of support for curiosity-driven (“blue skies”) research within the university system. “A grey philistinism [Garvin writes] has established itself in our universities, under leaders who imagine that books are obsolete, and presumably possess none themselves.” He then goes on to attack the application of science-based research methods such as team-based investigation to humanities, and observes that even for sciences there is no longer scope for free inquiry: “Researchers are being required by bureaucrats to specify what they are going to discover before the money to do the research is made available.” The article provoked a large number of supportive comments from readers and a rather narrow response from UCD’s management team. The questions Garvin raises deserve consideration, and he should be commended for raising them. Indeed, they cut to the heart of whether universities are fit for purpose. Does the linking of funding to student numbers (whether under- or post-graduate) and to the pursuit of State-guided research programmes, the desire to profit from commercially significant results, and the need (or at least desire) to excel in international league tables impact on academics’ ability to investigate the world and pass on their results freely? (For full disclosure, I should point out that I’m a computer science academic who, until last year, worked at UCD.) There are, of course, major differences in topic area and research philosophy between research in sciences and in humanities, and I’m sure it’s fair to say that the same research approaches won’t translate seamlessly from one to the other: studying and commenting on the writings of others wouldn’t constitute computer science, for example. Having said that, though, there are advantages to be had from considering the ways in which techniques translate across fields. I’m sure some projects in history (to use Garvin’s example) would benefit from a team-based approach, although clearly others wouldn’t — any more than all physics research is conducted in large teams with large machines. As a science researcher I have to point out that the crux of this argument — specifying results before being granted funding — is of course nonsense. When I apply for a grant I’m required to describe what I intend to investigate, the methods I’ll use and the expected benefits of the investigation. This of course is a long way from specifying what I’ll find, which of course I don’t know, in detail: that’s the whole point, after all. I don’t find this to be the mental straitjacket that Garvin seems to think it is: quite the contrary, spelling these matters out is an essential planning phase to design and manage what is likely to be a complex, elongated and collaborative endeavour. If I can’t explain what I want to do in a research proposal, then I don’t know myself. But the real question isn’t about management: its about justification. Research proposals can (and probably should) be written for any research programme, but they’re typically used to get funding. And it’s here that we encounter the constraints on blue-skies research. A funding agency, whether public or private, will only fund research from which they’ll get an acceptable impact and return on investment. Different agencies will judge that return differently. Commercial funding usually follows topics that will lead, directly or indirectly, to future profits: it’s hard to see how a company’s board would be meeting its fiduciary duties if it did otherwise. Private foundations and State agencies such as research councils generally have a different definition of impact, for example the abstract scientific significance of a topic or its ability to effect social change. So to that extent, an academic who takes on research funding agrees, at least tacitly, to support the funder’s chosen model for assessing their research. There are some nasty examples where this causes problems, the classic case being research sponsored by tobacco companies, and taking funding doesn’t excuse from their duties as scientists to collect and report their findings honestly and completely, but in the main the system doesn’t cause too many dilemmas. Given that many universities derive a large fraction of their income (approaching 100% for some) from the State, it is natural that the State will have an interest in directing resources to topics that it regards as more significant. To do otherwise would be to neglect the State’s responsibility to the electorate. Academics are not democratically accountable: it is unreasonable to expect that they should be responsible for the allocation funds they have no authority to collect. However, there’s another point that needs to be made. Most academic appointments centre around teaching and conducting research, and not around getting funding. This has important implications for Garvin’s argument. The UK and Irish university systems are such that there is absolutely no constraint on an academic’s ability to do curiosity-driven research: I can conduct research on whatever topic I choose, and publish it (or not) as I see fit, at my absolute discretion (absent ethical considerations). I would be meeting fully my obligations to my employer, since these relate to my meeting my teaching and research commitments. But I can’t expect an external agency to fund me additionally, and it would be surprising if it were otherwise. Put another way, my academic position is my opportunity to conduct my own  curiosity-driven research, but if I require resources beyond my own I have to justify the wider benefits that will derive from that additional funding. There may be other side-effects: certainly in sciences, anyone without a significant funding profile would struggle to be promoted. But a requirement to justify the value one expects from a research programme before it’s funded is hardly a crimp on academic freedom. There’s also a mis-understanding in some circles about how science is conducted: a perception that scientists go rather robotically about the conduct of the scientific method, collecting data and then drawing conclusions. This is hardly a surprising mis-perception, given that it’s almost implicit in the structure of a typical paper. But it nonetheless mis-states the importance of intuition and curiosity in deciding which hypothesis — amongst the infinite possibilities — one chooses to explore. (Peter Medawar. Is the scientific paper a fraud? The Listener 70, pp. 337-378. 12 September 1963.) Without curiosity there can’t really be any science: in fact one could go further and say that science is simply curiosity with a bit of useful structure attached. Garvin’s other point, about the lack of influence of academics on public life, was picked by Kevin Myers in the Irish Independent as evidence of cowardice and a disinclination to risk-taking. From the perspective of the humanities this may be true, and it’s not for me to comment, but from the perspective of the sciences academics have become more engaged over time in the big issues: climate science, online-privacy and evolutionary biology are the three highest-profile examples of where science academics have stepped into the public realm. It’s also important to note that these three examples are first and foremost social in their impact: the science is there to inform public understanding, debate and policy. It’s unfortunate that in many of these discussions “academic” has been taken tacitly to imply “academic in the humanities” — as though scientists were somehow less “academic” (whatever that means). In the past it was perhaps more expected for humanities academics to engage as public intellectuals, but a lot of the modern world’s problems simply can’t be understood, let alone be debated and addressed, without the serious and deep engagement of the scientists themselves in the debate.  This, perhaps, is the biggest challenge: to capture, leverage and pass on the excitement for learning and for research into the wider community. As long as that continues to be a central goal, universities remain fit for purpose.

Just as we can think of large-scale, detailed financial modeling as an exercise in simulation and linked data, we can perhaps also see the detection of hazards as an exercise in sensor fusion and pervasive computing, turning a multitude of sensor data into derived values for risk and/or situations requiring attention. There’s a wealth of research in these areas that might be applicable. Environmental and other sensing is intended to make real-world phenomena accessible directly to computers. Typically we simply collect data and archive it for later analysis; increasingly we also move decision-making closer to the data in order to take decisions in close to real time about the ways in which the data is sensed in the future (so-called adaptive sensing) or to allow the computers to respond directly in terms of the services they provide (autonomic or pervasive systems). Can we treat the financial markets as the targets of sensing? Well, actually, we already do. An index like the FTSE is basically providing an abstracted window onto the behaviour of an underlying process — in this case a basket of shares from the top 100 companies listed on the London exchange — that can be treated as an observation of an underlying phenomenon. This further suggests that the technology developed for autonomic and pervasive computing could potentially be deployed to observe financial markets. In some sense, pricing is already based on sensing. A put option, for example — where the buyer  gains the right to compel the seller to buy some goods at some point in the future at some defined cost — will, if exercised, have a definite value to the buyer then (when executed). It’s value now (when sold) will be less than this, however, because of the risk that the option will not be exercised (because, for example, the buyer can sell the goods to someone else for more than the seller has contracted to pay for them). Deciding what value to assign to this contract is then a function over the expected future behaviour of the market for the underlying goods. This expectation is formed in part by observing the behaviour of the market in the past, combined with the traders’ knowledge of (or guesses about) external factors that might affect the price. These external factors are referred to in pervasive computing as context, and are used to condition the ways in which sensor streams are interpreted (see Coutaz et alia for an overview). One obtains context from a number of sources, typically combining expert knowledge and sensor data. A typical pervasive system will build and maintain a context model bringing together all the information it knows about in a single database. We can further decompose context into primary context sensed directly from a data source and secondary context derived by some reasoning process. If we maintain this database in a semantically tractable format such as RDF, we can then reason about what’s happening in order to classify what’s happening in the real world (situation recognition) and respond accordingly. Crucially, this kind of context processing can treat all context as being sensed, not just real-world data: we often “sense” calendars, for example, to look for clues about intended activities and locations, integrating web mining into sensing. Equally crucially, we use context as evidence to support model hypotheses (“Simon is in a meeting with Graeme and Erica”) given by the situations we’re interested in. A lot of institutions already engage in automated trading, driven by the behaviour of indices and individual stocks. Cast into sensor-driven systems terminology, the institutions develop a number of situations of interest (a time to buy, hold, sell and so forth for different portfolios) and recognise which are currently active using primary context sensed from the markets (stock prices, indices) and secondary context derived from this sensed data (stock plummeting, index in free-fall). Recognising a situation leads to a particular behaviour being triggered. Linked data opens-up richer opportunities for collecting context, and so for the management of individual products such as mortgages. We could, for example, sense a borrower’s repayment history (especially for missed payments) and use this both to generate secondary context (revised risk of default) and to identify situations of interest (default, impaired, at-risk). Banks do this already, of course, but there are advantages to the sensor perspective. For one, context-aware systems show us that it’s the richness of links between  context that is the key to its usefulness. The more links we have, the more semantics we have over which to reason. Secondly, migrating to a context-aware platform means that additional data streams, inferences and situations can be added as-and-when required, without needing to re-architect the system. Given the ever-increasing amount of information available on-line, this is certainly something that might become useful. Of course there are massive privacy implications here, not least in the use of machine classifiers to evaluate — and of course inevitably mis-evaluate — individuals’ circumstances. It’s important to realise that this is going on anyway and isn’t going to go away: the rational response is therefore to make sure we use the best approaches available, and that we enforce audit trails and transparency to interested parties. Credit scoring systems are notoriously opaque at present — I’ve had experience of this myself recently, since credit history doesn’t move easily across borders — so there’s a screaming need for systems that can explain and justify their decisions. I suspect that the real value of a sensor perspective comes not from considering an individual institution but rather an entire marketplace. To use an example I’m familiar with from Ireland, one bank at one stage pumped its share price by having another bank make a large deposit — but then loaned this second bank the money to fund the deposit. Contextualised analysis might have picked this up, for example by trying to classify what instruments or assets each transaction referred to. Or perhaps not: no system is going to be fully robust against the actions of ingenious insiders. The point is not to suggest that there’s a foolproof solution, but rather to increase the amount and intelligence of surveillance in order to raise the bar. Given the costs involved in unwinding failures when detected late, it might be an investment worth making.

Science should be a two-way street. In the same way that we might view the financial system as a computational model that can be simulated, there are undoubtedly things that computing can learn from finance. In particular, a lot of financial instruments and techniques evolved because of uncertainty and risk: issues that are absolutely a part of computer science in general and of sensorised systems in particular. The financial crunch hasn’t invalidated the  significance that these techniques may have for computer science and systems architecture. There ‘s an increasing acceptance that economics as a whole has lessons for computing. The connection may not be obvious, but if one considers that most modern systems of any scale are built from components, and often from agents whose goals are not necessarily completely aligned and whose actions need not be trusted, one begins to see how a model that assumes only self-interest might have relevance. Typically economics is used to provide models for large-scale behaviour, rather than being treated as a way to formulate or program the agents themselves. (For a survey of recent results see Neumann, Baker, Altmann and Rana (eds). Economic models and algorithms for distributed systems. Birkhäuser. 2010.) How about the individual instruments, though? Let’s take a step back and think what a financial instrument actually is, or rather what it’s goals are. To make money, clearly, but making money is easy if you know the exact circumstances in which a transaction will take place. It’s generally accepted in the economics of ordinary trade, for example, that both sides of the transaction win from the trade: the seller will only sell if the goods sell for more than he spent in creating them, while the buyer will only buy if the goods have a higher utility value to him than the cost of their purchase. (It’s interesting that it’s necessary to point this out, since a lot of people tacitly assume that trade is only good for one side or the other, not for both.) The different between this sort of economic activity and financial instruments such as shares and derivatives is that the traders know (most of) the circumstances that affect their valuations, whereas the value of a mortgage or share is affected by events that haven’t happened yet. If we represent instruments as computational objects, then we can look at the ways in which this uncertainty might be managed. A common way is to identify the risks that affect the value of an instrument, and hedge them. To hedge a risk, you find another instrument whose value will change in a way that balances that of the initial risk. The idea is that a loss on the first is compensated, at least in part, by a gain on the second. The best example I can find — given my very limited understanding — is airline fuel. An airline knows it needs fuel in the future, and can even estimate how much it will need, but doesn’t want to buy it all up-front because things might change. This means that the airline doesn’t know how much the fuel will cost, since the price may change. A large increase would cost money since tickets already sold would reflect the earlier, lower fuel price. Passing this on later as a surcharge is extremely unpopular. What the airline can do is to buy a fuel future, a call option, that gives it the right to buy fuel from someone at a fixed price per litre. That is, someone guarantees that they will sell the airline the fuel it needs in (say) six months at a given price. The option itself costs money, above and beyond the cost of the fuel, but if the prices are right the airline is insulated against surges in the fuel price. If in six months the cost of fuel is higher than the cost in the call option, the airline exercises the option, buys the fuel, and makes money versus what it would have paid; if the price is less than that in the call option, the airline just writes-off the cost of the option and buys the fuel in the market. Either way the airline caps its exposure and so controls its future risk. (If this all sounds one-sided, the entity selling the option to the airline needs to make money too — perhaps by having a source of fuel at a known price, or having a stockpile on hand, or simply betting that the prices will move in a certain way.) There is already experience in writing financial instruments, including derivatives and the like, using programming languages. (See, for example, Peyton Jones and Eber. Composing contracts: an adventure in financial engineering. Proc. ICFP. Montreal, CA. 2000.) Instruments represented as computational objects can be linked to their hedges. If we’re simulating the the market, we can also simulate the values of hedges and see under what circumstances their values would fall alongside the original assets. That shows up risks of large crashes. At present this is done at a coarse statistical level, but if we link instruments to their metadata we can get a much finer simulation. We can potentially use a similar technique for any system that’s exposed to future uncertainty, though. Systems that do inference have to live the risk that their inferences will be wrong. In a pervasive computing system such as a smart building, for example, one typically looks at a range of sensors to try to determine what’s happening and respond accordingly (by opening doors, providing information, sounding alarms or whatever else is needed by the application). Most such actions occur by inference, and can typically be wrong. How might the lessons of finance help? We have a process that is dealing with future uncertainty, and whose utility is governed by how well it manages to address that uncertainty relative to what’s “really happening” in the world. If we rely on our single inference process, and it makes a mistake — as it inevitably will — we’re left completely exposed: the utility of the system is reliant on the correctness of the single inference function. Put in the terms above, it isn’t hedged. So we might require each inference process to come with a hedge. That is to say, each process not only specifies what should happen as a result of its determination of the current situation; it also specifies what should happen if we get it wrong, if later evidence shows that the inference we made was wrong. This might be something simple like reversing the action we took: we turned the lights on because we thought a room was occupied, and in mitigation if we’re wrong we turn the lights off again. The “cost” of this action is some wasted power. Some processes aren’t reversible, of course: if we (think we) recognise someone at a door, and open it to admit them, and then discover we made a mistake and it’s not the person we thought, just locking the door again won’t work. We could however, take some other action (sound an alarm?), and accept a “cost” that is expressed in terms of the inappropriate or malicious actions the person might have taken as a result of our mistake. Moreover because we’ve combined the action and its hedge, we can assess the potential costs involved and perhaps change the original inference algorithm to for example require more certainty. Essentially each behaviour that’s triggered comes with another, related behaviour that’s triggered in the event that the first one shouldn’t have been. There are some interesting lifecycle issues involved turning this into a programming system, since the lifetime of the hedging behaviour might be longer than that of the original behaviour: even after the person has left the room, we might want to sound the alarm if we realise we made a mistake letting them in. The costs of mistakes might also be a function of time, so that problems detected later are more costly. The point here is not that pervasive and sensor systems can get things wrong — of course they can — but that we need to design systems on the assumption that they will get things wrong, and account for the costs (financial and otherwise) that this exposes us to and the actions we can take to mitigate them. The financial system does this: it recognises that there is uncertainty in the world and tries to construct a network of agents which will gain value regardless of the uncertain events that happen. The process isn’t perfect, as we’ve seen in recent financial events, but it’s at least a recognition within the system of the impact that uncertainty will have. That’s something we could learn in, and incorporate into, computer science and systems engineering.

Many people expected a financial crisis; many also expected it to be caused by automated trading strategies driving share prices, As it turned out, that aspect of computing in finance didn’t behave as badly as expected, and the problems arose in the relatively computing-free mortgage sector. Is there any way computing could have helped, or could help avoid future crises? Over brunch earlier this week my wife Linda was worrying about the credit crunch. It’s especially pressing since we’re in Ireland at the moment, and the whole country is convulsed with the government’s setting-up of the National Asset Management Agency (NAMA) to take control of a huge tranche of bad debts from the Irish banks. She asked me, “what can you do, as a computer scientist, to fix this?” Which kind of puts me on the spot thinking about a topic about which I know very little, but it got me thinking that there may be areas where programming languages and data-intensive computing might be relevant for the future, at least. So here goes…. The whole financial crisis has been horrendously complex, of course, so let’s start by focusing on the area I know best: the Irish “toxic tiger” crash. This has essentially been caused by banks making loans to developers to finance housing and commercial property construction, both of which were encouraged through the tax system. Notably the crash was not caused by loans to sub-prime consumers and subsequent securitisation, and so is substantively different to the problems in the US (although triggered by them through tightening credit markets). The loans were collateralised on the present (or sometimes future, developed) value of the land — and sometimes on “licenses to build” on land rather than the land itself — often cross-collateralised with several institutions and developments, and financed by borrowings from the capital markets rather than from deposits (of which Ireland, as a country of 4M people, has rather little). As capital availability tightened across 2008-9 the property market stalled, the value of the land dropped (by 80% in some cases, and by 100% for the licenses to build), and the banks have been left with bad loans and capital shortcomings in the range of at least EUR60B which now the government, for reasons strongly suspected to be political and ideological rather than being necessarily in the best interests of the taxpayer, are taking onto the public balance sheet rather than allowing them to be carried by the banks’ owners and bondholders. The crash has also, as might be expected, revealed enormous shortcomings in banks’ risk management, their understanding of their own holdings and exposures, some unbelievably lax supervision by the authorities, and a lot of suspected corruption and corporate fraud. (The above is a gross simplification, of course. The Irish Economy blog is the best source of details, being run by a large fraction of Ireland’s leading academic economists who have been depressingly accurate as to how the crisis would unfold.) So what can computer science say about this mess? To start with, there are several key points we can pull out of the above scenario:

1. Modelling. Its hard to know what the effect of a given stressor would be on the system: what exactly happens if there’s sector-wide unemployment, or a localised fall in purchasing?. Most banks seem to have conducted modelling only at the grossed statistical level.
2. Instrument complexity. Different financial instruments respond to stimuli in different ways. A classic example is where unpaid interest payments are rolled-up into the principal of a mortgage, changing its behaviour completely. Parcelling-up these instruments makes their analysis next to impossible.
3. Traceability. The same assets appear in different places without being detected up-front, which makes all the instruments involved significantly more risky and less valuable.
4. Public trust. The “stress tests” conducted by regulators were conducted in secret without independent scrutiny, as have been the valuations applied to the bad loans. The public is therefore being asked to sign-off on a debt of which it knows nothing.

Clearly this is very complicated, and statistical modelling is only going to provide us with an overview. The simplifications needed to get the mathematics to work will be heroic, despite the power of the underlying analytic techniques. So let’s treat the problem as one of simulation rather than analysis.A mortgage is generally treated as data with a particular principal, interest rate, default rate and so on. It can however also be viewed as process, a computational object: at each accounting period (month) it takes in money (mortage payment) and consequently has a value going forward. There is a risk that the payment won’t come in, which changes the risk profile and risk-weighted value of the mortgage. It will respond in a particular way, perhaps by re-scheduling payments (increasing the term of the mortgage), or trying to turn itself into a more liquid object like cash (foreclosing on the loan and selling the house), Foreclosure involves interacting with other objects that model the potential behaviour of buyers, to indicate how much cash the foreclosure brings in: in a downturn, the liklihood of payment default increases and the cash value of mortgages forclosed-upon similarly reduces. The point is that there’s relatively little human, banking intervention involved here: it’s mostly computational. One could envisage a programming language for expressing the behaviours of mortgages, which defines their responses to different stimuli and defines an expected present value for the mortgage discounted by the risks of default, the amount recoverable by foreclosure, and so on. So the first thing computer science can tell us about the financial crash is that the underlying instruments are essentially computational, and can be represented as code. This provides a reference model for the behaviour we should expect from a particular instrument exposed to particular stimuli, expressed clearly as code. We can go a stage further. If loans are securitised — packaged-up into another instrument whose value is derived from that of the underlying assets, like a futures contract — then the value of the derivative can be computed from the values of the assets underlying it. Derivatives are often horrifically complicated, and their might be significant advantages to be had in expressing their behaviours as code. How do we get the various risk factors? Typically this is done at a gross level across an entire population, but it need not be. We now live in an exabyte era. We can treat the details of the underlying asset as metadata on the code: the location of a house, its size and price history, the owners’ jobs and so forth. We currently have this data held privately, and as statistical aggregates, but there’s no reason why we can’t have associate the actual details to each loan or instrument, and therefore to each derivative constructed from them. This after all is what linked data is all about. means that each financial instrument is inherently computational, and carries with it all the associated metadata. This little package is the loan, to all intents and purposes. So the second thing computer science can tell us is that we can link instruments, assets and data together, and track between them, using the standard tools and standards of the semantic web. This means we can query them at a high semantic level, and us these queries to extract partitions of the data we’re interested in examining further. There’s no scientific reason that this can’t be done across an entire market, not simply within a single institution. The net result of the above is that, given this coding and linking, the financial system can be run in simulation. We can conduct stress tests at a far finer resolution by for example running semantic queries to extract a population of interest, making them all redundant (in simulation), and seeing what happens not only to their loans, but to the securitised products based on them — since everythings just a computation. Multiple simulations can be run to explore different future scenarios, based on different risk weightings an so forth. (This sounds like a lot of data, so let’s treat the UK housing market as a Fermi problem and see if it’s feasible. There are 60M people in the UK. Assume 2-person families on average, yielding 30M domestic houses to be mortgaged. Some fraction of these are mortgage-free, say one third, leaving 20M mortgages. The workforce is around 40M working in firms with an average of say 20 employees, each needing premises, yielding a further 2M commercial mortages. If each mortgage needs 10Kb of data to describe it, we have 22M objects requiring about 250Tb of data: a large but not excessive data set, especially when most objects execute the same operations and share a lot of common data: certainly well within the simulation capabilities of cloud computing. So we’re not in computationally infeasible territory here.) We can actually go a few stages further. Since we have a track-back from instrument to metadata, we can learn the risks over time by observing what happens to specific cohorts of borrowers exposed to different stimuli and stresses. Again, linked data lets us identify the patterns of behaviour happening in other data sets, such as regional unemployment (now directly available online in the UK and elsewhere). The more data goes online the easier it is to spot trends, and the easier and finer one can learn the behaviors the system is exhibiting. As well as running the simulation forwards to try to see what’s coming, we can run it backwards to learn and refine parameters that can then be used to improve prediction. Therefore the third think computer science can tell us is that the financial markets as a whole are potentially a rich source of machine learning and statistical inference, which can be conducted using standard techniques from the semantic web. Furthermore, we can conduct simulations in the open. If banks have to represent their holdings as code, and have to link to (some of) the metadata associated with a loan, then regulators can run simulations and publish their results. There’s a problem of commercial confidentiality, of course, but one can lock-down the fine detail of metadata if required (identifying owners by postcode and without names, for example). If each person, asset and loan has a unique identifier, it’s easier to spot cross-collateralisations and other factors that weaken the values of instruments, without needing to be able to look inside the asset entirely. This exposes a bank’s holdings in metadata terms — residential loans in particular areas — but that’s probably no bad thing, given that the lack of information about securitise contributed to the fall. This is the last thing computer science can tell us. Open-source development suggests that having more eyes on a problem reduces the number, scope and severity of bugs, and allows for re-use and re-purposing far beyond what might be expected a priori. For a market, more eyes means a better regulatory and investor understanding of banks’ positions, and so (in principle) a more efficient market. For all I know, banks already do a lot of this internally, but making it an open process could go a long way to restore confidence in both taxpayers and future investors. There’s no time like a crash to try out new ideas.

What contributions can computer scientists uniquely make to the latest scientific challenges? The answer may require us to step back and look at how instruments affect science, because the computer is the key instrument for the scientific future. In the late seventeenth century, science was advancing at an extraordinary rate — perhaps the greatest rate until modern times. The scientists of this era were attempting to re-write the conceptual landscape through which they viewed the universe, and so in many ways were attempting something far harder than we typically address today, when we have a more settled set of concepts that are re-visited only periodically. This also took place in a world with far less of a support infrastructure, in which the scientists were also forced to be tool-makers manufacturing the instruments they needed for their research. It’s revealing to look at a list of scientists of this era who were also gifted instrument-makers: Newton, Galileo, Hooke, Huygens and so on. Antonie van Leeuwenhoek is a classic example. He revolutionised our understanding of the world by discovering single-celled micro-organisms, and by documenting his findings with detailed drawings in letters to the Royal Society. The key instrument in his research was, of course, the microscope, of which he manufactured an enormous number. Whilst microscopes were already known, van Leeuwenhoek developed (and kept secret) new techniques for the manufacture of lenses which allowed him significantly to advance both the practice of optics and the science of what we would now term microbiology. The important here is not that early scientists were polymaths, although that’s also a fascinating topic. What’s far more important is the effect that tooling has on science. New instruments not only provide tools with which to conduct science; they also open-up new avenues for science by revealing phenomena that haven’t been glimpsed before, or by providing measurements and observations of details that conflict with the existing norms. The point is that tools and science progress together, and therefore that advances in instrument-making are valuable both in their own right and in the wider science they facilitate. Not all experimental scientists see things this way. It’s fairly common for those conducting the science to look down on the instrument-makers as mere technicians, whose efforts are of a different order to those involved in “doing” the science. It’s fair to say that the scientists of the seventeenth century wouldn’t share (or indeed understand) this view, since they were in a sense much closer to the instrument as a contributor to their research. Looked at another way, new experiments then typically required new instruments, rather than as now generally being conducted with a standard set of tools the researcher has to hand. What are the instruments today whose advance will affect the wider scientific world? “Traditional” instrument-making is still vitally important, of course, and we can even regard the LHC as a big instrument to used in support of particular experiments. But beyond this we have “non-traditional” instruments, of which computers are by far the most commonplace and potentially influential. I’ve talked previously about exabyte-scale science and the ways in which new computing techniques will affect it. Some experimenters overlook the significance of computational techniques — or, if they do see them, regard them as making technician-level rather than science-level contributions to knowledge. Even more experimenters overlook the impact that more rarefied computer science concerns such as programming languages, meta-data and search have on the advancement of knowledge. These views are restricted, restricting, and (in the worst case) stifling. They are also short-sighted and incorrect. At the large scale, computational techniques often offer the only way of “experimenting” with large-scale data. They can be used to confirm hypotheses in the normal sense, but there are also examples where they have served to help derive new hypotheses by illuminating factors and phenomena in the data that were previously unsuspected, and furthermore could not have been discovered by any other means. The science is advanced by the application of large-scale computing to large-scale data, possibly collected for completely different purposes. In that sense the computer is behaving as an instrument that opens-up new opportunities in science: as the new microscope, in fact. This is not simply a technical contribution to improving the way in which traditional science is done: coupled with simulation, it changes both what science is done and how it is done, and also opens-up new avenues for both traditional and non-traditional experiments and data collection. A good example is in climate change, where large-scale simulations of the atmosphere can confirm hypotheses, suggest new ones, and direct the search for real-world mechanisms that can confirm or refute them. At the other end of the scale, we have sensor networks. Sensor networks will allow experimental scientists directly to collect data “in the wild”, at high resolution and over long periods — things that’re difficult or impossible with other approaches. This is the computer as the new microscope again: providing a view of things that were previously hidden. This sort of data collection will become much more important as we try to address (for example) climate change, for which high-resolution long-term data collected on land and in water nicely complement larger-scale space-based sensing. Making such networks function correctly and appropriately is a significant challenge that can’t be handled as an after-thought. At both scales, much of the richness in the data comes from the ways it’s linked and marked-up so as to be searched, traversed and reasoned-with. While some experimental scientists develop strong computational techniques, very few are expert in metadata, the semantic web, machine learning and automated reasoning — although these computer science techniques are all key to the long-term value of large-scale data. As with the earliest microscopes, the instrument-maker may also be the scientist, but that causes problems perhaps more severe today than in past centuries. Like it or not, we live in an era of specialisation, and in an era where it’s impossible to be really expert in one field let alone the several one might need in order to make proper contributions. But the development of new instruments — computational techniques, sensing, reasoning, matadata cataloguing  — is nevertheless key to the development of science. In the years after van Leeuwenhoek, several microbiologists formed close collaborations with opticians who helped refine and develop the tools and techniques available — allowing the microbiologists to focus on their science while the opticians focused on their instruments. (Isn’t it interesting how “focused” really is the appropriate metaphor here?) Looked at broadly, it’s hard to say which group’s contribution was more influential, and in some senses that’s the wrong question: both focused on what they were interested in, solving hard conceptual, experimental and technological problems along the way, and influencing and encouraging each other to perfect their crafts. It’s good to see this level of co-operation between computer scientists and biologists, engineers, sustainable-development researchers and the rest beginning to flower again, at both ends of the scale (and at all points in between). It’s easy to think of instruments as technical devices devoid of scientific content, but it’s better to think of them both as having a core contribution to make to the concepts and practice of science, and as having a fascination in their own right that gives rise to a collection of challenges that, if met, will feed-back and open-up new scientific possibilities. The microscope is a key example of this co-evolution and increased joint value from the past and present: the computer is the new microscope for the present and future.