Just as we can think of large-scale, detailed financial modeling as an exercise in simulation and linked data, we can perhaps also see the detection of hazards as an exercise in sensor fusion and pervasive computing, turning a multitude of sensor data into derived values for risk and/or situations requiring attention. There’s a wealth of research in these areas that might be applicable.
Environmental and other sensing is intended to make real-world phenomena accessible directly to computers. Typically we simply collect data and archive it for later analysis; increasingly we also move decision-making closer to the data in order to take decisions in close to real time about the ways in which the data is sensed in the future (so-called adaptive sensing) or to allow the computers to respond directly in terms of the services they provide (autonomic or pervasive systems).
Can we treat the financial markets as the targets of sensing? Well, actually, we already do. An index like the FTSE is basically providing an abstracted window onto the behaviour of an underlying process — in this case a basket of shares from the top 100 companies listed on the London exchange — that can be treated as an observation of an underlying phenomenon. This further suggests that the technology developed for autonomic and pervasive computing could potentially be deployed to observe financial markets.
In some sense, pricing is already based on sensing. A put option, for example — where the buyer gains the right to compel the seller to buy some goods at some point in the future at some defined cost — will, if exercised, have a definite value to the buyer then (when executed). It’s value now (when sold) will be less than this, however, because of the risk that the option will not be exercised (because, for example, the buyer can sell the goods to someone else for more than the seller has contracted to pay for them). Deciding what value to assign to this contract is then a function over the expected future behaviour of the market for the underlying goods. This expectation is formed in part by observing the behaviour of the market in the past, combined with the traders’ knowledge of (or guesses about) external factors that might affect the price.
These external factors are referred to in pervasive computing as context, and are used to condition the ways in which sensor streams are interpreted (see Coutaz et alia for an overview). One obtains context from a number of sources, typically combining expert knowledge and sensor data. A typical pervasive system will build and maintain a context model bringing together all the information it knows about in a single database. We can further decompose context into primary context sensed directly from a data source and secondary context derived by some reasoning process. If we maintain this database in a semantically tractable format such as RDF, we can then reason about what’s happening in order to classify what’s happening in the real world (situation recognition) and respond accordingly. Crucially, this kind of context processing can treat all context as being sensed, not just real-world data: we often “sense” calendars, for example, to look for clues about intended activities and locations, integrating web mining into sensing. Equally crucially, we use context as evidence to support model hypotheses (“Simon is in a meeting with Graeme and Erica”) given by the situations we’re interested in.
A lot of institutions already engage in automated trading, driven by the behaviour of indices and individual stocks. Cast into sensor-driven systems terminology, the institutions develop a number of situations of interest (a time to buy, hold, sell and so forth for different portfolios) and recognise which are currently active using primary context sensed from the markets (stock prices, indices) and secondary context derived from this sensed data (stock plummeting, index in free-fall). Recognising a situation leads to a particular behaviour being triggered.
Linked data opens-up richer opportunities for collecting context, and so for the management of individual products such as mortgages. We could, for example, sense a borrower’s repayment history (especially for missed payments) and use this both to generate secondary context (revised risk of default) and to identify situations of interest (default, impaired, at-risk). Banks do this already, of course, but there are advantages to the sensor perspective. For one, context-aware systems show us that it’s the richness of links between context that is the key to its usefulness. The more links we have, the more semantics we have over which to reason. Secondly, migrating to a context-aware platform means that additional data streams, inferences and situations can be added as-and-when required, without needing to re-architect the system. Given the ever-increasing amount of information available on-line, this is certainly something that might become useful.
Of course there are massive privacy implications here, not least in the use of machine classifiers to evaluate — and of course inevitably mis-evaluate — individuals’ circumstances. It’s important to realise that this is going on anyway and isn’t going to go away: the rational response is therefore to make sure we use the best approaches available, and that we enforce audit trails and transparency to interested parties. Credit scoring systems are notoriously opaque at present — I’ve had experience of this myself recently, since credit history doesn’t move easily across borders — so there’s a screaming need for systems that can explain and justify their decisions.
I suspect that the real value of a sensor perspective comes not from considering an individual institution but rather an entire marketplace. To use an example I’m familiar with from Ireland, one bank at one stage pumped its share price by having another bank make a large deposit — but then loaned this second bank the money to fund the deposit. Contextualised analysis might have picked this up, for example by trying to classify what instruments or assets each transaction referred to. Or perhaps not: no system is going to be fully robust against the actions of ingenious insiders. The point is not to suggest that there’s a foolproof solution, but rather to increase the amount and intelligence of surveillance in order to raise the bar. Given the costs involved in unwinding failures when detected late, it might be an investment worth making.