Simon Dobson (Posts about innovation)

Disloyalty cards

Simon Dobson — Wed, 02 Mar 2011 16:00:53 GMT

I seem to be carrying a lot of loyalty cards at the moment. Given my complete lack of loyalty to any particular brand, is there some way to lose most of the cards and still get the benefits? Most large stores have loyalty cards these days: swipe them at the till and receive some small but non-zero reward at a later date. There are all sorts of variations: Tesco UK has a very generous rewards programme; Starbucks' card is also a debit card that you can load-up with cash to pay for coffee; Costa has a more traditional "buy ten, get one free" approach; Waterstone's collects points to put towards later book purchases.. (You can already see where a lot of my disposable income goes...) What the store gets from this, of course, is market intelligence at the level of the individual and group. Tesco (to use one example) can use my loyalty card to match purchases to me, in a way that's otherwise impossible. Using this information they can tailor offers they make to me: give me cash vouchers to spend on anything, but also targeted vouchers for individual products that I don't normally buy, but which my purchasing history says I might, if offered. The supermarket can also group the purchases of people in a particular neighbourhood and use it to stock local stores in ways that appeal to the actual shoppers who frequent the place: if you doubt this, go to a downtown Tesco and then a suburban one, and compare the variation in stock. The fact that loyalty cards are so common tells you how valuable stores find this kind of market intelligence, and it's purchased from customers for a very small price. We reveal an enormous amount about our lifestyles and activities from our everyday purchases. Once you have a really huge amount of data, you can extract information from it in ways that are by no means obvious. You can also be more speculative in how you process the data, since the penalties for being wrong are so low. A traditional pervasive computing system is real-time and has a high penalty for inaccuracy; a loyalty card operates on a slower timescale and only has opportunity costs for incorrect recommendations. Thought about from this perspective, I could even see ways of testing hypotheses you formed about individuals, and using this to refine the situation recognition algorithms: if their habits suggest they have a baby, offer them a really good deal for baby products and use their future actions to confirm or refute the hypothesis. These approaches work both in the real world and online, of course. Amazon gets all your profile information without offering a loyalty card, simply by the way e-commerce works. Tesco online links e-shopping to real-world shopping, and so can compare your long-term trend purchases against your "impulse" (or just forgetfulness) buys in the local shop, and correlate this against your home address. None of this is necessarily bad, of course. I like Amazon profiling me: they introduce me to books I might not otherwise read, and that's an unmitigated good thing (for me). But it's important to realise that these things are happening, their good and bad effects. If you care about your privacy, don't carry a loyalty card. On the other hand, if you value personalised service, carry a load of them and help them classify you. It's also important to realise where the breaks in the coverage are. If Tesco knew I frequented Starbucks, for example, they might offer me a different kind of coffee to someone who doesn't. (Don't offer a coffee snob a deal on cheap instant.) There are lots of correlations between different areas of people's lives that could be exploited -- "exploited" in a nice way, at least in principle, but also profitably for the companies involved. What's needed is a way to tie together more aspects of individuals' lives, either as individuals or in bulk. One way to do this might be to get rid of per-store loyalty cards and replace them with a single profiling card: a "disloyalty" card. The disloyalty card acts as a hub for collecting details of what an individual buys where. The individual stores signing-up to the card offer points according to their normal practices, but the points can be spent at any store that participates in the disloyalty card scheme. For the consumer, this means that they collect points faster and can therefore spend them on more valuable goods and services. What's in it for the stores. More market intelligence, across an individual's entire purchasing profile. One could do this individually, to allow stores to see what customers buy in other stores and cross-sell to them. Alternatively one could anonymise the data at an individual level but allow correlations across groups: everyone who lives in Morningside, or everyone who buys coffee early in the morning on a week-day at Haymarket station. If the disloyalty card operator acted as a trusted third party, individual stores could even cross-sell to individuals without ever knowing to whom they were making the offer: that way the stores still "own" their own customers, but can access more data about them en masse, online and offline, across a range of businesses. I'm not necessarily suggesting this would be a good thing for the individual or indeed for the operation of the retail market as a whole -- it provides enormous opportunities for profiling and severely and increasingly disenfranchises people who choose to sit outside the scheme -- but it's a commercial possibility that should be considered both by commercial interests and by regulators. Like many ideas, there's precedent. Back in the 1970's we all used to collect Green Shield stamps, which were given out by a range of shops and could be redeemed collectively at special stores. (My grandfather, who was particularly methodical and assiduous in all he did, was amazingly effective at collecting his way to what he wanted in the Green Shield stores.) This scheme has the group-collecting aspect of disloyalty cards without the market intelligence aspect, which brings the whole scheme into the 21st century and gives is a measurable monetary value for all concerned.

Where do all the comments go?

Simon Dobson — Mon, 21 Feb 2011 08:00:13 GMT

I don't get many comments on this blog: fortunately I do it for the opportunity to write amid the torrent of administrivia rather than for the affirmation. But I do get more than's readily apparent, which tells us something about social media and opens up an opportunity. Blogging is up there with Twitter and Facebook as the epitome of social media: a way to share parts of your life and mind digitally with more-or-less unselected parts of the world. You could argue that it's also the epitome of egotism to think that anyone else might want to share those parts of your life and mind that you foist upon them, but that's another story. The social media space has consolidated somewhat to yield several large services with a surrounding cloud of also-rans, which is pretty much what you expect in a market that places a premium on networking and connectedness: by definition one large network is more valuable (in many senses) than several smaller ones. But it shows no signs of consolidating into a single hegemony that captures all the aspects of social media we've come to find: Twitter and Facebook are distinct but linked, and likely to remain that way. The same is true for blogging platforms. The reasons for this are complicated but relate to the ability of one company to climb several steep learning curves simultaneously. It's more efficient to focus on innovating in one area -- climbing the single curve quickly -- rather than trying to provide a unified experience that involves multiple areas of innovation. This is Adam Smith's pin factory for the knowledge economy. This economically attractive choice has consequences, though. Since there isn't a single network, one needs to access multiple networks to reach the widest community. Many people -- including me -- do this by tying their services together. My Facebook status is updated from my Twitter feed, for example: I almost never update it directly. The same is true for my LinkedIn status. My web site has the Twitter stream and the blog side by side, and if I post to the blog a notification goes out of on Twitter. Actually, if you're reading this at all, you probably know at least some of this by virtue of how you got here :-). This is the socialisation of social media, in effect, but it means that what one thinks of as a single conversation is actually going on in several places simultaneously, and that's where we need to ask where all the comments go. I get very few direct comments on the blog, despite some posts getting quite a lot of readers: the conversion rate of readers to commenters is significantly less than 1 comment per 500 distinct page views. However, that's not the whole story, because I also get comments by the other social media channels: as comments on my Facebook page, as replies and retweets of the announcements, as email from people who see these, and so on. That makes it hard to measure the impact of a post as measured by the comments it generates. The popularity of a post isn't what's important to me (although they're always nice to get, of course), but it does mean that a significant benefit of social media -- the ability to engage in a conversation over people's opinions -- is impeded by the fractured nature of the comment stream, even when it's being aided by wider distribution. So I think there's an opportunity here for someone: how can we aggregate comments? We aggregate blogs themselves using RSS feeds: can we do the same for the comments, and feed them back to a single point (the blog itself, perhaps) so that they're available for further comment regardless of the channel on which they were issued? It's not a trivial task, because you need to identify which comments relate to which blog posting, but it's certainly not insurmountable. Might make a good final-year undergraduate computer science project. Actually this sort of technology is useful more broadly. Any web site or book that attracts comment is likely to attract them in different fora: how do you find them? How do you bring them to one place? This has both intellectual and commercial implications: intellectually it increases the network of the social media comment space and so should make it more efficient; commercially it improves the means for assessing the popularity of a web site which is often what drives advertising revenues. Overall it would let the increasing specialisation of social media continue without this irretrievably fragmenting the very things that make it most valuable: the two-way flow of ideas.

Who invented meringue?

Simon Dobson — Thu, 17 Feb 2011 08:00:15 GMT

What the invention of complicated foods tells us about discovery, innovation, and university research funding. Over lunch earlier this week we got talking about how different foods get discovered -- or invented, whichever's the most appropriate model. The point of the discussion was how unlikely a lot of foods are to have actually been created in the first place. The lineage of some quite complicated foods is fairly easy to discern, of course. Bread: leave out some wet flour overnight and watch it rise to form sourdough. Do the same for malt and you get beer (actually the kind that of beer that in Flanders is called lambic). Put milk into a barrel, load it onto the back of a donkey and transport it to the next town, and you'll have naturally-churned butter. It's fairly easy to see how someone with an interest in food would refine the technique and diversify it, once they knew that the basic operation worked in some way and to some degree. But for other foods, it's exactly this initial step that's so problematic. I think the best example is meringue. Consider the steps you need to go through to discover that meringues exist. First, you have to separate an egg -- which is obvious now, but not so obvious if you don't know that there's a point to it. Then you need to beat the white for a long time, in just the right way to introduce air into it. If you get this wrong, or don't do it for long enough, or do it too enthusiastically (or not enthusiastically enough) you just get slightly whiter egg white: it's only if you do it properly that you get the phase change you need. Of course you're probably doing this with a wholly inappropriate instrument -- like a spoon -- rather than a fork or a balloon whisk (which you don't have, because nobody knows there are things that need air beating into them yet). Then you need to determine, counter-intuitively, that making the egg white heavier (with sugar) will improve the final result when cooked. Then you have to work out that cooking this liquid -- which has actually to be a process of drying, not cooking -- is actually quite a good idea despite appearances. It's hard enough to make a decent meringue now we know they exist: I find it hard to imagine how one would do it if one didn't even know they existed, and furthermore didn't know that beating egg whites in a particular way will generate the phase change from liquid to foam. (Or even know that there are things called "phase changes" at all for that matter.) Thinking a little harder, I actually can imagine how meringues got invented. In the Middle Ages a lot of very rich aristocrats competed with their peers either by knocking each other off horses at a joust or by exhibiting ever-more-complex dishes at feasts. These dishes -- called subtleties -- were intended to demonstrate the artistry of the chef and hence the wealth and taste of his patron, the aristocrat. Pies filled with birds, exact scale models of castles, working water-wheels made out of pastry, that kind of thing. In order to do this sort of thing you need both a high degree of cooking skill and a lot of unusual food-based materials to work in. You can find these as part of your normal cooking, but it's probably also worth some experimentation to find new and unusual effects that will advance this calorific arms race a little in your favour. So maybe meringue was invented by some medieval cook just doing random things with foodstuffs to see what happens. The time spent on things that don't work -- leaving pork fat outside to see if it ferments into vodka, perhaps? -- will be amortised out by the discovery of something that's really useful in making really state-of-the-art food. Contrary to popular belief the Middle Ages was a time of enormous technological advance, and it's easy to think of this happening in food too. So food evolves under the combined effects of random chance operations shaped by survival pressures. Which is exactly what happens in biology. A new combination gets tried by chance, without any anticipation of any particular result, and the combinations that happen to lead to decent outcomes get maintained. At that point the biological analogy breaks down somewhat, because the decent outcomes are then subjected to teleological refinement by intelligent beings -- cooks -- with a goal in mind. It's no longer random. But the initial undirected exploration is absolutely essential to the process of discovery. Bizarrely enough, this tells us something more general about the processes of discovery and innovation. They can't be goal-directed: or, more precisely, they can't be goal-directed until we've established that there's a nugget of promise in a particular technique, and that initial discovery will only be performed because of someone's curiosity and desire to solve a larger problem. "Blue-skies" research is the starting point, and you by definition can't know -- or ever expect to know -- what benefits it might confer. You have to kiss an awful lot of frogs to have a reasonable expectation of finding a prince, and blue-skies, curiosity-driven research is the process of identifying these proto-princes amongst the horde of equally unattractive alternatives. But someone's got to do it.

Why I don’t sign NDAs

Simon Dobson — Mon, 17 Jan 2011 09:00:27 GMT

Non-disclosure agreements (NDAs) are something I try not to sign, for various reasons. For one thing, they give a false sense of security; for another, they interfere with me doing my job. An increasing amount of academic research is supported in one way or another by companies, either directly or through co-funding agreements. This trend is only likely to increase as State funding becomes less common. As well as this, academics are sometimes approached to do consultancy or development work for companies on a more close-to-market basis. This can be great for all concerned: the companies get access to (hopefully) unbiased expert advice that can perhaps take a longer-term view, while the academics get real-world problems to work on and a reality check on some of their ideas. Despite all this, there are still some problems. Chief among them, I've found, are non-disclosure agreements (NDAs) whereby one or both sides agree not to disclose proprietary information to third parties. Some NDAs are one-way, so (for example) the university agrees not to disclose company information; many are symmetrical and protect both sides, which is obviously more desirable. (Interestingly it's often university-led NDAs that are asymmetric, implying that the companies have no intellectual input...) Although they sound like they're used for competitive reasons -- and sometimes they are -- it's more likely that they used to protect information for use in later patent applications, where discussion with a third party might be regarded as "publication" and so endanger the patent. Anyone who works in commercial research has to be sensitive to this, and since I used to run a company myself I'm conscious of protecting others' intellectual property. So why does this cause a problem? Mainly because of the bluntness of the instrument by which the protection happens. In my job, I have a lot of discussions that involve half-formed ideas. Many of these seem pretty much to condense out of the ether into several people's heads simultaneously: it's in no way uncommon to have two or three people discuss the same "novel" idea within days of each other. I suppose it's just that there's nothing really new under the sun, and people who share a common technical milieu will often see the same problems and arrive at similar solutions. Often the people involved are students: either undergraduates looking for projects, or PhD students with ideas for theses or papers. These people are my "core constituency" in the sense that my main job is to teach and supervise them in a research-led way. You can probably see where this is going. Suppose a student comes to me with an idea, and that this idea is related in some way to an idea presented by a company with whom I've signed an NDA. What do I do? Refuse to discuss the matter with the student, even though they're fired-up about it? Try to re-focus them onto another area, even though it might be a great idea, because I can't discuss it? Send them to someone else, even though I might be the right person to supervise the work? What I can't do is get involved, because however hard I try, I'll never be able to prove that information covered by the NDA had no effect on what I said or did -- or didn't say or do, for that matter. That leaves both me and the university open to legal action, especially if by some chance the student's work got a high profile and damaged the company, for example by developing an open-source solution to something they were working on. This is something of a dilemma. I like working with companies; I love working with students; and I don't like the feeling that my freedom to discuss technology and ideas is being legally constrained. I therefore minimise my exposure to NDAs and confidentiality agreements. It's sometimes unavoidable, for example as part of EU project consortium agreements. But as a general rule I don't think NDAs sit well with academics, and there's too much danger of damaging the general openness of research within a university: too much of a sacrifice just to get a single funded project. I'll happily agree to keep information confidential, but the risks of signing a blunt and broad agreement to that effect are just too great.