Mapping Strategy

Butterflies for Dummies

Check out this succinct review in last Sunday's Boston Globe of an important but oft-misunderstood concept critical to any forecasting or planning endeavor [emphasis and link added]:

The butterfly effect is a deceptively simple insight extracted from a complex modern field. As a low-profile assistant professor in MIT's department of meteorology in 1961, [the late Edward] Lorenz created an early computer program to simulate weather. One day he changed one of a dozen numbers representing atmospheric conditions, from .506127 to .506. That tiny alteration utterly transformed his long-term forecast, a point Lorenz amplified in his 1972 paper, "Predictability: Does the Flap of a Butterfly's Wings in Brazil Set Off a Tornado in Texas?"

In the paper, Lorenz claimed the large effects of tiny atmospheric events pose both a practical problem, by limiting long-term weather forecasts, and a philosophical one, by preventing us from isolating specific causes of later conditions... It is extremely hard to calculate such things with certainty... Realistically, we can't know. "It's impossible for humans to measure everything infinitely accurately," says Robert Devaney, a mathematics professor at Boston University. "And if you're off at all, the behavior of the solution could be completely off." When small imprecisions matter greatly, the world is radically unpredictable.

Moreover, Lorenz also discovered stricter limits on our knowledge, proving that even models of physical systems with a few precisely known variables, like a heated gas swirling in a box, can produce endlessly unpredictable and non-repeating effects.

"Lorenz went beyond the butterfly," says Kerry Emanuel, a professor in the department of earth, atmospheric, and planetary sciences at MIT. "To say that certain systems are not predictable, no matter how precise you make the initial conditions, is a profound statement." Instead of a vision of science in which any prediction is possible, as long as we have enough information, Lorenz's work suggested that our ability to analyze and predict the workings of the world is inherently limited.

What few articles on this subject touch upon, but I find endlessly fascinating, is how this undeniable, well-grounded scientific insight interacts with human nature -- specifically our ingrained need/desire to feel knowledgeable and in control. (I use the term 'we' in both the individual and corporate sense here.)

It has been my experience that few individuals and very few (if any) organizations ever err on the side of humility in this regard. Perceptual failures are seldom in the direction of assuming less than is actually predictable. More often, the response is something like this:

yes I know about the butterfly effect, chaos theory and all that... but it's kinda academic... and you don't know my boss... you see, I/he/she/they/we need to predict XYZ anyway...
because my/our future depends on it... so just give us a number... give us your best guess about the probability... please?

People fall back on a prediction/probability paradigm either because it's what they know (or have the planning tool-set to address) or because, even knowing that approach is flawed, they find it more comfortable... and comforting (or politically expedient in a particular organizational culture) to pretend that what is fundamentally uncertain is perhaps predictable after all. The chance of rain is 62.5%...

Predicting Armageddon

Sadly, In light of this, this seems undervalued [emphasis added].

"If Iran continues its nuclear weapons programme, we will attack it," said [Deputy Prime Minister] Shaul Mofaz... "Other options are disappearing. The sanctions are not effective. There will be no alternative but to attack Iran in order to stop the Iranian nuclear programme."

One thing notable about this graphic is how little reaction there has been to this news. To my reading (and I've followed this closely the last several years) it marks a significant step up in rhetoric by the top echelons of the Israeli government. The second thing I find remarkable is how the market has trended steadily upwards over the past four months or so.

This is but one of many 'quarterly' prediction markets Intrade has been running on an airstrike on Iran. All of them, obviously (at least so far) have rewarded the skeptics, however most have trended downward as the market close approached. This one isn't doing that.

I'm of two minds on how much stock to put in this market. (Sorry, pun intended).

On the one hand, military missions are, or at least ought to be, closely guarded secrets. And this would be no ordinary military mission. It is truly existential. Sure a few thousand people might be privvy to some kind of knowledge or direct insight into what was going on, but I doubt any of them would risk being shot in order to make a little extra on the side. So, on that basis, this has many of the characteristics of silly/failed markets in things like papal elections and supreme court nominations.

On the other hand, this is not as surprising as a terrorist attack (a dumb thing in which to make highly specific markets IMHO, even though there are many related indicators that might be useful).

Most nations don't want to go to war. (Iran may be an exception, but we're talking here about an attack on them, not by them). Most nations 'do the (diplomatic) dance'. They do relatively rote, predictable things in the lead-up to any conflict. And those things tend to follow patterns that are not random.

They may not be easily predictable, but there is information out there, held by people not constrained by an oath of secrecy, that is meaningful in discerning whether this will go down or not. Sadly, if I were to bet on it (and I don't -- at least not on real money markets) I would have to say that this market is way undervalued.

UPDATE (July 1, 2008): Almost a month after I first made this post, the trend continues (see below). Despite Israeli moves that make such a strike seem more likely (whether for real or for bluff we can't know), the market hasn't had a very high 'beta' (volatility). This despite (or perhaps because of) significantly higher trading volumes in June. A good summary of recent events around this can be found here.

Overt Airstrike on Iran on or before Dec. 31, 2008

Obstacles to Prediction Market Adoption

Harvard Business School Associate Professor Andrew McAfee asks, in a recent blog post, why prediction markets haven't been taken up more readily in the corporate world. He writes [links added]:

After writing the [Google prediction markets] case, teaching it a few times, and spending some time understanding the mechanics and utility of prediction markets, I share the puzzlement articulated by James Surowiecki in his book The Wisdom of Crowds:

". . . the most mystifying thing about [prediction] markets is how little interest corporate America has shown in them... companies have remained, for the most part, indifferent to this source of potentially excellent information, and have been surprisingly unwilling to improve their decision making by tapping into the collective wisdom of their employees."

Why is this? It’s not because the technology is hard to acquire... what is the real stumbling block? Is it that companies don’t really want the most accurate information about future events to come out and be widely known?

Sean Silverstone Silverthorne, the editor of HBS Working Knowledge expands on the question in a post last week, outlining what he sees as "three initial roadblocks":

• Unfamiliar with the Concept -- Most managers, especially the veterans, grew up hearing not to trust crowds, to avoid mob rule. Instead, they were encouraged to value expertise and be ready to pay consultants. It’s a major brain-bender to suddenly believe crowds can be smarter than experts.
• Internal Friction -- Corporate prediction markets rankle folks like product managers when company employees start betting on a bad outcome for their product.
  
  
• Implementation Issues -- Until recently, there wasn’t a whole lot of best practices around how to run efficient prediction markets. Results can be seriously skewed by having too few participants, asking the wrong type of questions, or by running a market too long or too short a time.

From my experience working directly with prediction market vendors and corporate implementers, and having watched, thought about and pontificated on this phenomenon for over a dozen* years, Silverstone's points seem reasonable as high-level headings. Several caveats and elaborations may be useful on this rapidly-moving target, however. *(I first heard about them in a pitch by Ken Kittlitz to the Digital Commerce Society of Boston circa 1995/96 -- shortly after I nominally helped co-found that late and loosely constructed organization, and shortly after Kittlitz founded the Foresight Exchange)

First, the number of managers completely unfamiliar with the concept of prediction markets has dwindled markedly over the past five years -- and at an accelerating rate. Silverstone is, I'm sorry to say, somewhat behind in writing that:

A relatively small number of companies including Google, HP, Intel, Yahoo, and Eli Lilly have jumped on this idea by creating prediction markets to aggregate the opinions of employees...

That was true five years ago. It is not the case today.

Doing some research into this recently, I was pleasantly startled to discover that there are at least fifty large corporations running prediction markets 'for real' now, and countless others running small, informal and for the most part un-trackable pilots. (Virtually all of the major vendors offer such trials, though with varying degrees of publicity and enthusiasm as well as varying degrees of ease in setting them up -- though none are terribly difficult). And that's not counting the number of academic institutions and news media organizations that run prediction markets for other reasons (e.g., experimental learning and securing reader attention, respectively).

Add to that the fact that HBS and HBR are covering this topic, conferring a halo of corporate legitimacy, that Surowiecki's book continues to thrust the key concepts into business management if not lay-public consciousness, that barely a week goes by without a major publication giving some ink to the subject (I've stopped even blogging them individually, there are so many) and that the proportion of people in the management class who trade various financial instruments is very high. I.e., they recognize, at some level, that prices represent crowd wisdom -- a body of information to which they may or may not be privy, but which they ignore at the peril of their 401K.

So, given that backdrop, I would turn Silverstone's point around and ask: What excuse can a manager offer, in May, 2008, for being totally 'unfamiliar with the concept' of prediction markets?

Wait. Don't answer that. If your boss isn't reading this, your CEO or your board very well may be, and that's not an idle quip. Some of the most important prediction market initiatives I've seen have come down from the very top, rather than from middle management...

...which leads me to Silverstone's second point: "Internal Friction". That would be putting it mildly.

Some cultures palpably chafe at the idea of unleashing democratic forces within their walls. This is not a scintillatingly deep or novel insight. We know this to be true for countries, and families, as well as for bowling leagues and PTOs for that matter. (I've also observed this when running scenario workshops within certain corporate and, treading lightly here, societal cultures).

Humans almost universally seek some kind of power or at least stability. They work hard for it. And they're loathe to give it up to some impersonal, anonymous and (it can sometimes appear) fickle and BS-penetrating tool (however powerful) that doesn't even have to justify its reasons. (Imagine an election in which each voter had to justify his or her reasons -- to someone -- before casting a vote and you'll understand what I mean.)

Which is all to say that the issue goes well beyond friction to questions of culture, organizational structure, leadership, and the role or perceived role of middle managers. Classically the role of 'middle management' (even the term has taken on a derogatory ring) has been to act as filters and conveyors of information moving between senior management and workers.

Prediction markets usurp some of that role and that can be, quite understandably, unsettling to them. Nobody ever said business was fair. But, bottom line: it is a real reason why prediction markets need to be contextualized carefully in order to succeed. Repeat after me: it's not about the software (mostly).

Which brings me to Silverstone's last point: Implementation Issues. Let's be clear about one thing: 'implementation' here does not refer to the kind of thing one thinks about when installing a big new software system. The software for these things is pretty darned simple in that regard.

The much bigger questions have to do with organizational issues. And those can be sweeping.

For example:

What questions is an organization, a) truly interested in knowing the answer to (strategic), b) willing to tell the employees they might not know the answer to (potentially humiliating) and, c) willing to feed back information on how and why we did or did not act on that input (empowering of morale and loyalty... or cynicism... depending on how it is handled). Just the first part of that is a big, hairy strategy question that goes right to the heart of what a company wants to be when it grows up and where it thinks it is currently 'stuck'.

Bottom line: prediction markets harbor all the paradigm-changing, employee-empowering breakthrough-idea-generating, process-improving, cost-saving potential of the suggestion box... as well as all of its potential as a reason to deride management.

Here are the twists that can and do cause managers to think carefully about the transparency they're prepared to unleash: prediction markets are real-time, they're at least nominally public, and the information they generate isn't really a suggestion (casual) so much as it is a serious investment of time, thought and expectation on the part of those asked to participate. One has to ask then: With all of that potential for discomfort, why have so many big companies begun using them in earnest?

From Risk to Uncertainty

I don't agree with everything in this piece by Thomas Homer-Dixon that appeared last week in the Toronto Globe and Mail, but this quote is an absolute gem (emphasis added):

Our global financial system has become so staggeringly complex and opaque that we’ve moved from a world of risk to a world of uncertainty. In a world of risk, we can judge dangers and opportunities by using the best evidence at hand to estimate the probability of a particular outcome. But in a world of uncertainty, we can’t estimate probabilities, because we don’t have any clear basis for making such a judgment. In fact, we might not even know what the possible outcomes are. Surprises keep coming out of the blue, because we’re fundamentally ignorant of our own ignorance. We’re surrounded by unknown unknowns.

It's something I've said for a long time:

It's tempting to think that all things are predictable given enough information, enough minds, enough time and enough computing power. It's just not true. (Which is not to say that some things are not predictable... and with incredible precision... a phenomenon that leads to overestimating the scope of problems and questions that lend themselves to such methods.)

Telling which is which is the trick...

I would go even further to say that really smart people who, by life experience know that some things are fundamentally unpredictable still draw an unvoiced sense of emotional comfort in their business life from the idea that some wise expert somewhere has been to the future (for all intents and purposes) and if we could just find him or her things would be OK... and/or that a really sophisticated computer model or prediction market (the 'collective mind') can provide crystal ball-level insights.

Sometimes yes. Often, no.

I liken Mr. Homer-Dixon's observations to those tragically massive car pile-ups that happen a few times of year in fog-prone areas like the Central Valley of California. Everyone is driving along at a reasonable speed, with reasonable spacing between vehicles. People are sipping coffee, tuning radios, maybe talking on cell phones. Slightly distracted, but mostly responsible. All is normal.

Then the first guy hits a fog bank and can't see squat. He taps his brakes. The second guy sees red lights and fog coming up fast and taps his brakes just a little bit harder, and so on. In just a few seconds, hundreds of cars end up in a tangled heap and people die. All because the guy in front was convinced by every one of his senses and not without justification based on experience that the visibility on the next 100 yards of road would be the same as on the last 100 yards of road.

Popping up Like Mushrooms

Not clear that there's anything new here conceptually but the presence of another news-oriented prediction market during a U.S. election year is further evidence of the power of a good idea whose time has come.

Hubdub - a new Web site [Nigel] Eccles and three colleagues in Edinburgh, Scotland, assembled - where customers will bet for fun, not money, on the outcomes of real news stories. The site launches Monday [1/28]...

After signing up, you'll receive 1,000 "Hubdub dollars," play money that works only on the site. You can look at stories about, say, whether Gregg Williams will be named the next head coach of the Washington Redskins or who will win the Florida Republican primary.

Guess right, and you'll win more Hubdub dollars. Lose, and your account will draw down. In the spirit of the board game Monopoly, where simply sticking it out is rewarded, you'll also get 20 new Hubdub dollars ever day you log in.

Hmm... can you say millisecond-scale auto-logins leading to hyper-inflation?   :)

"What I realized was it wasn't the fact that I had lost money, but my pride and feeling of self-worth had taken a real knock," Eccles said. "I think that's the case for a lot of people - it's not so much the cash, it's being right."

Or in his case, being wrong. Methinks someone needs to re-write his press-releases. H/T: Bob Weber.

One of the Benefits of Working in "Cube Land"

Your choice of seating at work may affect much more than your sensory pleasure at knowing when someone is eating tuna fish or onions versus citrus fruit or chocolate. Google has discovered that it's the most important factor in your being in the loop: in particular, whether you win or lose based on your close colleagues' prediction market trading.

(See 'heat map' at right: successful traders are depicted in green; losing traders in red).

(H/T: Jeffrey Henning)

'Real' traders take note! (Pick your desk-mates carefully.)

Random thought: this adds a whole new dimension to the elaborate strategies seasoned business travelers use to get primo seating on airplanes. Exit row? No thanks. Just give me the seat next to Warren Buffett.

The Future Will Be... Different From What We Expect

Great insights from science writer Ed Regis [emphasis added]:

...after watching all those forecasts not come true, and in fact become falsified in a crashing, breathtaking manner, I began to question the entire business of making predictions...

...the source of this incredible mismatch between confident forecast and actual result [is that] the universe is a complex system in which countless causal chains are acting and interacting independently and simultaneously... all of them running in parallel, and each of them often affecting the course of the others...

It is therefore

...hopeless to try to specify in advance what's going to happen as they jointly work themselves out.  In the face of that complexity, it becomes difficult if not impossible to know with any assurance the future state of [a] system...

(other than tides, moon phases, etc.)

Further, it's an illusion to think that supercomputer modeling is up to the task of truly reliable crystal-ball gazing.  It isn't.  Witness the epidemiologists who predicted that last year's influenza season would be severe (in fact it was mild); the professional hurricane-forecasters whose models told them that the last two hurricane seasons would be monsters (whereas instead they were wimps).  Certain systems in nature, it seems, are computationally irreducible phenomena, meaning that there is no way of knowing the outcome short of waiting for it to happen.

After accepting this reality (something that's as or more true for systems in which human beings play a role, e.g., social, economic, political, business, etc.) the next best things to knowing and predicting are imagining and recognizing:

• imagining multiple, archetypal ways in which the future might evolve, and
  
• collectively being able to recognize (if only a little bit faster than the other guy with whom you are competing) the key events and sequences of events that signal (if only weakly) that one or more paradigms (and not others) are starting to unfold.

Corporate Prediction Markets Pass Major Milestone

The airtime that Newfutures' North American VP, Norris Clark got on CNBC with Maria Bartiromo last week is a milestone unto itself in the steady rise of prediction markets into mainstream consciousness and respectability. What he had to say put an exclamation point on that milestone [emphasis added]:

"...we've been setting up over 100,000 prediction markets for our corporate clients."

It's unclear how that statement squares with the undated claim, on Newfutures' website, that they have

"...created and operated more than 100,000 markets on a wide variety of topics both public and corporate."

But by any measure, it's a big number, begging the question: Have we reached, or are we already beyond a tipping point in the corporate use of these tools for strategic competitive advantage?

Bet2give: Can Markets Be Compassionate?

Just launched this week is bet2give, a prediction market offshoot of Newsfutures that enables "those who are best at predicting the future... to decide how to improve the future, one donation at a time."

I didn't want to spoil their cool and clever Christmas-colored typeface with a link, so here it is).

First take on the high concept: a great 'mash-up'. Sorta like Kiva plus Wall Street or Libertarians with a soft streak. Maybe Ayn Rand working a soup kitchen. Well OK, maybe not. But you get the idea...

Their tag-line: "What if you could win more than money?" I like it.

'Funny-money' (i.e., internal-currency, points-based) prediction markets--meaning nearly all U.S. and most corporate ones--have always operated on the theory that prestige, ego and accomplishment are powerful independent motivators to attract participants and have them take the market seriously--in other words to "play hard". They've suffered however, from the fact that those are their only drivers.

Get tired of the game's world and its limited prestige potential and it's easy to lose interest and move on. That's how it worked for me anyway. After obsessively managing a portfolio of propositions on Newsfutures several years ago I got bored and found better uses of my time. With no link to anything real beyond their world it was interesting but not compelling.

Real-money markets by contrast (while not without their own powerful motivations obviously) come with certain baggage. That includes conflicts of interest, negative public perceptions about and comparisons with seedier forms of gambling, and on a very practical level, U.S. restrictions on real-money markets that aren't licensed by the SEC or by state gaming commissions. That's a set of regulations that's kept U.S.-based real-money markets in the horse-and-buggy era while UK-based Tradesports has rocketed into the stratosphere.

Which is all a round-about way of saying that bet2give is definitely one to watch. I'm not sure how powerful it will be to have my local homeless shelter competing with your local soup kitchen, but there's potential for rough and focused play when say for example, my quasi-political charity competes with yours that's fighting against its objectives (use your imagination on that one).

Even when that kind of stuff is not at issue, bet2give will make me feel a whole lot better about playing with a 'pot' of money that I would never even think to devote to something most would compare with betting on horses.

What Can and Cannot Be Predicted (and Thoughts On Telling the Difference)

I just ran across this post ("Debating the Viability of Terrorism-Prediction Markets") over at the 'Footnoted' blog at the Chronicle of Higher Education discussing the risks of terrorism how best to assess them and whether they can be known at all. (If you've grown weary hearing about war, politics and terrorism, just skip to my non-terrorist business conclusions in the last paragraph.

In a recent interview [PDF] with the Federal Reserve Bank of Richmond, W. Kip Viscusi is asked about the public-policy response to the threat of terrorism and whether we are weighing the costs and benefits in a generally rational way. "The reason this is tricky is we don’t have very good numbers on what these risks are," Viscusi says. "The estimates of the probability of a terrorist attack or the number of people who are going to die in the coming year are all over the map. So if you can’t assess the likelihood of a terrorist attack or how deadly it is going to be, it is really hard to say how much you should spend to try to prevent it." [emphasis added]

Vscusi is characterized later in the article (and not without reason) as "one of the foremost academic experts on risk". His comment is in response to a persistent school of thought that claims, as Bruce Schneier does, that from an actuarial perspective, "terrorism doesn't happen".

Without the qualifier, the idea sounds terribly cold. With all reverence for the families of those who have lost loved ones in terrorist incidents however, it's not. Businesses and governments--not to mention non-profit institutions, individuals and families--all need to assess risks as rationally as possible and take measures to hedge them. Like it or not, the value of a human life can be defined--at least partially--in dollar terms. Anyone who's ever taken a course in economics or statistics (or balanced a checkbook for that matter) has figured out that infinite spending to protect against risk is infinitely foolish and that infinite spending on risk 'A' (and no spending at all on risks 'B', 'C', and 'D') is only a variant on the same wrong-headed assumption.

That's not what's really at issue. What is at issue is the degree to which the political leanings of some lead them to believe that we can know how much to spend combating terrorism and that the 'right' number is obviously much less than we are spending today (e.g., tactical domestic measures, overhead for business, strategic overseas measures, etc.) To which my response is: really? Show me your pre-9-11 white paper predicting the order-of-magnitude sea-change that occurred in that 'industry' on 9-11.

Bryan Caplan is one of Viscussi's critics. The CHE post notes:

"I am frankly puzzled," Caplan writes at EconLog. Citing the work of John Mueller,we have a long experience with terrorism, which has "shown it to be an extremely small problem in the broad scheme of things. How much longer does Viscusi want to wait before he'll conclude that the risk is very low?"

Unfortunately, long experience in and of itself is not sufficient for prediction, even at a macro level. Hold that thought for a quick diversion.

Here's where it gets weird. In criticizing Viscussi, Caplan, an econ prof at famously free-market GMU, ends up in league with Schneier, who as far as I can tell, tends towards the opposite end of the political spectrum. Both conclude, for entirely different reasons, that the future threat of terrorism can be known and condensed to a dollar figure and that rational budgets (both public and private) can be set accordingly. Oh that it were so.

Caplan, in particular "favors the establishment of a prediction market to help assess the likelihood of a terrorist attack". That's something I can conditionally applaud. If the results are used to "help assess", then we're fine. The possibility that a prediction market might help draw in and roll up marginal, highly distributed, even intuitive information that can supplement traditional (and sadly inadequate) intelligence-gathering mechanisms is certainly a good thing.  I've been a huge fan of prediction markets for years. They should be used for more things than they are today. To my delight, more and more are catching on.

But...

...as longtime readers know, I've also concluded that there are some (and arguably many) problems for which prediction markets are not only silly but grossly misleading. I don't have the space to review them all of them here. Thinking that they can precisely predict and quantify particular terrorist threats or even the threat of terrorism generally is a notion that falls into that category. Almost by definition, a successful terrorist venture is compartmentalized, secret and surprising.

In short, Caplan, Schneier and others appear to have an ideologically-induced blind spot that leads them to declare certainty where it does not exist. Let me explain.

Schneier and Caplan draw their essential argument from a backward-looking, actuary-style catalog of terrorist incidents. This many people died. This much property was lost. Productivity was reduced by this much for this long. Etc. Etc. It all sounds very rational. If we had reason to believe that terrorism were a natural, forecastable, perhaps even cyclical phenomenon, that approach would be absolutely correct. We don't.

The main problem, as I've noted before, is that:

The  unpredictability of terrorism renders any backwards-looking, purely quantitative, actuarial mode of analysis inappropriate and ineffective. That is, future deaths due to terrorism are something that neither Schneier nor anyone else can possibly predict with any degree of confidence.

Until 2001, the biggest single terrorist incident had caused around 300 deaths. Then in the space of a few hours, that number went up by an order of magnitude. There was no consensus (or even a significant plurality) of expert opinion predicting that that would happen - much less when, where and how. [emphasis in original]

And that's the problem.

Sudden, step-function, order-of-magnitude change, precipitated by a small group that has every reason to keep its plans secret is inherently unpredictable. It can only be imagined. If that sounds familiar outside of the terrorist context, it should. Businesses face this kind of challenge all the time; it is the very nature of business, in fact: snowboards vs. skis; digital photography vs. wet-process; PCs vs. mainframes; VoIP vs. legacy telephony; biotech vs. big pharma. The list is very long. It's harder to name an industry that hasn't been touched by this kind of change at some point (often precipitating the re-invention and re-definition of the former "industry") than it is to come up with a long list of examples of industries that have be altered in this way. Bottom line: it's important to differentiate between problems that lend themselves to forecasting and those that can only be dealt with via imaginative scenarios.