Mapping Strategy

Obstacles to Prediction Market Adoption

Harvard Business School Associate Professor Andrew McAfee asks, in a recent blog post, why prediction markets haven't been taken up more readily in the corporate world. He writes [links added]:

After writing the [Google prediction markets] case, teaching it a few times, and spending some time understanding the mechanics and utility of prediction markets, I share the puzzlement articulated by James Surowiecki in his book The Wisdom of Crowds:

". . . the most mystifying thing about [prediction] markets is how little interest corporate America has shown in them... companies have remained, for the most part, indifferent to this source of potentially excellent information, and have been surprisingly unwilling to improve their decision making by tapping into the collective wisdom of their employees."

Why is this? It’s not because the technology is hard to acquire... what is the real stumbling block? Is it that companies don’t really want the most accurate information about future events to come out and be widely known?

Sean Silverstone, the editor of HBS Working Knowledge expands on the question in a post last week, outlining what he sees as "three initial roadblocks":

• Unfamiliar with the Concept -- Most managers, especially the veterans, grew up hearing not to trust crowds, to avoid mob rule. Instead, they were encouraged to value expertise and be ready to pay consultants. It’s a major brain-bender to suddenly believe crowds can be smarter than experts.
• Internal Friction -- Corporate prediction markets rankle folks like product managers when company employees start betting on a bad outcome for their product.
  
  
• Implementation Issues -- Until recently, there wasn’t a whole lot of best practices around how to run efficient prediction markets. Results can be seriously skewed by having too few participants, asking the wrong type of questions, or by running a market too long or too short a time.

From my experience working directly with prediction market vendors and corporate implementers, and having watched, thought about and pontificated on this phenomenon for over a dozen* years, Silverstone's points seem reasonable as high-level headings. Several caveats and elaborations may be useful on this rapidly-moving target, however. *(I first heard about them in a pitch by Ken Kittlitz to the Digital Commerce Society of Boston circa 1995/96 -- shortly after I nominally helped co-found that late and loosely constructed organization, and shortly after Kittlitz founded the Foresight Exchange)

First, the number of managers completely unfamiliar with the concept of prediction markets has dwindled markedly over the past five years -- and at an accelerating rate. Silverstone is, I'm sorry to say, somewhat behind in writing that:

A relatively small number of companies including Google, HP, Intel, Yahoo, and Eli Lilly have jumped on this idea by creating prediction markets to aggregate the opinions of employees...

That was true five years ago. It is not the case today.

Doing some research into this recently, I was pleasantly startled to discover that there are at least fifty large corporations running prediction markets 'for real' now, and countless others running small, informal and for the most part un-trackable pilots. (Virtually all of the major vendors offer such trials, though with varying degrees of publicity and enthusiasm as well as varying degrees of ease in setting them up -- though none are terribly difficult). And that's not counting the number of academic institutions and news media organizations that run prediction markets for other reasons (e.g., experimental learning and securing reader attention, respectively).

Add to that the fact that HBS and HBR are covering this topic, conferring a halo of corporate legitimacy, that Surowiecki's book continues to thrust the key concepts into business management if not lay-public consciousness, that barely a week goes by without a major publication giving some ink to the subject (I've stopped even blogging them individually, there are so many) and that the proportion of people in the management class who trade various financial instruments is very high. I.e., they recognize, at some level, that prices represent crowd wisdom -- a body of information to which they may or may not be privy, but which they ignore at the peril of their 401K.

So, given that backdrop, I would turn Silverstone's point around and ask: What excuse can a manager offer, in May, 2008, for being totally 'unfamiliar with the concept' of prediction markets?

Wait. Don't answer that. If your boss isn't reading this, your CEO or your board very well may be, and that's not an idle quip. Some of the most important prediction market initiatives I've seen have come down from the very top, rather than from middle management...

...which leads me to Silverstone's second point: "Internal Friction". That would be putting it mildly.

Some cultures palpably chafe at the idea of unleashing democratic forces within their walls. This is not a scintillatingly deep or novel insight. We know this to be true for countries, and families, as well as for bowling leagues and PTOs for that matter. (I've also observed this when running scenario workshops within certain corporate and, treading lightly here, societal cultures).

Humans almost universally seek some kind of power or at least stability. They work hard for it. And they're loathe to give it up to some impersonal, anonymous and (it can sometimes appear) fickle and BS-penetrating tool (however powerful) that doesn't even have to justify its reasons. (Imagine an election in which each voter had to justify his or her reasons -- to someone -- before casting a vote and you'll understand what I mean.)

Which is all to say that the issue goes well beyond friction to questions of culture, organizational structure, leadership, and the role or perceived role of middle managers. Classically the role of 'middle management' (even the term has taken on a derogatory ring) has been to act as filters and conveyors of information moving between senior management and workers.

Prediction markets usurp some of that role and that can be, quite understandably, unsettling to them. Nobody ever said business was fair. But, bottom line: it is a real reason why prediction markets need to be contextualized carefully in order to succeed. Repeat after me: it's not about the software (mostly).

Which brings me to Silverstone's last point: Implementation Issues. Let's be clear about one thing: 'implementation' here does not refer to the kind of thing one thinks about when installing a big new software system. The software for these things is pretty darned simple in that regard.

The much bigger questions have to do with organizational issues. And those can be sweeping.

For example:

What questions is an organization, a) truly interested in knowing the answer to (strategic), b) willing to tell the employees they might not know the answer to (potentially humiliating) and, c) willing to feed back information on how and why we did or did not act on that input (empowering of morale and loyalty... or cynicism... depending on how it is handled). Just the first part of that is a big, hairy strategy question that goes right to the heart of what a company wants to be when it grows up and where it thinks it is currently 'stuck'.

Bottom line: prediction markets harbor all the paradigm-changing, employee-empowering breakthrough-idea-generating, process-improving, cost-saving potential of the suggestion box... as well as all of its potential as a reason to deride management.

Here are the twists that can and do cause managers to think carefully about the transparency they're prepared to unleash: prediction markets are real-time, they're at least nominally public, and the information they generate isn't really a suggestion (casual) so much as it is a serious investment of time, thought and expectation on the part of those asked to participate. One has to ask then: With all of that potential for discomfort, why have so many big companies begun using them in earnest?

From Risk to Uncertainty

I don't agree with everything in this piece by Thomas Homer-Dixon that appeared last week in the Toronto Globe and Mail, but this quote is an absolute gem (emphasis added):

Our global financial system has become so staggeringly complex and opaque that we’ve moved from a world of risk to a world of uncertainty. In a world of risk, we can judge dangers and opportunities by using the best evidence at hand to estimate the probability of a particular outcome. But in a world of uncertainty, we can’t estimate probabilities, because we don’t have any clear basis for making such a judgment. In fact, we might not even know what the possible outcomes are. Surprises keep coming out of the blue, because we’re fundamentally ignorant of our own ignorance. We’re surrounded by unknown unknowns.

It's something I've said for a long time:

It's tempting to think that all things are predictable given enough information, enough minds, enough time and enough computing power. It's just not true. (Which is not to say that some things are not predictable... and with incredible precision... a phenomenon that leads to overestimating the scope of problems and questions that lend themselves to such methods.)

Telling which is which is the trick...

I would go even further to say that really smart people who, by life experience know that some things are fundamentally unpredictable still draw an unvoiced sense of emotional comfort in their business life from the idea that some wise expert somewhere has been to the future (for all intents and purposes) and if we could just find him or her things would be OK... and/or that a really sophisticated computer model or prediction market (the 'collective mind') can provide crystal ball-level insights.

Sometimes yes. Often, no.

I liken Mr. Homer-Dixon's observations to those tragically massive car pile-ups that happen a few times of year in fog-prone areas like the Central Valley of California. Everyone is driving along at a reasonable speed, with reasonable spacing between vehicles. People are sipping coffee, tuning radios, maybe talking on cell phones. Slightly distracted, but mostly responsible. All is normal.

Then the first guy hits a fog bank and can't see squat. He taps his brakes. The second guy sees red lights and fog coming up fast and taps his brakes just a little bit harder, and so on. In just a few seconds, hundreds of cars end up in a tangled heap and people die. All because the guy in front was convinced by every one of his senses and not without justification based on experience that the visibility on the next 100 yards of road would be the same as on the last 100 yards of road.

Popping up Like Mushrooms

Not clear that there's anything new here conceptually but the presence of another news-oriented prediction market during a U.S. election year is further evidence of the power of a good idea whose time has come.

Hubdub - a new Web site [Nigel] Eccles and three colleagues in Edinburgh, Scotland, assembled - where customers will bet for fun, not money, on the outcomes of real news stories. The site launches Monday [1/28]...

After signing up, you'll receive 1,000 "Hubdub dollars," play money that works only on the site. You can look at stories about, say, whether Gregg Williams will be named the next head coach of the Washington Redskins or who will win the Florida Republican primary.

Guess right, and you'll win more Hubdub dollars. Lose, and your account will draw down. In the spirit of the board game Monopoly, where simply sticking it out is rewarded, you'll also get 20 new Hubdub dollars ever day you log in.

Hmm... can you say millisecond-scale auto-logins leading to hyper-inflation?   :)

"What I realized was it wasn't the fact that I had lost money, but my pride and feeling of self-worth had taken a real knock," Eccles said. "I think that's the case for a lot of people - it's not so much the cash, it's being right."

Or in his case, being wrong. Methinks someone needs to re-write his press-releases. H/T: Bob Weber.

One of the Benefits of Working in "Cube Land"

Your choice of seating at work may affect much more than your sensory pleasure at knowing when someone is eating tuna fish or onions versus citrus fruit or chocolate. Google has discovered that it's the most important factor in your being in the loop: in particular, whether you win or lose based on your close colleagues' prediction market trading.

(See 'heat map' at right: successful traders are depicted in green; losing traders in red).

(H/T: Jeffrey Henning)

'Real' traders take note! (Pick your desk-mates carefully.)

Random thought: this adds a whole new dimension to the elaborate strategies seasoned business travelers use to get primo seating on airplanes. Exit row? No thanks. Just give me the seat next to Warren Buffett.

The Future Will Be... Different From What We Expect

Great insights from science writer Ed Regis [emphasis added]:

...after watching all those forecasts not come true, and in fact become falsified in a crashing, breathtaking manner, I began to question the entire business of making predictions...

...the source of this incredible mismatch between confident forecast and actual result [is that] the universe is a complex system in which countless causal chains are acting and interacting independently and simultaneously... all of them running in parallel, and each of them often affecting the course of the others...

It is therefore

...hopeless to try to specify in advance what's going to happen as they jointly work themselves out.  In the face of that complexity, it becomes difficult if not impossible to know with any assurance the future state of [a] system...

(other than tides, moon phases, etc.)

Further, it's an illusion to think that supercomputer modeling is up to the task of truly reliable crystal-ball gazing.  It isn't.  Witness the epidemiologists who predicted that last year's influenza season would be severe (in fact it was mild); the professional hurricane-forecasters whose models told them that the last two hurricane seasons would be monsters (whereas instead they were wimps).  Certain systems in nature, it seems, are computationally irreducible phenomena, meaning that there is no way of knowing the outcome short of waiting for it to happen.

After accepting this reality (something that's as or more true for systems in which human beings play a role, e.g., social, economic, political, business, etc.) the next best things to knowing and predicting are imagining and recognizing:

• imagining multiple, archetypal ways in which the future might evolve, and
  
• collectively being able to recognize (if only a little bit faster than the other guy with whom you are competing) the key events and sequences of events that signal (if only weakly) that one or more paradigms (and not others) are starting to unfold.

Corporate Prediction Markets Pass Major Milestone

The airtime that Newfutures' North American VP, Norris Clark got on CNBC with Maria Bartiromo last week is a milestone unto itself in the steady rise of prediction markets into mainstream consciousness and respectability. What he had to say put an exclamation point on that milestone [emphasis added]:

"...we've been setting up over 100,000 prediction markets for our corporate clients."

It's unclear how that statement squares with the undated claim, on Newfutures' website, that they have

"...created and operated more than 100,000 markets on a wide variety of topics both public and corporate."

But by any measure, it's a big number, begging the question: Have we reached, or are we already beyond a tipping point in the corporate use of these tools for strategic competitive advantage?

Bet2give: Can Markets Be Compassionate?

Just launched this week is bet2give, a prediction market offshoot of Newsfutures that enables "those who are best at predicting the future... to decide how to improve the future, one donation at a time."

I didn't want to spoil their cool and clever Christmas-colored typeface with a link, so here it is).

First take on the high concept: a great 'mash-up'. Sorta like Kiva plus Wall Street or Libertarians with a soft streak. Maybe Ayn Rand working a soup kitchen. Well OK, maybe not. But you get the idea...

Their tag-line: "What if you could win more than money?" I like it.

'Funny-money' (i.e., internal-currency, points-based) prediction markets--meaning nearly all U.S. and most corporate ones--have always operated on the theory that prestige, ego and accomplishment are powerful independent motivators to attract participants and have them take the market seriously--in other words to "play hard". They've suffered however, from the fact that those are their only drivers.

Get tired of the game's world and its limited prestige potential and it's easy to lose interest and move on. That's how it worked for me anyway. After obsessively managing a portfolio of propositions on Newsfutures several years ago I got bored and found better uses of my time. With no link to anything real beyond their world it was interesting but not compelling.

Real-money markets by contrast (while not without their own powerful motivations obviously) come with certain baggage. That includes conflicts of interest, negative public perceptions about and comparisons with seedier forms of gambling, and on a very practical level, U.S. restrictions on real-money markets that aren't licensed by the SEC or by state gaming commissions. That's a set of regulations that's kept U.S.-based real-money markets in the horse-and-buggy era while UK-based Tradesports has rocketed into the stratosphere.

Which is all a round-about way of saying that bet2give is definitely one to watch. I'm not sure how powerful it will be to have my local homeless shelter competing with your local soup kitchen, but there's potential for rough and focused play when say for example, my quasi-political charity competes with yours that's fighting against its objectives (use your imagination on that one).

Even when that kind of stuff is not at issue, bet2give will make me feel a whole lot better about playing with a 'pot' of money that I would never even think to devote to something most would compare with betting on horses.

What Can and Cannot Be Predicted (and Thoughts On Telling the Difference)

I just ran across this post ("Debating the Viability of Terrorism-Prediction Markets") over at the 'Footnoted' blog at the Chronicle of Higher Education discussing the risks of terrorism how best to assess them and whether they can be known at all. (If you've grown weary hearing about war, politics and terrorism, just skip to my non-terrorist business conclusions in the last paragraph.

In a recent interview [PDF] with the Federal Reserve Bank of Richmond, W. Kip Viscusi is asked about the public-policy response to the threat of terrorism and whether we are weighing the costs and benefits in a generally rational way. "The reason this is tricky is we don’t have very good numbers on what these risks are," Viscusi says. "The estimates of the probability of a terrorist attack or the number of people who are going to die in the coming year are all over the map. So if you can’t assess the likelihood of a terrorist attack or how deadly it is going to be, it is really hard to say how much you should spend to try to prevent it." [emphasis added]

Vscusi is characterized later in the article (and not without reason) as "one of the foremost academic experts on risk". His comment is in response to a persistent school of thought that claims, as Bruce Schneier does, that from an actuarial perspective, "terrorism doesn't happen".

Without the qualifier, the idea sounds terribly cold. With all reverence for the families of those who have lost loved ones in terrorist incidents however, it's not. Businesses and governments--not to mention non-profit institutions, individuals and families--all need to assess risks as rationally as possible and take measures to hedge them. Like it or not, the value of a human life can be defined--at least partially--in dollar terms. Anyone who's ever taken a course in economics or statistics (or balanced a checkbook for that matter) has figured out that infinite spending to protect against risk is infinitely foolish and that infinite spending on risk 'A' (and no spending at all on risks 'B', 'C', and 'D') is only a variant on the same wrong-headed assumption.

That's not what's really at issue. What is at issue is the degree to which the political leanings of some lead them to believe that we can know how much to spend combating terrorism and that the 'right' number is obviously much less than we are spending today (e.g., tactical domestic measures, overhead for business, strategic overseas measures, etc.) To which my response is: really? Show me your pre-9-11 white paper predicting the order-of-magnitude sea-change that occurred in that 'industry' on 9-11.

Bryan Caplan is one of Viscussi's critics. The CHE post notes:

"I am frankly puzzled," Caplan writes at EconLog. Citing the work of John Mueller,we have a long experience with terrorism, which has "shown it to be an extremely small problem in the broad scheme of things. How much longer does Viscusi want to wait before he'll conclude that the risk is very low?"

Unfortunately, long experience in and of itself is not sufficient for prediction, even at a macro level. Hold that thought for a quick diversion.

Here's where it gets weird. In criticizing Viscussi, Caplan, an econ prof at famously free-market GMU, ends up in league with Schneier, who as far as I can tell, tends towards the opposite end of the political spectrum. Both conclude, for entirely different reasons, that the future threat of terrorism can be known and condensed to a dollar figure and that rational budgets (both public and private) can be set accordingly. Oh that it were so.

Caplan, in particular "favors the establishment of a prediction market to help assess the likelihood of a terrorist attack". That's something I can conditionally applaud. If the results are used to "help assess", then we're fine. The possibility that a prediction market might help draw in and roll up marginal, highly distributed, even intuitive information that can supplement traditional (and sadly inadequate) intelligence-gathering mechanisms is certainly a good thing.  I've been a huge fan of prediction markets for years. They should be used for more things than they are today. To my delight, more and more are catching on.

But...

...as longtime readers know, I've also concluded that there are some (and arguably many) problems for which prediction markets are not only silly but grossly misleading. I don't have the space to review them all of them here. Thinking that they can precisely predict and quantify particular terrorist threats or even the threat of terrorism generally is a notion that falls into that category. Almost by definition, a successful terrorist venture is compartmentalized, secret and surprising.

In short, Caplan, Schneier and others appear to have an ideologically-induced blind spot that leads them to declare certainty where it does not exist. Let me explain.

Schneier and Caplan draw their essential argument from a backward-looking, actuary-style catalog of terrorist incidents. This many people died. This much property was lost. Productivity was reduced by this much for this long. Etc. Etc. It all sounds very rational. If we had reason to believe that terrorism were a natural, forecastable, perhaps even cyclical phenomenon, that approach would be absolutely correct. We don't.

The main problem, as I've noted before, is that:

The  unpredictability of terrorism renders any backwards-looking, purely quantitative, actuarial mode of analysis inappropriate and ineffective. That is, future deaths due to terrorism are something that neither Schneier nor anyone else can possibly predict with any degree of confidence.

Until 2001, the biggest single terrorist incident had caused around 300 deaths. Then in the space of a few hours, that number went up by an order of magnitude. There was no consensus (or even a significant plurality) of expert opinion predicting that that would happen - much less when, where and how. [emphasis in original]

And that's the problem.

Sudden, step-function, order-of-magnitude change, precipitated by a small group that has every reason to keep its plans secret is inherently unpredictable. It can only be imagined. If that sounds familiar outside of the terrorist context, it should. Businesses face this kind of challenge all the time; it is the very nature of business, in fact: snowboards vs. skis; digital photography vs. wet-process; PCs vs. mainframes; VoIP vs. legacy telephony; biotech vs. big pharma. The list is very long. It's harder to name an industry that hasn't been touched by this kind of change at some point (often precipitating the re-invention and re-definition of the former "industry") than it is to come up with a long list of examples of industries that have be altered in this way. Bottom line: it's important to differentiate between problems that lend themselves to forecasting and those that can only be dealt with via imaginative scenarios.

sdrawkcaB gniknihT - Mind Game or Creative Lever?

Related to my post yesterday about simulated hindsight, this Times of London article notes the value not only of working from a given end-point, but of actually 'telling' a story in reverse.

E.g., the firemen drove away, they packed up their gear, they put out the blaze, they broke down the door, they arrived on the scene, the mother called 911, the boy fled from his room, he screamed, his hair caught fire, he was playing with matches, he stole some matches from his father's desk drawer.

It's not easy to follow--or tell. That's the point in some contexts. Police have found it useful in attempting to catch a suspect in a lie.

Traditional police interview methods were used in the study, and in those that employed the reverse order tactics – described as “cognitive load interviews” – the interviewer asked the suspects to recall a series of events from the most recent backwards.

Officers were less likely to detect the liars when traditional methods were used in the interviews but were more likely to detect lies when the subjects were asked them to recall events in a reverse order.

It's analogous to something good proofreaders know well: spelling errors are easier to catch when one reads backwards. I was tipped off to the Times piece by innovative UK-based consultant Dave Snowden, over at Cognitive Edge. (H/T: Bob Weber) Snowden favors the backwards-telling approach because:

...by getting people to construct history in reverse... they explored more possibilities and were more open to novel discovery

That's all to the good. Snowden seems to position narrative as an improvement on scenario planning however. That's a bit hasty and sweeping.

...if you have strong opinions about what should happen, then it is easy to influence the evolution of a scenario that will support your proposed actions. Its also easy to describe how the past led to the present in such a way as to vindicate your view of history... I drew the ideas [for narrative] from various readings in the cognitive sciences which indicated that reverse time flow was harder, and disrupted what would otherwise be entrained processes. [link and emphasis added]

It's true that traditional (non-modular, non-interactive), 1970's/'Shell-style' scenario planning is particularly vulnerable to influence by strong personalities with political motives and rigid views. Any process can be steered by someone in power uncomfortable with open-ended 'novel discovery' who makes it clear to subordinates and unscrupulous consultants that there's a "right" and a "wrong" answer to the process s/he is paying for. A centralized process is 'breakable' with concerted, centralized influence. Practically every client we work with has taken us aside at some point to warn us of this in one form or another:

Before we get going, you need to know about Mr. Smith, the head of our XYZ division. If he ever gets the floor, he will kill this. Everyone will be forced to go along with his ideas. He's a smart and he'll twist this. He's a bully and a blowhard and he's got the influence to bend people to his will and make this come out his way. Watch out.

And every time, Mr. Smith (and it's usually a 'Mr.') finds himself--in our meeting--in a situation where his personal air-time is far more constrained than he's used to and his contributions are on par with everyone else. Looking for an opportunity to drive the outcome, he instead finds himself sorting through modular scenario piece-parts, unable to know in advance how the collective intelligence of 20-40 people will render them all into an interlocking scenario map.

When scenarios are built in a more distributed fashion from modular piece-parts in the context of a highly interactive, participatory, fast-moving, tightly choreographed session however, any one person's ability to steer becomes minimal. The process is democratizing, taking advantage of emergent 'wisdom of crowds' effects that nobody can see, much less steer until the end when the whole picture is developed. Some hierarchical, top-down cultures (both corporate and national) don't like our process for precisely that reason.

Story-telling is only part of it and teams will often independently elect to build a scenario story in reverse--quite possibly for the reasons Snowden suggests. His process--to my knowledge--takes advantage of similar dynamics. My contrast here is not with narrative but with traditional ('paleo-') scenarios--the kind most people are familiar with.

I've made the analogy before that traditional scenario methods are to mainframe computers as modular, interactive scenarios are to PC networks. It's not perfect, but it holds up for 'hacking' and influence as well: a network of independent actors tasked with constructing a scenario 'map' is far more resilient than a monolithic (e.g., top-down) process.

Markets make for an even better analogy (one of the reasons I continue to link them with modular scenarios). Broadly-based, highly liquid markets (including PMs) are less subject to influence than monopolies/oligopolies (one/few sellers) and monopsonies/oligopsonies (one/few buyers)  .

I don't know Snowden's process intimately, but I've been through some interesting demonstrations by one of his acolytes professional colleagues, Patti Anklam, whom I hold in high regard. Her main website can be found here. [italicized stuff added] Fascinating stuff. Powerful. And different from scenarios. Hardly a substitute for them (or vice versa!). I see two major distinctions:

1. Reverse-order storytelling, because of its "high cognitive load" (my brain hurts!), favors those with greater ability to bear such a load and/or greater experience using the technique. In other words: the same executives liable to steer any process, as well as (sometimes) the consultants. Requiring such an approach may to put a damper on inclusiveness and equal participation. Great for police questioning and trying to trip up a suspect; not necessarily so great when trying to build a team and gain their enthisiastic buy-in to a common vision.
2. The novelty-seeking open-endedness that narratives tend to produce can be just the opposite of what some organizations need. When the problem is completely unbounded (little data; near-infinite possibility; minimal understanding), scenarios are of little use and narrative may be most appropriate. When the problem is more about finding distinct points of divergence and convergence between several "schools of thought" (some data, numerous but not infinite possibilities, modest understanding) then scenarios tend to be more appropriate.

Perils of Prediction: The Elusiveness of Certainty and the Value of 'Simulated Hindsight'

I'm ten days late in posting on a great short take in The Economist ("The Perils of Prediction"), reviewing Nassim Nicholas Taleb's book, "The Black Swan: The Impact of the Highly Improbable"

...almost all forecasters work within the parameters of the Gaussian bell curve, which ignores large deviations and thus fails to take account of “Black Swans”. Mr Taleb defines a Black Swan as an event that is unexpected, has an extreme impact and is made to seem predictable by explanations concocted afterwards.

Taleb is correct: people like to create (and gravitate to) ex post facto explanations. We usually think of those as being of very little use (unless we're in the business of publishing sensationalist books). By definition, such explanations don't help to anticipate or deal well with the next unprecedented thing--even though people would like to think that they will. Unprecedented means, well... unprecedented. There will never be another 9-11, even though there will probably be other big nasty surprises that kill lots of people that we analogize to 9-11.

That's all by way of background to explain why we use a simple trick in our scenario workshops called "simulated hindsight". Short take: tap people's natural hunger for ex post facto sense-making stories explaining unexpected developments... using hypothetical future 'facts'.

Disoriented yet?   :)

Here's the slightly longer take: Assume it's 2012. Put away this morning's newspaper, as well as all of your assumptions and prognostications about what might happen in July and August, and in 2008 and 2009, and so on. That's all part of 'history' now. It's your job to explain it.

Be in 2012. Assume that the world has already turned out in a certain way that I've spelled out in a tightly-crafted one-page document we call an 'endstate'. Put on your historian 'hat' (or your research analyst 'hat' or reporter/news-anchor 'hat'). Your job is to tell us how the world got to be the way it is now in 2012. What were the major turning points since 2007? What developments led to what other ones? What things didn't happen that some people thought were virtually certain? What were some of the pivotal surprises that drove things to turn out the way they did?

Using a set of 150 discrete, short descriptions of things that might or might not have happened between June, 2007 and "now" (2012)--we call them 'events'--construct a story of how we got "here". Several other teams will do the same with their own separate 'endstates'. Pay no attention to them. They'll get to tell their 'history' stories too and then we'll compare and contrast them and look for the key points of intersection and divergence.

That's simulated hindsight and I've seen it work powerfully to unlock the thinking of hundreds of executives (probably thousands by now, come to think of is), enabling them to contemplate how various kinds of surprises might change their business.

The Economist's review of Taleb's book offers other gems [emphasis and link added]:

...humans have an uncontrollable urge to be precise, for better or (all too often) worse. That is a fine quality in a watch-repair man or a brain surgeon, but counter-productive when dealing with uncertainty... Why, [Taleb] asks, do we take absence of proof to be proof of absence? ...Mr Taleb argues convincingly that the spectacular collapse in 1998 of Long-Term Capital Management was caused by the inability of the hedge fund's managers to see a world that lay outside their flawed models. And yet those models are still widely used today...

...corporate “scenario planners” are better than they used to be at thinking about Black Swan-type events... [Taleb] suggests concentrating on the consequences of Black Swans, which can be known, rather than on the probability that they will occur, which can't (think of earthquakes). But he never makes professional predictions because it is better to be “broadly right rather than precisely wrong”.

All of which helps to explain why--despite their unquestionable value in an organization--financial executives, engineers, and those with talent for managing the details of day-to-day operations tend to be much less comfortable confronting the broad implications of longer-term uncertainties or dealing with the imprecision inherent in potentially sudden, unprecedented change. I'm not that good at the other stuff. Driving across the Golden Gate bridge recently, I gave thanks that we've got our domains of complementary expertise.

UPDATE: One amusing side-effect of Googling "Perils of Prediction" (the title of the Economist article) was the number of other pieces that came up (934), including this general treatment:

Perhaps the most difficult pitfall to avoid is when the predictor fails to take into account a factor that may not even exist at the time of making the prediction.

This take on the special problems of forecasting technology evolution:

In 1967, the 100-year-old company Keuffel & Esser was commissioned to study the future. A major failure of its analysis was not seeing that its own flagship product would become obsolete in just a few years. K&E was the country's leading slide-rule manufacturer, and it was blindsided by the product it failed to see, the electronic calculator.

This insanely precise (and internally inconsistent) long-term economic forecast (commentary here):

George Mason University's Stephen Fuller called up a PowerPoint slide predicting that in 2057, the average annual household income for the region will be $1,307,000. Whoo hoo! That sounded great. Then he pointed out that in 50 years, the average Washington area house will cost a whopping $14,061,000.

This description of the "butterfly effect" in perhaps its most classic form:

A little discrepancy in the pattern of air flowing more than 4,000 miles away had made the difference between an accurate forecast and a bust. The change in the winds in Alaska had displaced storms in the southeast by several hundreds of miles-endangering people living near Orlando, not New Orleans.

This semi-prescient August, 1998 assessment of economic models designed to predict currency crashes (right on the cusp of LTCM's almost world-cataclysmic implosion):

Investment banks and academic economists are building complicated models to predict currency crashes. Don't expect them to work.

And finally, this amusing if cautionary catalogue of bad predictions by Cynthia Crossen in the WSJ last January 8th (subscribers only):

"The giant airplane of 300- to 400-passenger capacity, while technically possible," wrote a U.S. aviation official in 1944, "appears to offer little economic advantage and to involve a great sacrifice of convenience for the traveler, owing to the inevitable reduction in scheduling frequency which results from using such large units."
...
In 1937, Hadley Cantril, a psychology professor at Princeton University, studied the relative prophetic ability of various types of people. He sent a questionnaire asking for predictions about world affairs to several hundred people... The two groups who were most confident that their predictions were correct were the bankers and the Communists.