Obstacles to Prediction Market Adoption

Harvard Business School Associate Professor Andrew McAfee asks, in a recent blog post, why prediction markets haven't been taken up more readily in the corporate world. He writes [links added]:

After writing the [Google prediction markets] case, teaching it a few times, and spending some time understanding the mechanics and utility of prediction markets, I share the puzzlement articulated by James Surowiecki in his book The Wisdom of Crowds:

". . . the most mystifying thing about [prediction] markets is how little interest corporate America has shown in them... companies have remained, for the most part, indifferent to this source of potentially excellent information, and have been surprisingly unwilling to improve their decision making by tapping into the collective wisdom of their employees."

Why is this? It’s not because the technology is hard to acquire... what is the real stumbling block? Is it that companies don’t really want the most accurate information about future events to come out and be widely known?

Sean Silverstone Silverthorne, the editor of HBS Working Knowledge expands on the question in a post last week, outlining what he sees as "three initial roadblocks":

• Unfamiliar with the Concept -- Most managers, especially the veterans, grew up hearing not to trust crowds, to avoid mob rule. Instead, they were encouraged to value expertise and be ready to pay consultants. It’s a major brain-bender to suddenly believe crowds can be smarter than experts.
• Internal Friction -- Corporate prediction markets rankle folks like product managers when company employees start betting on a bad outcome for their product.
  
  
• Implementation Issues -- Until recently, there wasn’t a whole lot of best practices around how to run efficient prediction markets. Results can be seriously skewed by having too few participants, asking the wrong type of questions, or by running a market too long or too short a time.

From my experience working directly with prediction market vendors and corporate implementers, and having watched, thought about and pontificated on this phenomenon for over a dozen* years, Silverstone's points seem reasonable as high-level headings. Several caveats and elaborations may be useful on this rapidly-moving target, however. *(I first heard about them in a pitch by Ken Kittlitz to the Digital Commerce Society of Boston circa 1995/96 -- shortly after I nominally helped co-found that late and loosely constructed organization, and shortly after Kittlitz founded the Foresight Exchange)

First, the number of managers completely unfamiliar with the concept of prediction markets has dwindled markedly over the past five years -- and at an accelerating rate. Silverstone is, I'm sorry to say, somewhat behind in writing that:

A relatively small number of companies including Google, HP, Intel, Yahoo, and Eli Lilly have jumped on this idea by creating prediction markets to aggregate the opinions of employees...

That was true five years ago. It is not the case today.

Doing some research into this recently, I was pleasantly startled to discover that there are at least fifty large corporations running prediction markets 'for real' now, and countless others running small, informal and for the most part un-trackable pilots. (Virtually all of the major vendors offer such trials, though with varying degrees of publicity and enthusiasm as well as varying degrees of ease in setting them up -- though none are terribly difficult). And that's not counting the number of academic institutions and news media organizations that run prediction markets for other reasons (e.g., experimental learning and securing reader attention, respectively).

Add to that the fact that HBS and HBR are covering this topic, conferring a halo of corporate legitimacy, that Surowiecki's book continues to thrust the key concepts into business management if not lay-public consciousness, that barely a week goes by without a major publication giving some ink to the subject (I've stopped even blogging them individually, there are so many) and that the proportion of people in the management class who trade various financial instruments is very high. I.e., they recognize, at some level, that prices represent crowd wisdom -- a body of information to which they may or may not be privy, but which they ignore at the peril of their 401K.

So, given that backdrop, I would turn Silverstone's point around and ask: What excuse can a manager offer, in May, 2008, for being totally 'unfamiliar with the concept' of prediction markets?

Wait. Don't answer that. If your boss isn't reading this, your CEO or your board very well may be, and that's not an idle quip. Some of the most important prediction market initiatives I've seen have come down from the very top, rather than from middle management...

...which leads me to Silverstone's second point: "Internal Friction". That would be putting it mildly.

Some cultures palpably chafe at the idea of unleashing democratic forces within their walls. This is not a scintillatingly deep or novel insight. We know this to be true for countries, and families, as well as for bowling leagues and PTOs for that matter. (I've also observed this when running scenario workshops within certain corporate and, treading lightly here, societal cultures).

Humans almost universally seek some kind of power or at least stability. They work hard for it. And they're loathe to give it up to some impersonal, anonymous and (it can sometimes appear) fickle and BS-penetrating tool (however powerful) that doesn't even have to justify its reasons. (Imagine an election in which each voter had to justify his or her reasons -- to someone -- before casting a vote and you'll understand what I mean.)

Which is all to say that the issue goes well beyond friction to questions of culture, organizational structure, leadership, and the role or perceived role of middle managers. Classically the role of 'middle management' (even the term has taken on a derogatory ring) has been to act as filters and conveyors of information moving between senior management and workers.

Prediction markets usurp some of that role and that can be, quite understandably, unsettling to them. Nobody ever said business was fair. But, bottom line: it is a real reason why prediction markets need to be contextualized carefully in order to succeed. Repeat after me: it's not about the software (mostly).

Which brings me to Silverstone's last point: Implementation Issues. Let's be clear about one thing: 'implementation' here does not refer to the kind of thing one thinks about when installing a big new software system. The software for these things is pretty darned simple in that regard.

The much bigger questions have to do with organizational issues. And those can be sweeping.

For example:

What questions is an organization, a) truly interested in knowing the answer to (strategic), b) willing to tell the employees they might not know the answer to (potentially humiliating) and, c) willing to feed back information on how and why we did or did not act on that input (empowering of morale and loyalty... or cynicism... depending on how it is handled). Just the first part of that is a big, hairy strategy question that goes right to the heart of what a company wants to be when it grows up and where it thinks it is currently 'stuck'.

Bottom line: prediction markets harbor all the paradigm-changing, employee-empowering breakthrough-idea-generating, process-improving, cost-saving potential of the suggestion box... as well as all of its potential as a reason to deride management.

Here are the twists that can and do cause managers to think carefully about the transparency they're prepared to unleash: prediction markets are real-time, they're at least nominally public, and the information they generate isn't really a suggestion (casual) so much as it is a serious investment of time, thought and expectation on the part of those asked to participate. One has to ask then: With all of that potential for discomfort, why have so many big companies begun using them in earnest?

Comments

[Noam Danon]

Great analysis of the prediction markets arena.
There's one point I slightly disagree with - people still don't know what prediction markets are. My experience shows, that 9 out of 10 people you talk to, have never even heard of it.
The interesting thing about them though, is that all 9, after hearing about them, are extra excited and agree they seem like a wonderul idea.

I'm hoping all this is good news for the prediction markets industry...

[Art]

Nice refinement, Noam. (I'm not sure we really disagree.) Managers may not have had direct experience with the tools and techniques, but the cognitive 'soil' for "getting it" quickly is far more fertile than it was, say, five years ago. I asserted to someone recently that it feels like we've "crossed the chasm". I.e., the market for prediction markets has moved from experimental/fringe (pre-2004) through risk-takers and early adopters (2004-2007, plus or minus) to having enough clear, visible, quantitatively grounded proof points out there that one cannot say "this is too risky for me to waste time on". Naturally, there will always be laggards.

[sean silverthorne]

Hey,

Thanks for referring to my blog on this subject and sorry it took so long to circle back to your thoughtful comments on what I wrote originally.

Several thoughts.

1. It's Silverthorne, not Silverstone. Error is often made, but my mother cringes.

2. Yes, 50 big companies are looking into PMs, and "countless" other organizations as well (I actually bet you could count them) but these are companies such as Google and HP that can afford to and think it valuable to explore envelope-pushing opportunities. Fifty is not a big number. Tens of thousands middle-and large-companies are clueless on the subject. More to the point, I've heard a Google exec talk about using PMs inside that organization and there is little agreement on their value below the C-level. If Google rank-and-file aren't sold (yet) I doubt this is a technology on the fast track to corporate acceptance.

2. Yes, more and more managers have heard of prediction markets, but it's a big stretch to think they understand how they work or could sell their value to non-converts. Some could; most can't. Prediction markets need more troops on the ground to sell the concept upstairs, IMHO.

3. I'm not sure if we agree or disagree on this, but I think organizational cultural issues are the biggest roadblock to widespread adoption of PMs. Too many hide-bound interests (and managers) are DNA'd against a truthful and public exchange of information in the form of informed bets.

All that said, I think PMs will become a valuable tool in assessing and informing corporate strategy and tactics. I just think the roadmap for this is 5-10 years rather than 2- 5 years.

Sean

[Art Hutchinson]

Hey Sean - Thanks for stopping by. Apologies on the name thing. My brain has a way of doing that sometimes.

We probably agree more than disagree on this stuff: glass half-full vs. glass half-empty. Fifty on a hockey-stick curve to five thousand is a big deal. Fifty this year and sixty the next is not.

It's hard to tell which is true when you're in it. (Analysts like to predict hockey sticks; the fact that they're wrong a lot doesn't mean they're not right from time to time...)

Re. counting the companies doing random pilots: Yeah, one could add up the number of free trials various vendors have seen. It would be pretty thin at the edges though. E.g., some junior person started a trial on their lunch hour and then forgot about it.

Organizational roadblocks: yeah. And perhaps even roadblocks in human nature. Since I wrote this post, I've had two clients begin implementing prediction markets. In each case there's been some subtle reluctance to go full-bore for them. (One just took time. Now it's working well. The other is punking along far more slowly; culture plays a big part in both.) These things are inherently dis-empowering to management -- even when they feel they *have* to express excitement about them.

To make an extreme analogy, it's as if the CEO stood up in front of a room filled with employees (big enough and dark enough for it to be fairly anonymous) and said: "OK, show of hands, who thinks this new market we're entering is gonna amount to a hill o' beans?" Whatever the answer, he's got to do something with it. He's effectively naked up there while folks vote. And simply asking, he's ceded some control and power. It takes a particular level of personal confidence ('brass ones') to do that. So yeah: culture.

Again, apologies on muffing your name, Sean.