WaPo Misguided on 'Experts' vs. the Swarm

H/T to reader 'AT' for this WaPo piece purporting to have discovered a new Internet phenomenon. Unfortunately, it reads more like a lightly buttered press release gushing with premature praise and hope for the firms featured in it than a serious examination of the competing philosophies of finding correct answers and discerning the outcome of future events.

The article takes the academically and empirically well-proven notion of collective intelligence (aka the wisdom of crowds), and presumptively twists it into the more comforting and familiar (if statistically invalid) notion that "father-knows-best"--only we haven't done a good enough job of finding the right "father" up until now.

...a small number of Web sites [are] seeking to turn the wisdom of the Internet on its head by sifting through its vast number of users to identify a handful of experts. If this novel approach withstands scrutiny, the reverberations could extend well beyond sports betting to include stock trading, popular culture and other realms.

For the past decade, much of the Internet has been animated by the "wisdom of crowds"; the notion that the tremendous masses drawn to the Web can together provide collective knowledge that outperforms even that of experts. By marshaling the knowledge and tastes of millions of people, the Web has fundamentally changed the way people can gain knowledge about their world...

But this wisdom of the crowd could be outsmarted by what Michael Arrington, editor of the TechCrunch blog, recently dubbed the "wisdom of the few." Sites like PicksPal rely on input from the masses chiefly as a venue for auditioning prospective experts, on the theory that these virtuosos could provide even more accurate information and predictions than the crowd.

"If you figure out which ones did the best and get rid of the ones who have no idea, you'd do even better. Distill it down to the people who really know," Arrington said.

While generations have looked to pundits for guidance, it has often taken a long time for their expertise to be recognized, and many have remained in obscurity. Now the Internet promises new ways to discover those who might otherwise get overlooked.

Prediction market stalwarts Justin Wolfers and Robin Hanson take the ideas to task--though readers would have to persist to near the end of the article to discover such criticisms (ironically from the experts whose opinions actually matter in this case.)

Wolfers... said collective wisdom... is more likely to be accurate than Web sites claiming to feature experts. Someone must have a track record stretching back decades before it is statistically possible to conclude whether success results from talent or random chance, he said. "Folks who look like experts today are very likely to be lucky," Wolfers said. "If they're conditioning it only on past history, it's likely to be a lost cause."

...Even if sites are able to identify expertise, several business professors questioned why experts would donate their wisdom to the Web rather than striking out on their own to make money. "If they're really consistent, it's kind of hard to see how the sites will survive. The experts will leave," said Robin Hanson, an economics professor at George Mason University.

What the article doesn't take space to note (but which I know Wolfers and Hanson know well) is that prediction markets do an even better job of rewarding experts because 1) they are able to increase their influence as a direct result of, and in proportion to the stupidity or inaccuracy of the dilettantes and 2) experts betting anonymously or pseodonymously can bring in information that they otherwise might be reluctant to offer under their own name.