James Surowiecki on the Wisdom of Crowds

From P2P Foundation
Jump to: navigation, search

Webcast

"Hear the award-winning New Yorker columnist talk about the power of large groups (including on the Internet) to make remarkably intelligent decisions. The session will also cover the problems and challenges that large groups present."

URL = http://server1.sxsw.com/2006/coverage/wisdom_of_crowds_pn_int_2006_lo.mp4


Print Interview

Q&A session at the publisher's site, at http://www.randomhouse.com/features/wisdomofcrowds/Q&A.html


Excerpts

How did you discover the wisdom of crowds?

The idea really came out of my writing on how markets work. Markets are made up of diverse people with different levels of information and intelligence, and yet when you put all those people together and they start buying and selling, they come up with generally intelligent decisions. Sometimes, though, they come up with remarkably stupid decisions—as they did during the stock-market bubble in the late 1990s. I was interested in what explained the successes and the failures of markets, and as I got further into it I realized that it wasn't just markets that were smart. In fact, crowds of all sorts were often remarkably wise.

Could you define "the crowd?"

A "crowd," in the sense that I use the word in the book, is really any group of people who can act collectively to make decisions and solve problems. So, on the one hand, big organizations—like a company or a government agency—count as crowds. And so do small groups, like a team of scientists working on a problem. But just as interested—maybe even more interested—in groups that aren't really aware themselves as groups, like bettors on a horse race or investors in the stock market. They make up crowds, too, because they're collectively producing a solution to a complicated problem: the bets of people betting on a horse race determine what the odds on the race will be, and the choices of investors determine stock prices.

Under what circumstances is the crowd smarter?

There are four key qualities that make a crowd smart. It needs to be diverse, so that people are bringing different pieces of information to the table. It needs to be decentralized, so that no one at the top is dictating the crowd's answer. It needs a way of summarizing people's opinions into one collective verdict. And the people in the crowd need to be independent, so that they pay attention mostly to their own information, and not worrying about what everyone around them thinks.

And what circumstances can lead the crowd to make less-than-stellar decisions?

Essentially, any time most of the people in a group are biased in the same direction, it's probably not going to make good decisions. So when diverse opinions are either frozen out or squelched when they're voiced, groups tend to be dumb. And when people start paying too much attention to what others in the group think, that usually spells disaster, too. For instance, that's how we get stock-market bubbles, which are a classic example of group stupidity: instead of worrying about how much a company is really worth, investors start worrying about how much other people will think the company is worth. The paradox of the wisdom of crowds is that the best group decisions come from lots of independent individual decisions.

What kind of problems are crowds good at solving and what kind are they not good at solving?

Crowds are best when there's a right answer to a problem or a question. (I call these "cognition" problems.) If you have, for instance, a factual question, the best way to get a consistently good answer is to ask a group. They're also surprisingly good, though, at solving other kinds of problems. For instance, in smart crowds, people cooperate and work together even when it's more rational for them to let others do the work. And in smart crowds, people are also able to coordinate their behavior—for instance, buyers and sellers are able to find each other and trade at a reasonable price—without anyone being in charge. Groups aren't good at what you might call problems of skill—for instance, don't ask a group to perform surgery or fly a plane.

Why are we not better off finding an expert to make all the hard decisions?

Experts, no matter how smart, only have limited amounts of information. They also, like all of us, have biases. It's very rare that one person can know more than a large group of people, and almost never does that same person know more about a whole series of questions. The other problem in finding an expert is that it's actually hard to identify true experts. In fact, if a group is smart enough to find a real expert, it's more than smart enough not to need one.

Can you explain how a betting pool can help predict the future?

Well, predicting the future is what bettors try to do every day, when they try to figure out what horse will win a race or what football team will win on Sunday. What horse-racing odds or a point spread represent, then, is the group's collective judgment about the future. And what we know from many studies is that that collective judgment is often remarkably accurate. Now, we have to be careful here. In the case of a horse race, for instance, what the group is good at predicting is the likelihood of each horse winning. The potential benefits of this are pretty obvious. If you're a company, say, that's trying to decide which product you should put out, what you want to know is the likelihood of success of your different options. A betting pool—or a market, or some other way of tapping into the wisdom of crowds—is the best way for you to get that information.

Can you give an example of a current company that is tapping into the "wisdom of crowds?"

There's a division of Eli Lilly called e.Lilly, which has been experimenting with using internal stock markets and hypothetical drug candidates to predict whether new drugs will gain FDA approval. That's an essential thing for drug companies to know, because their whole business depends on them not only picking winners—that is, good, safe drugs—but also killing losers before they've invested too much money in them." (http://www.randomhouse.com/features/wisdomofcrowds/Q&A.html)



More Information

For the book see the entry on the Wisdom of Crowds