Wisdom of Crowds

From P2P Foundation
Jump to navigation Jump to search

The Wisdom of Crowds = the theory that a larger group of diverse people can make better decisions, and display more intelligence than any smaller collection of experts

James Surowiecki's book of the same title focuses on collaborations whereby "a diverse group of individuals work largely independently of one another".

See the critique of Wisdom of Crowd applications in the Unwisdom of Crowds.

Key conditions for success

James Surowiecky: "“There are four key qualities that make a crowd smart.

1) It needs to be diverse, so that people are bringing different pieces of information to the table.

2) It needs to be decentralized, so that no one at the top is dictating the crowd’s answer.

3) It needs a way of summarizing people’s opinions into one collective verdict.

4) And the people in the crowd need to be independent, so that they pay attention mostly to their own information, and not worrying about what everyone around them thinks." (paraphrased by Sam Rose)

Surowiecki identifies five things that qualified crowds can -- if asked appropriately -- be very good at:

  • ascertaining (all the) pertinent facts surrounding an issue
  • predicting outcomes
  • making a decision among a discrete set or finite range of alternatives
  • determining an optimal process to follow (in simple or complicated situations, but not complex ones)
  • assessing causality (in simple or complicated situations, but not complex ones)

In all except the first type, the crowd must be given a set or range of alternatives to choose from, and, when they are, Surowiecki says, the 'errors' in judgement tend to cancel each other out, so that the crowd's consensus tends to be consistently better than that of executives, consultants and other experts."

Paraphrased by Dave Pollard at http://blogs.salon.com/0002007/categories/businessInnovation/2006/09/21.html#a1650

Optimal Use of Wisdom of Crowds Processes in Organizations

Proposed by Dave Pollard:

" the optimal process, for complicated (not complex) problems:

1. The executives identify and qualify a crowd of co-workers, customers (including prospective customers) and informed members of the public, and interview them, in interactive sessions witnessed by the organization's creative people, to augment their (the executives' and the crowd's) collective knowledge of the problem, knowledge of solutions that have worked in the past in similar situations, experience solving similar problems, knowledge of people who can help solve the problem, and knowledge of relevant tools, models and methods that can help.

2. The executives then charge the creative people (who by virtue of their involvement in step 1 now have a deep contextual understanding of the problem and how to approach it) with imagining new solutions that might work to solve the problem, working both individually and as a team. These creative people do not assess or rank these potential solutions -- their job is simply to identify alternatives.

3. The executives then canvass the crowd from step 1, presenting them with the solutions that have worked in past, those which the executives based on their experience think have potential, plus the alternatives that were surfaced in step 2. The crowd makes the final decision.

This learn-analyze-imagine-assess-decide-on-action process involves each group of stakeholders doing what they do best. If there are appropriate incentives for the crowd (and sometimes that's as simple as recognition and thanks), this process need not be cumbersome, and to some extent it can be automated (members of the 'crowd' can to some extent self-qualify by going through an online qualification survey, and step 3 can also be done entirely online). It is course frightening to executives, because it reveals their true, limited value in the decision-making process. In fact just about anyone can perform the three steps above (they are mostly administrative and facilitative), bringing into question the need for highly-paid executives, and a hierarchical decision-making organizational structure, at all. So this approach is clearly more amenable to egalitarian, non-hierarchical organizations. It's also bad news for the consultants and outside experts -- they aren't needed in the process at all." (http://blogs.salon.com/0002007/categories/businessInnovation/2006/09/21.html#a1650)


Difference between Wisdom of Crowds and Collective Intelligence

Sam Rose explains the issue, inspired by Henry Jenkins, at http://blog.p2pfoundation.net/?p=662:

"According to Henry Jenkins, the “wisdom of crowds” is applicable towards aggregating dispersed knowledge about quantifiable, objective data, while “collective intelligence” is intelligence that derives from collective behavior and stigmergic, and/or consensus decision making.

The need for independence among “crowd” members contrasts with the requirement for connection and collaboration to see collective intelligence work.

The Wisdom of Crowds generally breaks down when information sharing/group think starts to skew and bias people towards errors. Collective Intelligence overcomes this by looking at different ways that groups can systematically enhance and improve collaboration and cooperation." (http://blog.p2pfoundation.net/?p=662)

When aggregation is used in the wrong way, drowning out individual creativity and difference, what results is not wisdom, but Groupthink or the Dumbness of Crowds.

From a blog entry by Kathy Sierra at http://headrush.typepad.com/creating_passionate_users/2007/01/the_dumbness_of.html

Some examples of this dumbness, contrasted with true Collective Intelligence:

"Collective intelligence" is a pile of people writing Amazon book reviews.

"Dumbness of Crowds" is a pile of people collaborating on a wiki to collectively author a book. (Not that there aren't exceptions, but that's just what they are--rare exceptions for things like reference books. I'm extremely skeptical that a group will produce even a remotely decent novel, for example. Most fiction suffers even with just two authors.)

"Collective Intelligence" is all the photos on Flickr, taken by individuals on their own, and the new ideas created from that pool of photos (and the API).

"Dumbness of Crowds" is expecting a group of people to create and edit a photo together.

"Collective Intelligence" is about getting input and ideas from many different people and perspectives.

"Dumbness of Crowds" is blindly averaging the input of many different people, and expecting a breakthrough. (It's not always the averaging that's the problem it's the blindly part)

"Collective Intelligence" is about the community on Threadless, voting and discussing t-shirts designed by individuals.

"Dumbness of Crowds" would be expecting the Threadless community to actually design the t-shirts together as a group." (http://headrush.typepad.com/creating_passionate_users/2007/01/the_dumbness_of.html)

How the Wisdom of Crowds produces lowest common denominator choices

Mark Fawzi [1]:

"the designations of ‘condensed’ and ‘dispersed’ given below for crowds are relative to the ability of the members of the crowd to communicate with each other and affect each other’s judgment.

A dispersed crowd (without a non-arbitrary hierarchy) will produce averaged judgment. For example, asking each of 200 people (not at the same time or place) how many jelly beans are in a jar would result in an averaged judgment, which would eliminate values that are too high or two low, resulting in an estimate of the number of jelly beans in the jar (which is a measurable value) that is close to the actual value. In this case the crowd is nothing more than a decent statistical calculator. It has not exhibited any more wisdom than the tool it is being used as.

A condensed crowd (without a non-arbitrary hierarchy) may produce averaged or lowest-common-denominator judgment, depending on whether or not its judgment is rationally or psychologically driven. In case the judgment is about a measurable value it would most likely be rationally driven, and, thus, be an averaged judgment. In case the judgment is about a quality it would most likely be psychologically driven, and thus, be a lowest-common-denominator judgment. In the rational case, the assumption is that, even though the crowd’s members can communicate with and affect each other’s judgment, if each member is rational enough and the judgment to be made concerns a measurable value then the crowd will likely produce an averaged judgment (i.e. the average of independent judgments.) If, however, the crowd members affect each other’s judgment (which would happen mostly in the case of judgments about quality rather than judgments about a measurable value, i.e. when reason is suspended and psychology takes over) then the crowd’s judgment will tend towards the lowest common denominator.

A typical crowd is a mix of both the dispersed and condensed crowds. Thus, its range of judgment with respect to both measurable value and quality include both averaged as well as lowest-common-denominator judgments.

The problem with averaged judgment when it’s applied to quality (rather than measurable value), which can happen in a typical crowd, is that you end up with mediocre judgment (by definition.)

The problem with lowest-common-denominator judgment when it’s applied to measurable value or quality is that it uses the primitive part of our psychology. In other words, expect exactly the opposite of wisdom.

So a typical crowd is going to be either a mediocre judge or an unwise one. And nothing else." (http://evolvingtrends.wordpress.com/2006/07/07/web-25-from-hunter-gatherer-to-democratic-society/)

More Information

  1. This blog commentary from Dave Pollard includes a decision tree to ascertain in which specific instances the wisdom of crowd can be used optimally, see at http://blogs.salon.com/0002007/categories/businessInnovation/2006/09/21.html#a1650
  2. Yihong Ding: The Secret behind the wisdom of crowds: excellent contribution.
  3. Interview with James Surowiecky.

Listen and watch:

  1. James Surowieki on the pitfalls of the Wisdom of Crowds
  2. James Surowiecki on the Wisdom of Crowds

Key Book to Read

Book: The Wisdom of Crowds. Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations. James Surowiecki. Doubleday, 2004

"This book discusses the theory that a larger group of diverse people can make better decisions, and display more intelligence than any smaller collection of experts. Surowiecki's central concept is that the insights of a diverse group of individuals working independently can be aggregated together." (Review by Chris Cowan at http://humergence.typepad.com/the_never_ending_quest/2006/03/book_review_the.html)

"While our culture generally trusts experts and distrusts the wisdom of the masses, New Yorker business columnist Surowiecki argues that "under the right circumstances, groups are remarkably intelligent, and are often smarter than the smartest people in them." To support this almost counterintuitive proposition, Surowiecki explores problems involving cognition (we're all trying to identify a correct answer), coordination (we need to synchronize our individual activities with others) and cooperation (we have to act together despite our self-interest). His rubric, then, covers a range of problems, including driving in traffic, competing on TV game shows, maximizing stock market performance, voting for political candidates, navigating busy sidewalks, tracking SARS and designing Internet search engines like Google. If four basic conditions are met, a crowd's "collective intelligence" will produce better outcomes than a small group of experts, Surowiecki says, even if members of the crowd don't know all the facts or choose, individually, to act irrationally. "Wise crowds" need (1) diversity of opinion; (2) independence of members from one another; (3) decentralization; and (4) a good method for aggregating opinions. The diversity brings in different information; independence keeps people from being swayed by a single opinion leader; people's errors balance each other out; and including all opinions guarantees that the results are "smarter" than if a single expert had been in charge. Surowiecki's style is pleasantly informal, a tactical disguise for what might otherwise be rather dense material. He offers a great introduction to applied behavioral economics and game theory." (Publisher's Weekly, cited in http://innovationcommons.blogspot.com/2006/06/wisdom-of-crowds.html)