"Wisdom of the crowds" is a phrase that I'm hearing more and more lately, and I knew it was more than the soundbite description of "a large group makes better decisions than any single member of that group." According to Wikipedia[1], this basic definition is true, but it also comes with four necessary conditions:- Diversity of opinion: each person should have their own information, even if that information is their opinion of shared facts.
- Independence: group members' opinions are their own, and aren't affected by others in the group.
- Decentralization: when it makes sense, group members can rely on local knowledge or their own skills and specialization.
- Aggregation: there exists a way to take individual opinions to form a collective decision.
This makes intuitive sense, as we feel that "wisdom of the crowds" is somehow different from mere "mob mentality."
The one that often gets muddled and forgotten, though, is "Inde
pendence." We understand why this is important, yet often is the hardest to implement.
Take Digg for example. This seems like the purest form of "wisdom of the crowds." Someone submits a link, and the crowd decides via a thumbs-up-or-not decision whether this link is worthy of promoting. But already, this is colored simply by the submission itself. Without a true diversity of links submitted, you get a biased list of great links and content on the Web.
But worse than this is the fact that you can always see how many other people have decided that they like this link. This no doubt affects a Digger's propensity to first of all rate the link, then whether or not it gets promoted. For the user who is looking to follow the crowd, they are likelier to read and rate the link. For the users
with independent streaks, they may see a link that isn't getting as many Digg's as they think it should, and are likelier to promote it.
Another example of this is Google Moderator, the tool that allows people to submit questions or items through a web-based interface which then are voted on by users. Here, the submission has an even stronger effect on the bias of the output.
The earlier one submits their question or item, the less competition it has vs. later submissions, and is simply available for a longer period of time to garner more votes.
While it seems like only the best questions or items float to the top, in fact, this is very biased towards the early submissions.
There is an interesting research study that came out in the February 3, 2005 issue of "Nature" about crowd behavior among animals. According to the study, large groups of animals can be affected by only a few members of that group with knowledge about the location of food, safe shelter, etc. In fact, the study goes on, the larger the group, the smaller the proportion of informed individuals are needed to make very accurate decisions. Those looking to turn to the "wisdom of the crowds" for answers should well understand this, and heed these pitfalls. A very small subset of a larger group can have a huge impact on the decision that group makes, while looking from the outside as an unbiased "wise crowd." The crowd we though was wise could simply be a dumb mob.
Perhaps there's a crowd somewhere that will help us all design better products designed to take advantage of crowd wisdom...
[1] By the way, I wanted to see what kind of an authority Wikipedia has become, so out of curiosity, I compared how many search results there were for ["according to Wikipedia"] vs. ["according to Encyclopedia Britannica"]. Just for good measure, I also compared it to ["according to God"]. The results:
- ["according to Wikipedia"] = 2,210,000 results
- ["according to Encyclopedia Britannica"] = 3,960 results
- ["according to God"] = 2,530,000 results
Not only does Wikipedia dwarf EB in references by 560X, Wikipedia's authority is closing in very fast on God's. Wow. Here's hoping THAT crowd is wise.
No comments:
Post a Comment