Overcoming the Folly of the Crowd

In a previous post we looked at excitement - hype even - about the potential for Web 2.0, in particular what is known as the Wisdom of the Crowd. But majority does not equal truth. Voting is good for social decision making but not for advancing knowledge. It isn’t even a majority anyway that fills the forums and blogs and wikis. It is the voice of a fanatical few. And the Crowd equates fame with wisdom. The Crowd also fails to discriminate between sources of information. The message here is to avoid putting the Crowd on a pedestal as some font of wisdom. It is an unreliable unpredictable superstitious mosh-pit of ideas and data: potentially fruitful if managed; potentially dangerous if idealised and idolised.

Scale

The key is to manage the Crowd, and the main thing to watch is scale. On a large scale like Wikipedia, the Crowd does average out to something approaching a stable depiction of accurate information. An excellent example is the entry on homeopathy which had a wild ride before settling down to an objective rational entry, though debate remains brisk and may never end, because the Crowd is an incubator for ideas good and bad, scientific and superstitious.

The idea of community knowledge becomes more problematic as the Crowd gets smaller. If we look at a niche community such as ITIL practitioners - closer to home for readers of this article - then even the Wikipedia entry becomes the work of a handful of people, and specialist wikis are either the work of their original founders or die from apathy.

Bring the community down to the even smaller population of the staff of an organisation and the whole idea of the Wisdom of the Crowd starts to fall apart.
Wiki-style help or documentation systems work only with the energy of a responsible owner and/or an enthusiastic few. They tend to be patchy, difficult to navigate, rapidly outdated and eventually fall into dusty disuse (like just about every knowledge repository within organisations).

Discussion mechanisms are mildly useful in a geographically-dispersed company, but email serves just as well. More importantly, email doesn’t get displaced by the forums, so discussion tends to get fragmented between the mechanisms. In theory a forum is an open discussion and email a closed one, but in all but the largest and most dispersed organisations the practical difference is moot. The forum mechanism seems to provide a useful long-term record of what was discussed, but who will find it? Only those who remember what was said, or a lucky few who have the time and tools (e.g. Google Desktop) to be searching it. Open archiving of the emails would serve just as well.

Moving back up in scale to the internet, forums do provide value, and the most extraordinarily useful titbits turn up on Google. Unfortunately once again these need to be treated with extreme caution and treated only as suggestions because they are subject to even less review and debate than wiki entries.

The forum equivalent of Wikipedia is probably Yahoo! Answers, and even with the largest possible Crowd this provides information of mixed quality. The factor at work is again scale, and more so with forums. The forum world is highly fragmented and each Crowd is small. Taking ITIL as an example again, most ITIL-oriented forums are moribund. Of the few active ones, most are full of newbies asking inane questions and self-appointed experts providing questionable answers. Only the forum owned by the itSMF has middling activity level and attracts authorative participants, precisely because it is not owned or moderated by the Crowd.

Filtering

Even in the best of cases, the useful information is still polluted by misinformation. So the second consideration after scale is filtering. On the internet there is sometimes filtering in the form of moderation. There is occasionally filtering in the form of an expert review panel. The vast majority of websites have no filtering other than the discriminating ability of the reader, and sadly in the post-modernist world critical faculties are being destroyed not developed.

Within an organisation, there can be much better filtering. Staff are accountable for what they say, and anonymity may not be permitted. Resources can be assigned to review and quality-assure the information. In some cases an open forum is desired – to stimulate debate and creativity – but mostly the accuracy and reliability of the information is paramount. The wisdom of the organisational Crowd is only as good as the people who own and manage it.

In summary, the Crowd does gather data. That data gets turned into information if it can be searched adequately. That information tends towards useful reliable knowledge if it comes from a large enough community and is subject to sufficiently robust filtering. In rare cases the Crowd may accidentally produce wisdom, but only if the reader has the capacity to separate it from the dross.