Thursday, June 02, 2011

Wisdom of crowds (or not)

Read this report by Brandon Keim in Wired Science and you can download a free pdf file of the full paper from here. Although it is not directly related to book publishing (or selling) it is a phenomenon that has puzzled me for decades.

In a paper titled How social influence can undermine the wisdom of crowd effect, (edited by Burton H. Singer, University of Florida), authors Jan Lorenz, Heiki Rauhut, Frank Sweitzer and Dirk Helbing found that, “Although groups are initially “wise,” knowledge about estimates of others narrows the diversity of opinions to such an extent that it undermines the wisdom of crowd effect in three different ways. The “social influence effect” diminishes the diversity of the crowd without improvements of its collective error. The “range reduction effect” moves the position of the truth to peripheral regions of the range of estimates so that the crowd becomes less reliable in providing expertise for external observers. The “confidence effect” boosts individuals’ confidence after convergence of their estimates despite lack of improved accuracy." The recent global financial crisis is given as an example.

In lay language: In the study, members of one group used their own mental capacity to arrive at answers, and they got it mostly right. In the other group, when members are allowed to be influenced by answers of others, the answers became skewed, less accurate.

The authors say, “When individuals become aware of the estimates of others, they may revise their own estimates for various reasons: People may suspect that others have better information, they may partially follow the wisdom of the crowd, there may be peer pressure toward conformity, or the group may engage in a process of deliberation about the facts.”

This is something I have wondered about for a long time. (It is also for this reason we do not promote self-help and management books, though we sell a few. Please read the primary sources and make up your own mind, we’d say.)

One only has to visit any social networking site to witness the phenomenon, the hysteria of misinformation, everyone trying to influence everyone else. Social networks are neither good nor bad, they merely mirror society. While the mathematical model is fascinating (and does agree with many of my lay observations), I am more interested in the ‘why’. Are we hardwired that way? Is it part of our survival or coping mechanism? Are we merely being manipulated? (My assumption here is that the truth is better than the lie, though I am not interested in a Socratic argument about it. Am I being biased?)

From Thesaurus online:

TRUTH: accuracy, actuality, authenticity, axiom, case, certainty, correctness, dope, exactitude, exactness, fact, facts, factualism, factuality, factualness, genuineness, gospel truth, gospel, honest truth, infallibility, inside track, legitimacy, maxim, naked truth, nitty-gritty, perfection, picture, plain talk, precision, principle, rectitude, rightness, scoop, score, trueness, truism, truthfulness, unvarnished truth, veracity, verisimilitude, verity, whole story

LIE: aspersion, backbiting, calumniation, calumny, deceit, deception, defamation, detraction, dishonesty, disinformation, distortion, evasion, fable, fabrication, falsehood, falseness, falsification, falsity, fib, fiction, forgery, fraudulence, guile, hyperbole, inaccuracy, invention, libel, mendacity, misrepresentation, misstatement, myth, obloquy, perjury, prevarication, revilement, reviling, slander, subterfuge, tale, tall story, vilification, white lie, whopper

Below are two short stories which may or may not be related but have made me sit up and think.

One. When I was in college (more than forty years ago) I witnessed an incident. I lived in Block E of the Fifth College (yes, they were not very creative in naming the blocks or colleges). It was custom (a strange one, admittedly) to jeer any visitor from another block. This was normally done in good humour. Once, however, it turned ugly. The visitor was initially jeered and called names, as usual, which he acknowledged with a smile and a wave, as usual, then someone threw something. Soon an entire mob descended on him and proceeded to beat the poor chap as he ran to safely. I looked on from the third floor, where my room was, in shock. I was helpless. Although I was a senior member of the block, there was nothing I could do. That was my first introduction to a mob.

Two. I was in Trengganu in the seventies when I heard about the death of a close friend. I only heard it this morning, I said when some friends met in Kuantan that evening. Then someone else said, he heard it the night before, and, so, started the combative conversation, louder and more insistent. Someone else insisted she had heard it the morning before that, until it reached a point when one person claimed she heard of the incident the day before the death actually occurred! I could only sit and admire. It was a combative conversation at its finest!

Raman