How social influence can undermine the wisdom of crowd effect

Edited by Burton H. Singer, University of Florida, Gainesville, FL, and approved April 13, 2011 (received for review June 23, 2010)
May 16, 2011
108 (22) 9020-9025

Abstract

Social groups can be remarkably smart and knowledgeable when their averaged judgements are compared with the judgements of individuals. Already Galton [Galton F (1907) Nature 75:7] found evidence that the median estimate of a group can be more accurate than estimates of experts. This wisdom of crowd effect was recently supported by examples from stock markets, political elections, and quiz shows [Surowiecki J (2004) The Wisdom of Crowds]. In contrast, we demonstrate by experimental evidence (N = 144) that even mild social influence can undermine the wisdom of crowd effect in simple estimation tasks. In the experiment, subjects could reconsider their response to factual questions after having received average or full information of the responses of other subjects. We compare subjects’ convergence of estimates and improvements in accuracy over five consecutive estimation periods with a control condition, in which no information about others’ responses was provided. Although groups are initially “wise,” knowledge about estimates of others narrows the diversity of opinions to such an extent that it undermines the wisdom of crowd effect in three different ways. The “social influence effect” diminishes the diversity of the crowd without improvements of its collective error. The “range reduction effect” moves the position of the truth to peripheral regions of the range of estimates so that the crowd becomes less reliable in providing expertise for external observers. The “confidence effect” boosts individuals’ confidence after convergence of their estimates despite lack of improved accuracy. Examples of the revealed mechanism range from misled elites to the recent global financial crisis.

Continue Reading

Acknowledgments

This paper benefited from comments made by Stefan Herzog, Michael Mäs, Ryan O. Murphy, two anonymous reviewers, and research assistance by Hanna Thorn and Silvana Jud. This study was supported by funding from Eidgenössische Technische Hochschule Zürich. Preliminary versions of the experiment were designed and conducted by J.L. in 2006 (36) and 2005.

Supporting Information

Appendix (PDF)
Supporting Information
sd01.xls

References

1
F Galton, Vox populi. Nature 75, 7 (1907).
2
I Lorge, D Fox, J Davitz, M Brenner, A survey of studies contrasting the quality of group performance and individual performance, 1920-1957. Psychol Bull 55, 337–372 (1958).
3
C Hommes, J Sonnenmans, J Tuinstra, H van de Velden, Coordination of expectations in asset pricing experiments. Rev Financial Stud 18, 955–980 (2005).
4
I Yaniv, M Milyavsky, Using advice from multiple sources to revise and improve judgments. Organization Behav Hum Decis Process 103, 104–120 (2007).
5
J Surowiecki The Wisdom of Crowds: Why the Many Are Smarter than the Few and How Collective Wisdom Shapes Business, Economies, Societies, and Nations (Doubleday Books, New York, 2004).
6
E Vul, H Pashler, Measuring the crowd within: Probabilistic representations within individuals. Psychol Sci 19, 645–647 (2008).
7
SM Herzog, R Hertwig, The wisdom of many in one mind: Improving individual judgments with dialectical bootstrapping. Psychol Sci 20, 231–237 (2009).
8
H Rauhut, J Lorenz, The wisdom of crowds in one mind: How individuals can simulate the knowledge of diverse societies to reach better decisions. J Math Psychol 55, 191–197 (2010).
9
SE Page The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies (Princeton Univ Press, Princeton, NJ, 2007).
10
WA Mason, FR Conrey, ER Smith, Situating social influence processes: dynamic, multidirectional flows of influence within social networks. Pers Soc Psychol Rev 11, 279–300 (2007).
11
AV Banerjee, A simple model of herd behavior. Q J Econ 107, 797–817 (1992).
12
S Bikhchandani, D Hirshleifer, I Welch, A theory of fads, fashion, custom, and cultural change as informational cascades. J Polit Econ 100, 992–1026 (1992).
13
AE Mannes, Are we wise about the wisdom of crowds? the use of group judgments in belief revision. Manage Sci 55, 1267–1279 (2009).
14
GW Allport, The study of the undivided personality. J Abnorm Psychol Soc Psychol 19, 132–141 (1924).
15
SE Asch, Opinions and social pressure. Sci Am 193, 31–35 (1955).
16
HJ O'Gorman, The discovery of pluralistic ignorance: An ironic lesson. J Hist Behav Sci 22, 333–347 (1986).
17
DT Miller, C McFarland, When social comparison goes awry: The case of pluralistic ignorance. Social Comparison: Contemporary Theory and Research, eds J Suls, T Wills (Lawrence Erlbaum, Hillsdale, NJ), pp. 287–313 (1991).
18
RL Goldstone, TM Gureckis, Collective behavior. Topics Cogn Sci 1, 412–438 (2009).
19
MJ Salganik, PS Dodds, DJ Watts, Experimental study of inequality and unpredictability in an artificial cultural market. Science 311, 854–856 (2006).
20
A Tversky, D Kahneman, Judgment under uncertainty: Heuristics and biases. Science 185, 1124–1131 (1974).
21
S Plous The Psychology of Judgment and Decision Making (McGraw-Hill, New York, 1993).
22
RM Dawes, M Mulford, The false consensus effect and overconfidence: Flaws in judgment or flaws in how we study judgment? Organization Behav Hum Decis Process 65, 201–211 (1996).
23
D Lazer, A Friedman, The network structure of exploration and exploitation. Admin Sci Q 52, 667–694 (2007).
24
U Fischbacher, z-Tree: Zurich toolbox for ready-made economic experiments. Exp Econ 10, 171–178 (2007).
25
RH Hooker, Mean or median. Nature 75, 487 (1907).
26
F Galton, The ballot-box. Nature 75, 509–510 (1907).
27
C Genest, J Zidek, V Combining probability distributions: A critique and an annotated bibliography. Stat Sci 1, 114–135 (1986).
28
AP Dawid, et al., Coherent combination of experts’ opinions. TEST 4, 263–313 (1995).
29
KY Chen, LR Fine, BA Huberman, Eliminating public knowledge biases in information-aggregation mechanisms. Manage Sci 50, 983–994 (2004).
30
B Golub, MO Jackson, Naïve learning in social networks: Convergence, influence, and the wisdom of crowds. Am Econ J Microecon 2, 112–149 (2010).
31
S Dehaene, V Izard, E Spelke, P Pica, Log or linear? Distinct intuitions of the number scale in Western and Amazonian indigene cultures. Science 320, 1217–1220 (2008).
32
E Limpert, WA Stahel, M Abby, Log-normal distributions across the sciences: Keys and clues. Bioscience 51, 341–352 (2001).
33
RP Larrick, JB Soll, Intuitions about combining opinions: misappreciation of the averaging principle. Manage Sci 52, 111–127 (2006).
34
RD Sommerfeld, HJ Krambeck, M Milinski, Multiple gossip statements and their effect on reputation and trustworthiness. Proc Biol Sci 275, 2529–2536 (2008).
35
A Krogh, J Vedelsby, Neural network ensembles, cross validation, and active learning. Adv Neural Inform Process Syst 7, 231–238 (1995).
36
J Lorenz Repeated averaging and bounded confidence – modeling, analysis and simulation of continuous opinion dynamics. PhD thesis (Universität Bremen, Bremen, Germany, 2007).

Information & Authors

Information

Published in

Go to Proceedings of the National Academy of Sciences
Proceedings of the National Academy of Sciences
Vol. 108 | No. 22
May 31, 2011
PubMed: 21576485

Classifications

Submission history

Published online: May 16, 2011
Published in issue: May 31, 2011

Keywords

  1. collective judgment
  2. estimate aggregation
  3. experimental social science
  4. swarm intelligence
  5. overconfidence

Acknowledgments

This paper benefited from comments made by Stefan Herzog, Michael Mäs, Ryan O. Murphy, two anonymous reviewers, and research assistance by Hanna Thorn and Silvana Jud. This study was supported by funding from Eidgenössische Technische Hochschule Zürich. Preliminary versions of the experiment were designed and conducted by J.L. in 2006 (36) and 2005.

Notes

This article is a PNAS Direct Submission.
*In real-life situations with social influence, there may be additional effects, from which our experiment has abstracted: this includes competition, group pressure, and authority effects. For example, a criminologist could say: “I know the number of victims.” In contrast to such possibilities, our comparably mild and parsimonious kind of information feedback has the advantage that it enables a particularly controlled experimental setting in which there is little ambiguity about which kind of information feedback and social influence played a role.
Note that the framework of the diversity prediction theorem (9) can also be applied to logarithmically transformed data. For the case of logarithmically transformed data, the collective error of the logarithms is the logarithm of the geometric mean and one SD is the logarithm of the geometric SD. Considering the logarithmic nature of our data, one may argue that the geometric mean would have been a better design choice than the arithmetic mean for the information feedback in the aggregated information condition. However, this measure is hard to understand for most subjects because it necessitates confidence with logarithmic transformations. As the simple average (i.e., arithmetic mean) is known from daily life, this information is more meaningful for subjects. Hence, we decided for the arithmetic mean.
The empirical measurement of the social influence effect requires questions with moderate difficulty. In particular, subjects should not have a precise factual knowledge of an issue, because this would prevent adaption and social influence. We can empirically confirm that this was not the case for our questions and subjects: in only 1.5% of all cases, subjects responded at all five times in the most inner payment range of one particular question. This means in absolute values that 13 of 864 consecutive response runs were responded in the full payment range (144 subjects responded to six questions in a run of five consecutive responses). Three of these 13 “high-success runs” were performed by the same person and two from another person. All other high-success runs were performed by different persons.
§
It deserves to be mentioned that the initial diversity seems to be higher in the no information condition. It could be that subjects anticipate to feel uneasy if their published estimates are too distant from those of others. This could foster that their initial estimates tend to be more “conservative” in the conditions with information feedback. Interestingly, this discrepancy in initial variance is mainly caused by the questions about crime statistics and not about geographical facts.
Note that the collective error slightly declines under social influence, especially in the aggregated information condition, which is partially supported by the significance tests (SI Appendix). This is a result of two empirical facts. First, the distributions of estimates are right-skewed. As a consequence, the arithmetic mean is usually much larger than most estimates and also much larger than the true value. Second, it is an empirical fact for our choice of questions that the geometric mean (which is our aggregation measure to compute the collective error) is always slightly lower than the true value (Table 1). The mechanism of presenting the arithmetic mean in the aggregated condition thus triggers an upward drift toward the true value. This issue is interesting but deserves future studies, as this effect may be different for different sets of questions.

Authors

Affiliations

Chair of Systems Design, ETH Zürich, CH-8032 Zurich, Switzerland;
Heiko Rauhut2,1 [email protected]
Chair of Sociology, in particular of Modeling and Simulation, ETH Zürich, CH-8092 Zurich, Switzerland;
Frank Schweitzer
Chair of Systems Design, ETH Zürich, CH-8032 Zurich, Switzerland;
Dirk Helbing
Chair of Sociology, in particular of Modeling and Simulation, ETH Zürich, CH-8092 Zurich, Switzerland;
Santa Fe Institute, Santa Fe, NM 87501; and
Collegium Budapest–Institute for Advanced Study, 1014 Budapest, Hungary

Notes

2
To whom correspondence may be addressed. E-mail: [email protected] or [email protected].
Author contributions: J.L., H.R., F.S., and D.H. designed research; J.L. and H.R. performed research; J.L. and H.R. analyzed data; and J.L. and H.R. wrote the paper.
1
J.L. and H.R. contributed equally to this work.

Competing Interests

The authors declare no conflict of interest.

Metrics & Citations

Metrics

Note: The article usage is presented with a three- to four-day delay and will update daily once available. Due to ths delay, usage data will not appear immediately following publication. Citation information is sourced from Crossref Cited-by service.


Citation statements

Altmetrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited by

    Loading...

    View Options

    View options

    PDF format

    Download this article as a PDF file

    DOWNLOAD PDF

    Get Access

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Personal login Institutional Login

    Recommend to a librarian

    Recommend PNAS to a Librarian

    Purchase options

    Purchase this article to access the full text.

    Single Article Purchase

    How social influence can undermine the wisdom of crowd effect
    Proceedings of the National Academy of Sciences
    • Vol. 108
    • No. 22
    • pp. 8917-9316

    Media

    Figures

    Tables

    Other

    Share

    Share

    Share article link

    Share on social media