How social information can improve estimation accuracy in human groups

Previous works have shown that distributions of independent individual estimates are generally highly right-skewed, while distributions of their common logarithm are much more symmetric (12, 13, 18). This is because humans think in terms of orders of magnitude, especially when large quantities are involved, which makes the logarithmic scale more natural to represent human estimates (20). In these works, participants were mostly asked “easy” questions for which they had good prior knowledge (high demonstrability), such that the answers ranged over one to two orders of magnitude at most (12–14, 17–19, 21–23). To ensure that little information was present before the inclusion of our virtual experts and to more clearly identify the impact of social influence, we selected “hard” questions (low demonstrability). These questions involve very large quantities, and answers span several orders of magnitude, making the log transform of estimates even more relevant. To compare quantities that can differ by orders of magnitude, we normalize each estimate $E$ by the true answer $T$ to the question at hand and define the log-transformed estimate $X=\log (E/T)$. Note that the log transform of the actual answer $T$ is $X=0$.

Fig. 1A shows the distribution of $X$ before and after social information has been provided to the subjects (SI Appendix, Table S1). Although such distributions have often been presented as close to Gaussian distributions (13, 18), we find that they are much better described by Cauchy distributions because of their fat tails, which account for the nonnegligible probability of estimates extremely far from the truth. The Cauchy probability distribution function readswhere $m$ is the center/median and $\sigma$ is the width of the distribution. SI Appendix, Fig. S2A shows the distribution of estimates in the Japan experiment, and SI Appendix, Fig. S2B shows that, when the same questions were asked, distributions of personal estimates in France and Japan are almost identical.

For the Cauchy distribution, the mean and standard deviation (SD) are not defined. Therefore, good estimators of $m$ and $\sigma$ are the median and one-half the interquartile range (the difference between the third and the first quartiles) of the experimental distribution, respectively. In the following, $m_p$ ($m$) and ${\sigma }_p$ ($\sigma$) will refer to the median and one-half the interquartile range of the experimental distribution before social influence (after social influence), respectively.

Cauchy and Gaussian distributions belong to the so-called stable distributions family. More generally, $\{X_i\}$ being a set of estimates drawn from a symmetric probability distribution $f$ characterized by its center $m$ and width $\sigma$, we define the weighted average $X\prime ={\sum }_i{p}_i{X}_i$, with ${\sum }_i{p}_i=1$; $f$ is a stable distribution if $X\prime$ has the same probability distribution $f$ as the original $X_i$, up to the new width $\sigma \prime$. Indeed, the center $m$ remains the same because of the condition ${\sum }_i{p}_i=1$, but the width may decrease after averaging (law of large numbers), depending on the stable distribution $f$ considered. Cauchy and Gaussian represent two extremes of the stable distribution family, with Lévy distributions being intermediate cases: for the Cauchy distribution, the width $\sigma$ remains unchanged, whereas the narrowing of $\sigma$ is maximum for the Gaussian distribution (SI Appendix). In the case of actual human estimates, the relevance of a certain distribution $f$ can be related to the degree of prior knowledge of the group. When individuals have no idea about the answer to a question, the weighted average of arbitrary answers cannot be statistically better ($\sigma \prime <\sigma$) or worse ($\sigma \prime >\sigma$) than the arbitrary answers themselves, leading to a Cauchy distribution for these estimates (the only distribution for which $\sigma \prime =\sigma$). However, when there is a good prior knowledge, one expects that combining answers gives a better statistical estimate ($\sigma \prime <\sigma$; Gaussian). When the quantity to estimate is closely related to general intuition (ages, dates, etc.), estimates should hence follow a Gaussian-like distribution, while when individuals have very little knowledge about the answer, as in our experiment, estimates should be Cauchy-like distributed. The rationale for naturally observing stable distributions is explained in SI Appendix.

We use the term Cauchy-like, because Fig. 1A shows that the distributions of prior ($X_p$) and final ($X$) estimates are slightly skewed toward low estimates ($X<0$), reminiscent of the human cognitive bias to underestimate numbers, because of the nonlinear internal representation of quantities (24). As we will show, this phenomenon has strong implications on the influence of information provided to the group. We also observe a clear sharpening of the distribution of estimates after social influence mainly caused by the presence of the virtual experts, hence affecting the value of the social information $M=\log (I/T)$ and ultimately, the final estimate $X$ of the actual subjects. This sharpening becomes stronger as the percentage of experts increases (SI Appendix, Fig. S3).

Moreover, consistent with our introductory discussion of the measurement methods of group performance, we propose the two following indicators: (i) collective performance $|\text{median}(X_i)|$, which represents how close the center of the distribution is to zero (the log transform of the true value $T$), and (ii) collective accuracy $\text{median}(|X_i|)$, which is a measure of the proximity of individual estimates to the true value.

After having received social information, an individual $i$ may reconsider her personal estimate ${E_p}_i$. The natural way for humans to aggregate estimates is to use the median (22) or the geometric mean (18), which both tend to reduce the effect of outliers. Here, the social information that we provided to the subject was the geometric mean of the $\tau$ previous answers (including that of the virtual experts providing the true answer $E_i=T$): $I_i={({\prod }_{j=i-\tau }^{i-1}{E}_j)}^{1/\tau }$. Moreover, one can always represent the new estimate $E_i$ as the weighted geometric average of the personal estimate ${E_p}_i$ and the social information $I_i$. Hence, we can uniquely define the sensitivity to social influence $S_i$ by $E_i={{E_p}_i}^{1-S_i}{I_i}^{S_i}$. The value $S_i=0$ corresponds to subjects keeping their initial estimates, while $S_i=1$ corresponds to subjects adopting the estimate of their peers. In terms of log-transformed variables $X_i=\log (E_i/T)$, we obtainwhere the log-transformed social information is simply the arithmetic mean $M_i=(1/\tau ){\sum }_{j=i-\tau }^{i-1}{X}_j$, and thus, $S_i=(X_i-{X_p}_i)/(M_i-{X_p}_i)$. Note that, in this language, $S_i$ is simply the barycenter coordinate of the final estimate in terms of the initial personal estimate and the social information.

Fig. 1B shows that the experimental distribution of $S$ has a bell-shaped part that we roughly assimilate to a Gaussian, with two additional Dirac peaks exactly at $S=0$ and $S=1$ (SI Appendix, Table S2 shows the numerical values). Five types of behavioral responses can be identified: keeping one’s opinion (peak at $S=0$), adopting the group’s opinion (peak at $S=1$), making a compromise between one’s opinion and the group’s opinion ($0<S<1$), overreacting to social information ($S>1$), and contradicting it ($S<0$). Quite surprisingly, responses that consist of overreacting and contradicting were generally overlooked in previous works (21–23, 25), either considered as noise and simply not taken into account or sometimes included into the peaks at $S=0$ and $S=1$, despite these behaviors being not negligible (especially overreacting). We find that the median of $S$ is $0.34$, in agreement with previous results (15, 18, 25), meaning that individuals tend to give more weight to their own opinion than to information coming from others (14, 19). Moreover, the distributions of $S$ for the experiment performed in Japan and for men and women (in France) are very similar to that of Fig. 1B (SI Appendix, Fig. S4).

We find that the subjects’ behavioral reactions are highly consistent, reflecting robust differences in personality or general knowledge: in each session, according to the way that subjects modified their estimates on average in the first $24$ questions, we split the subjects into three subgroups. We first define “confident” subjects as the one-quarter of the group minimizing ${⟨|S_q|⟩}_q$, where $q$ is the index of the questions (i.e., the subjects who were on average closest to $S=0$), and the “followers” as the one-quarter of the group minimizing ${⟨|1-S_q|⟩}_q$ (i.e., closest to $S=1$). The other one-half of the group is defined as the “average” subjects. SI Appendix, Fig. S5 shows the distributions of $S$ for the three subgroups computed from questions 25–29. The differences are striking (SI Appendix, Fig. S6): for the group of confident subjects, the peak at $S=0$ is about seven times higher than the peak at $S=1$, while for the group of followers, it is less than twice larger. Moreover, the distribution for average subjects is found to be very close to the global distribution shown in Fig. 1B.

Fig. 2A shows that, on average, $S$ depends on the distance $D=\log (E_p/I)=X_p-M$ between personal and group estimates. Up to a threshold of $t\approx 2.5$ orders of magnitude, there is a linear cusp relation between $S$ and $D$. The farther away the social information $M$ is from a subject’s personal estimate $X_p$, the more likely the latter is to trust the group as $S$ increases. Fig. 2B shows the origin of this correlation: as social information gets farther from personal opinion, the probability to keep one’s opinion ($S=0$) decreases, while the probability to compromise increases. Interestingly, the adopting behavior does not change with $D$. The same phenomena have been observed in the Japan experiment (SI Appendix, Fig. S8).

We now introduce an individual-based model to understand the respective effects of individual sensitivity to social influence and information quality and quantity on collective performance and accuracy observed at the group level. In the model, we simulate a sequence of $20$ successive estimates performed by the agents (not including the virtual experts). A typical run of the model consists of the following steps for a given condition $(\rho ,\tau )$.

For all graphs, we ran 100,000 simulations, so that the model predictions error bars are negligible. Fig. 1B shows that the distribution of sensitivities to social influence $S$ obtained in the model (red curve in Fig. 1B) is similar by construction to the experimental one. Also, by construction of the model (step v above), the cusp dependence of the sensitivity to social influence with respect to $D=X_p-M$ is well-reproduced by the model (Fig. 2A, red curve with open symbols). We now address several nontrivial predictions of the model.