On the mis-presentation and misinterpretation of gender-related data: The case of Ingalhalikar’s human connectome study

In a PNAS paper on the human connectome, Ingalhalikar et al. conclude that their “results reveal fundamental sex differences in the structural architecture of the human brain” and link these differences to behavioral sex differences (1). Others have previously referred to the difficulties in linking sex differences in brain structure to sex differences in behavior (2). We would like to focus here on the selective choice of data that forms the basis for such strong conclusions. Specifically, Ingalhalikar et al. claim that they have found “fundamentally different connectivity patterns in males and females” (1), with males having greater intrahemispheric connectivity and females, greater interhemispheric connectivity. Indeed, inspection of figure 2A in ref. 1, which depicts over one brain the connections that were stronger in males compared with females, and over another brain the connections that were stronger in females compared with males, clearly gives this impression. However, does this impression present a loyal picture of the human connectome? Clearly figure 2 in ref. 1 does not represent reality, as in reality both males and females have both intra- and interhemispheric connections. The authors only found a difference between the strength of some of these connections—that is, a quantitative difference—yet they present it as a qualitative one. Moreover, Ingalhalikar et al. do not provide any data that can help the reader evaluate the size of the differences and therefore corroborate the authors’ conclusion that these differences are indeed fundamental. Specifically, the authors do not provide the number of connections (of the 95 × 95 assessed) that show a sex difference, so we do not know whether the two genders were mainly similar or mainly different. Nor do Ingalhalikar et al. provide any estimate of the size of these differences (such as Cohen’s d) that is needed to evaluate the extent of overlap between the distribution of the strength of these connections in males and in females. Such information is needed to determine whether the statistically significant differences are also meaningful. It should be stressed that for any given difference between means, there is a sample size for which this difference will be statistically significant. For example, in a behavioral study in which the imaging subjects were a subset (3), Cohen’s d as small as 0.02 (i.e., the difference between the means of males and females was 2% of the SD) were statistically significant because of the large sample size [∼3,500; the highest Cohen’s d in that study was 0.33 (3)].

In sum, instead of data that will enable the reader to judge whether the sex differences found are meaningful, the reader is left with figure 2 in ref. 1, which depicts in a dichotomous way only the connections whose strength was found to be different in males and females. No wonder the main message the reader is left with is of a “male brain” and a “female brain” that seem to have been taken from subjects from different galaxies, not just from different planets.