Pressing questions in the study of psychological and behavioral diversity

Extreme biased sampling of research participants and the neglect of their cultural context are increasingly recognized as threats to the generalizability of much of what we know about human thought and behavior (1, 2). In addition to reinforcing narrow views of what it means to be human, these parochial research practices have also shaped the methodological core of the human sciences by favoring tasks that are tailored to the skills, motivations, and social expectations of a very rarefied set of humanity (3). Despite decades of calls for reform, there is little evidence that increasing awareness of this threat has led to changes in practice or publishing (2, 4).

This special issue stems from the Sackler Colloquium on “Pressing Questions in the Study of Psychological and Behavioral Diversity” (September 7–9, 2017) designed to address these issues. Building on discussions emerging from two National Science Foundation-funded workshops held in 2016 and 2017, the colloquium convened scholars from a wide range of disciplines who have conducted important research with diverse populations. The colloquium participants identified barriers that discourage researchers from harnessing the full breadth of human diversity for understanding human thought and behavior. These included concerns about current peer review practices, methodological narrowness, lack of researcher diversity, and the need for theoretical frameworks that are sensitive to cultural, social, and ecological variation. Participants also shared innovations and insights from their own research that can foster and inform future investigations of human diversity.

In this introduction, we highlight three threads that weave together the 10 papers in this special issue. The first thread documents the theoretical and practical payoffs of engaging with a broader range of participants (and the pitfalls of failing to do so). The second thread identifies barriers to pursuing such work and proposes potential solutions to overcome them. The third thread raises important questions about appropriate ways to conduct robust research with diverse populations. Although many articles often touch on more than a single issue, we introduce them in the special issue based on their affinity with each of these different threads.

The first set of papers illustrates theoretical corrections and insights that can emerge when researchers expand their view of humanity. For example, Majid et al. (5) challenge the long-held assumptions in Western thought that there is a universal hierarchy of senses by showing that speakers of 20 diverse worldwide languages encode a range of colors, shapes, sounds, textures, tastes, and smells in very different ways. While Majid et al. leverage massive worldwide variation to make their argument, other papers illustrate striking cultural differences that can arise within the same locality. In the western United States, Alcalá et al. (6) document how pairs of children of either indigenous-heritage Mexican immigrants or of middle-class European American families worked together to devise efficient routes through a model store. The Mexican-heritage children tended to fluidly collaborate together in a sophisticated way seldom observed among middle-class European American children. This work points to the importance of careful observation in diverse communities to identify skills—in this case, fluidly collaborating together—that may be rare and unnoticed in default samples.

On the other side of the world in rural north India, Brooks et al. (7) show that low-caste respondents quickly learn to cooperate in repeated economic games, making them look a lot like both United States undergraduates and the kinds of rational actors assumed by many economists. However, in those same north Indian villages, a propensity for retaliation at perceived slights among high-caste respondents makes it very difficult for them to do the same. Thus, even a single village can represent a microcosm of culturally distinct patterns of behavior. While each of these studies is interesting in its own right, together they illustrate a recurring theme in the colloquium. A researcher who relies on just one of these groups to develop and vet a theory of human psychology would have a challenge determining what is basic, fundamental, or universal and what is rather particular to the cultural and social context in which it is being studied. Indeed, van Leeuwen et al. (8) demonstrate that this problem is not unique to humans by documenting reliably different patterns of sociality in four adjacent chimpanzee groups experiencing similar socioecological conditions.

The second set of papers shines a light on institutional disincentives and default assumptions that reinforce the status quo and discourage researchers from engaging with a broader range of humanity. Salari Rad et al. (9) focus on peer review and academic publishing by demonstrating continued bias toward English-speaking, European, and educated samples in one of the behavioral sciences’ leading journals, Psychological Science. The authors also document frequent omissions of key details about study samples and their culture context, a practice that has actually worsened with recent increases in samples recruited through the internet. As a potential remedy, Salari Rad et al. propose a number of guidelines for authors, reviewers, and editors aimed at improving attention to cultural context in conducting studies, reporting findings, and deciding which papers to accept for publication.

Brady et al. (10) focus on how the common tendency to overlook cultural context can lead to inaccurate generalizations, incorrectly viewing differences as deficiencies, dismissing non-WEIRD (Western, educated, industrialized, rich, and democratic) samples as outliers, and implementing interventions that hurt more than they help. As one solution to these myriad problems, Brady et al. encourage scholars to cultivate what they call “interpretive power”: the ability to understand and value individuals’ experiences and behavior in their cultural context. Using the specific case of attachment theory, Keller (11) argues that the universal application of a theory without regard to cultural differences can lead to unethical policies that unfairly and incorrectly diagnose deficiencies in child and parent behavior. Echoing a recurring call from many of the colloquium papers, Keller argues for the importance of embedded ethnography and first-hand knowledge for understanding the local rationale and norms underlying social interaction.

The third set of papers points to key tasks and challenges facing research communities that hope to build a robust and generalizable human science. Gurven (12) highlights the importance of developing theories that explain how widely varying social, cultural, and ecological contexts shape human psychology and behavior. Examining the failure of five-factor differentiation of personality to generalize to small-scale subsistence societies, he develops a theory of personality differentiation based on socio-ecological complexity that not only organizes the anomalies, but also provides a principled account for why we would expect five instead of two factors in WEIRD populations. More broadly, Gurven discusses how such theoretical frameworks are important for generating new questions and hypotheses, organizing emerging observations, and pointing researchers to the kinds of populations that would most fruitfully advance our understanding of human variation.

Turning to methodologies used by researchers to collect data from their fellow humans, Hruschka et al. (13) propose that attending to methodological “failures” can reveal researchers’ tacit (and often incorrect) assumptions about what diverse participants bring to any research protocol and what they consider relevant. The authors argue that systematic study of methodological failures and successful adaptations will help us go beyond our own intuitions and better understand the range of skills, motivations, and social expectations that respondents bring with them. Finally, Nzinga et al. (14) challenge the assumption that objectivity requires a distanced, uninvolved stance as the best way to reduce bias in research with communities, arguing instead that close engagement with communities also has distinct advantages. In addition, the authors document how this institutionalized assumption discourages nondominant researchers from studying their own communities, thereby reducing an important source of information about nondefault populations. To spur productive debate about this assumption, Nzinga et al. place distanced and close engagement on a level playing field and outline candidate risks and benefits associated each of these approaches.

Faced with continued, unreflective reliance on a narrow slice of humanity to inform the human sciences, the colloquium papers and the discussions inspired by them nonetheless give reason for hope. In addition to revealing the promise of reaching out to broader populations, the papers also chart pathways forward for building a more robust social and behavioral science. If recent editorial statements at prominent psychology journals are an indication of future commitments, then there is also reason to believe that these insights are beginning to guide decisions at key points in the scientific process (15, 16).

One hallmark of good science is the recognition that humans hold myriad biases and false beliefs that obscure their view of the world. This realization has led to the cultivation of institutions and modes of inquiry that help us identify and overcome our biases and in turn build models of the world that let us predict and change our futures in unprecedented ways. But the fight against bias is never complete and is bolstered by sustained reflection on how we generate knowledge. Indeed, Salari Rad et al. (9) propose that it may be useful to learn more about the lay beliefs that perpetuate unreflective reliance on narrow samples and neglect of cultural context.

Toward that end, we conclude by outlining candidate problematic lay beliefs of researchers and research institutions that were discussed at the colloquium and in the papers (Table 1). Some of these are still open for debate, while other beliefs already have strong evidence against them. For each of them, we suggest a necessary shift in scientific assumptions and practices in the case that the belief is wrong. These lay beliefs and their implications for how we do science deserve careful attention as we work toward building a robust science of Homo sapiens.