Testing the effects of Facebook usage in an ethnically polarized setting

In addition to testing our expectations around the effect of social media on outgroup regard, our research design also allows us to test the robustness of two key findings—that Facebook deactivation improved subjective wellbeing and decreased knowledge of current events (hereafter “news knowledge”)—from a recent Facebook deactivation study, which, as noted previously, was not conducted in a postconflict society but rather in the United States (2). We consider these two outcomes as separate from one another and, as such, we derive our expectations of the effect of deactivation for each of the outcomes independently.

While there is research suggesting that social media can reduce users’ levels of psychological distress by enabling them to stay in contact with their extended families, receive more social support, or access health information (41), a growing literature reports the negative psychological impact of the time spent on social media platforms. Such negative impact is attributed to the features of social media platforms that can encourage social comparisons, induce addictive behavior, or enable cyberbullying (42, 43). In preregistering our own hypothesis, we drew most heavily from the largest-scale causal evidence to date, which found that a month-long Facebook deactivation led to small but significant improvements in wellbeing among the US users (2). We did not expect that the postconflict nature of the context would lead to a different direction for the effect of Facebook deactivation, although it seemed plausible that the effect size, especially as it relates to anxiety levels, would be stronger given the emotionally charged nature of the commemorative period in which we conducted the study:

In addition to subjective wellbeing, we test the effect of Facebook deactivation on news knowledge. In the previously referenced Facebook deprivation study that took place in the United States (2), the authors found that deactivation reduced participants’ news knowledge index by 0.19 SD and speculated that the magnitude of the effect might differ, depending on the duration of the time spent deactivated from Facebook. We saw no reason why we would expect this dynamic to play out differently within a postconflict society. In an attempt to evaluate the external validity of that finding, we therefore test the following hypothesis:

We also preregistered a test of the impact of deactivation on a measure of political disaffection. We find suggestive evidence that the treatment of deactivation led to an increase in the level of political disaffection, i.e., negative affinity toward politics, which we operationalize as a composite index capturing levels of apathy, skepticism, and cynicism. This is yet another area in which exposure to social media may be affecting users and have important sociopolitical implications. We do not, however, focus on this outcome in the main text due to our lower confidence in the measurement of some of the indicators, but provide full results, as well as an explanation for our lower levels of confidence in this measure, within SI Appendix, section 14.

Our study focuses on Bosnia and Herzegovina, the most ethnically diverse of the former Yugoslav republics, and a site of a devastating war during the period from 1991 to 1995. Following the signing of the Dayton Peace Agreement, the country was divided into two entities, with about 51% of the territory granted to the Federation of BiH (predominantly Bosniak and Croat) and 49% to Republika Srpska (predominantly Serb). The Agreement also enforced a complex power-sharing system between the three ethnic groups, including a tripartite Presidency. More than two decades later, ethnonationalism continues to dominate the political arena. According to the latest 2013 census, Bosniaks make up 50.11% of the population, followed by Serbs at 30.78% and Croats at 15.43%, and finally 2.73% categorized as others.

Our treatment of Facebook deactivation took place from 7 July until 14 July 2019, encompassing the week around the Remembrance Day (11 July) of the genocide in Srebrenica. The genocide in Srebrenica is central and deeply intertwined with the memory of the Bosnian war as the worst atrocity in Europe after World War II. Online and offline discussions related to the war are held to a certain extent throughout the year, but significantly intensify during this period. The central event includes a mass funeral in Srebrenica for the victims’ remnants identified over the previous 12 mo, but there has also been an effort to provide digital content in the form of archives, museums, data collection, and journalistic accounts aimed at preservation of collective memories (44). These efforts are particularly valuable as a means of confronting a growing trend of genocide denialism. Despite the fact that the International Criminal Court of Justice (ICJ), among other institutions, ruled that the acts against the Bosniak Muslims committed in Srebrenica were acts of genocide under the international law (45), significant numbers of Republika Srpska officials and ethnic entrepreneurs continue to reject such rulings. The commemoration period serves as a natural prime of ethnic identities and, more generally, a proxy for periods during which identities (in this case, ethnic) are made salient and more fervently discussed.

Participants were recruited through Facebook advertisements, which we ran across BiH (SI Appendix, Fig. S1B) in both Cyrillic and Latin alphabets. Importantly, there is no language comprehension barrier between the members of three main ethnic groups in BiH. Prior to initiating this research, we obtained the approval of the New York University Institutional Review Board (protocol IRB-FY2019-3042). Participants were provided with a consent language as part of the email confirming their participation and were asked to respond to the email stating clearly that they agreed to it and chose to take part in the study. Out of the individuals who successfully completed the baseline survey, we selected only those who reported following at least one BiH news or political page on Facebook and reported being never or almost never on Twitter nor Reddit. We chose these criteria to increase the likelihood of our sample being exposed to political or intergroup content, as well as to reduce the extent to which the discussions on other social media platforms (Twitter or Reddit) might be influencing our outcomes of interest. Regarding the former, it should be noted that Facebook is by far the largest social media platform in BiH, encompassing 99.02% of social media market share, compared to Twitter and Reddit with 0.37% and 0.01% of the market share, respectively (46). Hence, active Twitter or Reddit users tend to be a more distinctive group and, by excluding them, we focus our attention on an average Facebook user responding to our advertisement.^† We block randomized the 556 resulting individuals into treatment and control, blocking on the variable from the baseline survey that measures the importance of ethnic identity to one’s self-identification. A total of 258 of individuals assigned to the control group and 263 of those assigned to the treatment group were successfully emailed. After we informed them of their assigned treatment, 92 from the treatment group and 64 from the control group either never responded or responded too late to be included as participants. Of those who did confirm their participation after receiving their group assignment, 31 participants (15 from the treatment group and 16 from the control group) failed to complete the study. Since attrition may introduce bias if it is systematically related to the outcomes of interest, we analyze the wide range of baseline characteristics of users who did not finish the final survey. We do so by first using the sample of all users included in randomization, where we detected significantly higher attrition in the treatment group. We also present an attrition analysis with the subsample of users who attritted after initially providing an affirmative response to our invitation to participate in the study (SI Appendix, section C). In both cases, the characteristics of those who attritted after being assigned to deactivate versus those who attritted after being assigned to remain active are largely balanced. We also estimate the probability of attrition for each user and show that our findings are robust to the inclusion of this control in our key models. This strengthens our belief that, although more people attrit within the treatment group, they do so for reasons largely unrelated to our outcomes of interest, which limits the potential of attrition biasing the internal validity of our results. We cannot, however, eliminate the possibility that those who have not finished the endline survey systematically differ on an unobservable characteristic related to our outcome and thus caution readers to interpret the results with this caveat in mind.

We received endline surveys from 159 participants in the treatment group and 194 participants in the control group, with their observable characteristics balanced between the treatment and the control group (SI Appendix, Table S3). The only imbalance detected is on the self-reported weekly frequency of accessing Facebook, with participants in the control group reporting somewhat higher values in the baseline survey compared to the treated group. We control for this covariate in the main results (model 2), which does not substantially change the estimated treatment effects. The final sample, with descriptive statistics reported in SI Appendix, Tables S1–S3, consists primarily of participants who identified themselves as Bosniaks (58.92%), followed by those identifying as Serbs (15.71%) and Croats (6.52%). Finally, 13.03% of respondents chose to identify as Bosnians and 4.82% chose to not report their ethnic identification. Ethnic and religious identities largely overlap within the context of the three communities in BiH.

The deactivation was monitored via a Python script that automatically checked Facebook URLs twice a day and sent a report to researchers with the IDs of participants who remained active. In addition, we manually checked the URLs at random times throughout the day. All of the initial URLs were also transformed into numeric IDs, so that we would be able to detect activity from the profile even if a user changed the way the username is shown to the general public. Once deactivated, the group complied with deactivation at a rate that exceeded 98%. Those who did not abide by the treatment and activated their profile at some point during those 7 d were contacted to describe their reason for activation and provided with a final warning (email and text); if they did not deactivate within 1 h after the email was sent to them, they were excluded from the study. The control group was asked to continue using Facebook as they regularly would, while also sharing the information on the exact daily amount of time they spent online by sending screenshots of the report that Facebook creates for each user.^‡

Participants took a comprehensive survey at the end of the experiment, and we focus on questions within three families of outcomes: outgroup regard, subjective wellbeing, and news knowledge (full list of questions in SI Appendix, Table S4). All of the questions were preregistered; following our preanalysis plan, we present results both for the composite indexes and for each of their corresponding indicators.^§ With the following regression, we estimate the intent-to-treat effect of Facebook deactivation on our outcomes of interest:where $Y_i$ is an outcome, T = $\left\{\mathrm{0,1}\right\}$ an indicator of treatment assignment, and $X_i$ a vector of covariates for the individual. Full covariate specification includes the following controls captured in the pretreatment survey: gender, age, employment status, ethnicity, weekly frequency of Facebook usage, and the perceived importance of country and ethnic membership to one’s identity. When incorporating covariates and in an effort to improve consistency of the estimated effect, we include mean-centered covariates and interact them with the treatment indicator (47, 48).

We first present our results on the impact of Facebook deactivation on subjective wellbeing and news knowledge, after which we present the effect that deactivation had on users’ interethnic attitudes.

In this section, we move beyond our original preregistered study in an effort to shed some light on our unexpected findings: Why might it be the case that moving off of Facebook at the time of a genocide commemoration led to higher levels of outgroup animosity? While a full consideration of this topic would require new preregistered studies to address this research question specifically, for now we use data from our original experiment—as well as some additional data we collected expressly for this purpose—to provide an exploration of the impact of one potential explanation: differences in the composition of one’s offline and online networks, especially as it relates to the degree of exposure to the outgroup.

As our starting point, we acknowledge that, in interpreting the role of any media on shaping social dynamics, focusing solely on direct media exposure is insufficient without taking into account the alternative activities that such exposure is crowding out (i.e., direct versus substitution effects) (51, 52). When asked how they spent their newly freed-up time, the treated group (deactivated from Facebook) reported spending more time than usual with friends and family as the most frequent response (Fig. 2); further analysis revealed that the treatment of Facebook deactivation increased the amount of time users spent with friends and family, as well as the amount of time spent reading the news on other online sources, at P < 0.06 significance level (SI Appendix, Fig. S3).

Given the commemorative period during which this study was conducted, it is important to note that people exhibit a tendency to share emotional experiences (53) and discuss contentious events, such as wartime-related topics and politics, with those with whom they feel closest. Scholars of social psychology explain how experiences of collective trauma and suffering feed back into the present, with narratives of family members often cited as the most powerful influences in transmitting and forming awareness of group victimization (54). Moreover, some scholars (55, 56) have suggested that vulnerability to echo chambers may be greatest in offline social networks and point out that we have lost sight of the fact that offline social networks are oftentimes even more homogeneous than the online networks. Previous research conducted in the context of a violent conflict also illustrates that in-person social interactions within homogeneous areas may in fact increase discriminatory outgroup behavior (57).

Even those who experience no direct contact with the outgroup may benefit from living within a diverse setting with ingroup members who do partake in such contact. This contextual effect can translate to online platforms as well, given that social media provides easier access to the extended networks of users’ online connections. Indeed, affordances of Facebook, particularly in settings with no language barrier, are conducive to extended (58) and vicarious contact (59, 60), where the former includes observing intergroup interactions and the latter knowing that an ingroup member befriended an outgroup member. Several studies find the positive effect of such contact on intergroup relations, although many of these studies are cross-sectional and warrant stronger empirical evidence. Our treatment of social media deactivation that reduced, if not entirely eliminated, participants’ contact with the outgroup may have made some users more dependent on offline echo chambers and moved them further away from the opportunity to engage with individual voices, as opposed to solely having the effect of reducing exposure to online divisive rhetoric. Drawing on insights from contact theory, we could therefore expect that the treatment of deactivation might have a more negative effect on outgroup regard on those who have no contact with the outgroup as part of their offline experiences either.

To explore whether this implication holds in the context of our study, we performed a series of analyses that were not explicitly part of our original preregistration and thus should be interpreted with that caveat in mind.^# To proxy for users’ offline networks, we use the latest Bosnia and Herzegovina census data (2013) on ethnic heterogeneities of BiH localities. We believe this proxy is reasonable as prior research has shown that county-level political and racial heterogeneity can help predict individual-level heterogeneity of political discussions, which we posit could hold for ethnic heterogeneity as well (61). Our conceptualization of offline networks goes beyond friends and family of users to include the composition of individuals with whom users might come into contact, both in the workplace and in daily activities. Measuring diversity is complex, with a variety of indexes employed, depending on the theoretical questions of interest. We test and check the robustness of our insights using three indexes with which ethnic heterogeneity can be quantified: an index of ethnolinguistic fractionalization, calculated using the Herfindahl concentration index, which is a widely employed measure of ethnic diversity; the share of the majority group, a measure that takes into account the size of the largest ethnic group in a particular town following the information from the latest census; and the Shannon diversity index, a mathematical measure most frequently used to characterize species diversity within a community (formulas and detailed descriptions of the indexes can be found in SI Appendix, section 10). For each of these three diversity indexes, we follow the same procedure. We first calculate the heterogeneity index of users’ networks and then subset the data into below and equal to or above the median of the index value, thus categorizing towns into a dichotomous more and less heterogeneous subcategory. We then rerun our original analysis within the two subsamples and evaluate whether the effect of deactivation on outgroup regard is more negative for users who live in more homogeneous communities. Indeed, this is exactly what we find using all three indexes of ethnic diversity (Fig. 3): Within the subsample of more homogeneous towns, the effect of deactivation on the composite index of outgroup regard is negative and highly statistically significant (SI Appendix, Tables S10–S12); on the contrary, for people living in more heterogeneous offline environments, there is no effect of deactivation on outgroup attitudes.

Hence, it appears that the negative effect of Facebook deactivation on outgroup regard is almost entirely driven by users within the communities in which the opportunities for offline intergroup contact are limited (and in some cases possibly nonexistent), and they therefore may be living within offline echo chambers that are stronger than the ones users find online. Although smaller sample size can lead to overestimating the magnitude of subgroup effects (62), we find consistent conclusions across different ways of measuring the effect. While the subgroup results in Fig. 3 rely on a binary indicator, the moderation analysis is robust to using a continuous measure of town heterogeneity (SI Appendix, Tables S16 and 17). Across all three methods of measuring diversity, the estimate on outgroup regard passes the Benjamini–Hochberg multiple-comparisons correction at the 0.05 level (SI Appendix, Tables S13–15).

For the heterogeneity of online networks, we invited interested participants to download and share with us the list of friends with whom they are connected on Facebook. We gathered this information from 134 participants, which constitutes 38% of our initial sample.^∥We simultaneously created a comprehensive database of names categorized by likely ethnicity, which enabled us to obtain percentages of Bosniak, Serb, and Croat names in the networks of each of our users, as identified from the three categories of names in our database. On average, we were able to estimate the ethnicity of 74.9% of the names within the online network of each user. Manual inspection of the other names suggests that most unmatched names mainly belong to one of the three categories: non-Bosnian names (possibly friends from other countries), nicknames (unique combinations of letters), and nonhuman accounts (coffee shops, artists, and locations.) Following the same steps as for the analysis of offline networks, the direction of the estimates suggests that the effect of deactivation on outgroup regard (SI Appendix, Tables S19 and S20) is more negative for users with more heterogeneous online networks (which, as expected, contrasts with the effect becoming more negative for users with more homogeneous offline networks, as previously described). Although the estimated interaction effect between the heterogeneity of one’s online network and the deactivation treatment is in the expected direction, the standard errors are too large to draw firm conclusions. It should be noted that this measure captures only users’ direct outgroup friends and is likely an underestimation of the total exposure to the outgroup as it overlooks exposure through one’s extended networks (friends of friends). Indeed, there is some evidence suggesting that such exposure may be substantial within the Balkan region. In 2016, Facebook introduced the first comprehensive measure of social connectedness across the world, constructing a dataset of aggregated information of friendship links between all Facebook users (63). Researchers have found that the level of connectedness is higher between regions who formed part of the same country in the past and highlight some regions in the former Yugoslavia as being several times more connected compared to similar pairs of other European regions (63).

We also use the available data to test whether those users whose online networks are more diverse than their offline networks are more negatively affected by the treatment of deactivation. Again, this analysis should be considered exploratory in light of the fact that it was not preregistered, yet we believe it suggests an important difference in the way Facebook deactivation affects people whose online and offline network diversity metrics differ. Using the same measures—ethnic fractionalization, Shannon entropy, and majority group share—we estimate the diversity of users’ online networks and create a binary variable that indicates users whose online networks are more diverse than their offline ones.** The interaction term between the treatment and this indicator reaches statistical significance with outgroup regard formed as a sum score and diversity measured with any of the three indexes (SI Appendix, Fig. S6 and Tables S21 and S22), while confidence intervals are wider when the index of outgroup regard is created as a principal component score. Across the two operationalizations, however, the direction of the estimates suggests that deactivating from social media had a comparatively more negative effect on outgroup attitudes for users whose online networks are more diverse than their offline ones. It is important to note that our results are aligned with two possibilities: that those who stayed active on Facebook experienced an improvement of outgroup attitudes because of their online contact and discussions engendered by the remembrance period or, alternatively, that those who deactivated their Facebook accounts experienced worsening of their outgroup attitudes because they were primarily exposed to discussions within homogeneous offline networks or the official discourse.

Theoretically, there is a potential, alternative, nonnetwork-based, explanation for our findings: that Facebook content about the commemoration promoted reconciliation, in which case the results we observe could be a product of the positive skew in the news content on the platform, irrespective of the homogeneity of either one’s offline or one’s online networks. To gauge the type of political content that was circulated during this period, we conduct a qualitative content analysis of the top five Facebook news pages in terms of the number of followers and—when these two categories do not overlap—Facebook pages of the top five news outlets in terms of the average number of website visitors per month. We find little to no evidence of unifying campaigns or promoted content and, in fact, find the contrary to be the case in our sample: Most articles are emphasizing group divisions (for more, see SI Appendix, section 1). The same, if not more, is true in the case of print media and public broadcasters, where, despite the existence of multiple media sources, pluralistic views remain rare (64). Of course, it is possible that those in the deactivated treatment group could have accessed different media articles about the commemoration through means other than Facebook. This, however, does not explain the observed heterogeneity in the effects depending on the composition of users’ networks. Moreover, media outlets increasingly rely on mobile resources to share the news and distribute information through social networks, with Facebook being the most frequent option (64). As such, many news outlets tend to promote their content through both their official websites and Facebook. Since a user could presumably access similar, if not the same, news in print, through a website or distributed via social media, we argue that it is the experience of engaging with such content that makes social media uniquely different, given the opportunity to participate in discussions, as well as to observe one’s social network and larger community engage with the online content.

Restoring social capital after conflict in societies with tenuous interethnic relations, as well as maintaining constructive interethnic relationships within a multiethnic society, is a complex and ever-evolving process. The rise of digital technologies and the increase in social media penetration across the world have for many changed the environment within which group processes unfold, yet we still lack rigorous evidence about the consequences of these developments. Going beyond the US context and partisan attitudes, we provide causal evidence of deactivation from Facebook leading to an unexpected decrease in users’ regard toward ethnic outgroups. In addition, we find preliminary evidence suggesting that such impact is conditional on the degree of ethnic homogeneity of one’s local community. We also find that recent findings from the United States (2) of a positive effect of social media deprivation on subjective wellbeing and a negative effect on levels of news knowledge generalize beyond US Facebook users. The uniqueness of the context in which our study was conducted, in terms of BiH’s history and intergroup relations, makes it an important test case for assessing the effect of social media usage on intergroup relations, but also requires that the findings be interpreted with the characteristics of the environment in mind. For now, our findings should be interpreted as being limited to a postconflict area that is characterized by varying levels of spatial group segregation and no language comprehension barrier between the members of main ethnic groups, as well as during an emotionally and politically charged period. Moreover, those actively using Facebook to propagate hatred and extremist views, of whom there are many, would possibly not show interest in participating in a study led by a US-based university and may thus not be fully captured within our sample. As such, our results do not speak to all of the possible effects that Facebook could have even in BiH and certainly not to all postconflict and ethnically polarized societies. Isolating the ways in which the features of our research design shaped the direction of our main effect would be a valuable next step in advancing understanding of social media’s impact on outgroup attitudes.

Our study is, of course, not without limitations. Studies with a smaller sample size, like ours, may overestimate the magnitude of the true effect (62). Given the lack of social media deprivation studies that—like ours—focus on interethnic regard, we hope to see larger-scale replications in the future to reach a more precise estimate of the true effect size. Such replications could also allow for a more robust exploration of heterogeneous treatment effects for which this study is underpowered. We also encourage future research to compare the diversity of online versus offline networks, which would better shed light on the mechanisms underlying the relationship between the time spent on social media and group attitudes. In particular, additional measures of networks that go beyond the experiences of direct contact would allow for a finer-grained analysis of the differences in outcomes depending on the composition of one’s network (although for now such studies on Facebook would likely require the cooperation of the company due to data access limitations for those outside of Facebook). Finally, as we deliberately set our study during a period around the Srebrenica genocide commemoration in which we expected emotions to be heightened and political discussions intensified, it remains to be seen how robust the effects we observed will be in a less emotionally charged period. Nevertheless, insufficient attention has been given to exploring the role that social media plays in enabling individuals to negotiate identity, preserve history, and cope with grief and societal trauma, a question to which we hope to contribute to answering in our future work.

Within the growing disappointments and valid concerns about the role of social media—many of which extend far beyond the scope of our paper—our findings do suggest that simply deactivating from social media is not a panacea to ethnic polarization, especially if the offline environment provides few to no opportunities for positive intergroup contact. Indeed, to answer the question of how time spent on social media affects users, scholars would do well to move away from social media determinism by paying attention to the contextual factors, alternative activities, and intergroup contact opportunities available to the individuals within the particular research context. How to continue doing so in a manner that can ultimately create a road map for strengthening of social capital through the tools of digital networks remains a crucial question in pushing this ever-important research agenda forward.

We are grateful to all members of the New York University (NYU) Center for Social Media and Politics and participants of the NYU Comparative Politics Workshop and Center for Social Media and Politics Election Seminar Series for excellent comments and suggestions. We thank Cyrus Samii, Gwyneth McClendon, Chris Dawes, Matthew Gentzkow, Andrew Guess, Kevin Munger, Riada Asimovic Akyol, Mikdat Yildirim, and Rajeshwari Majumdar for providing helpful feedback; Megan Brown for excellent technical support; and Edima Zejnilovic for invaluable expertise and assistance on the ground.