The US COVID-19 Trends and Impact Survey: Continuous real-time measurement of COVID-19 symptoms, risks, protective behaviors, testing, and vaccination

The US CTIS launched on April 6, 2020 and has run continuously since that time, with an average of more than 350,000 people participating each week over the first year of operation. The survey is implemented by the Delphi Group at CMU, with participants recruited via the Facebook platform. Every day, Facebook invites a new sample of active users ages 18 y or older to participate in the survey. Facebook uses stratified random sampling within US states to randomly select a sample of its users to see the survey invitation at the top of their News Feed. Users who click on the invitation are taken to the CMU-administered survey hosted on Qualtrics. To ensure privacy, Facebook does not see any individual survey response during or after the data collection. The survey is available in English, Spanish, Brazilian Portuguese, Vietnamese, French, and simplified Chinese.

The survey instrument was deployed in multiple waves from launch through April 5, 2021, with contents of each survey version summarized in Table 1. Revisions are ongoing as new public health needs arise. A number of core items have been included consistently across all survey versions, including questions about symptoms, contacts, and demographics. Key additions include items on mask wearing and occupation, added in September 2020, seasonal flu vaccination and schooling, added in November 2020, and COVID-19 vaccination, added in December 2020. As of April 5, 2021, the range of survey items spanned the following broad categories: household and individual symptoms, common comorbidities, contact patterns and mitigating behaviors, testing and diagnosis, worry and financial impact, schooling, vaccination, and demographics.

Full versions of all survey instruments can be found at https://cmu-delphi.github.io/delphi-epidata/symptom-survey/coding.html.

Analytic weights have been developed to adjust for differences between Facebook users and the United States population, and to adjust for biases related to coverage and nonresponse (8). When Facebook links users to the survey, it generates a random unique identifier that is passed to CMU. For users who complete the survey, CMU returns the corresponding identifiers to Facebook, which then calculates analytic weights in two steps:

The analytic weight does not identify the survey respondent. The weight for an individual is scaled to approximate the number of people in the adult population represented by that individual based on age, gender, location, and date. Facebook passes these weights to CMU. To protect respondent privacy, CMU cannot use these weights to identify specific Facebook users, and Facebook never receives individual survey responses and cannot link them to specific users.

In this study we examined a range of different outcomes measured in the CTIS over the period April 6, 2020 to April 5, 2021. We analyzed both aggregated macrolevel data and individual-level data to highlight some of the key questions that may be examined using CTIS. Across the examples presented in this paper, we have stratified analyses by individual characteristics, including age, race/ethnicity (using categories consistent with National Center for Health Statistics), and occupation, and by geographic divisions, including Census region, Census division, state, and county.

To examine the representativeness of the study sample, we compared characteristics of the sample to data from the American Community Survey 2019 (ACS) supplemental estimates.

We evaluated reported symptoms and symptom patterns in comparison to surveillance data on confirmed COVID-19 hospitalizations from the Department of Health and Human Services (9) and reported COVID-19 cases and mortality aggregated by the Johns Hopkins University Center for Systems Science and Engineering (10). To summarize relevant symptom patterns, we defined “COVID-like illness” (CLI) as reporting a fever of at least 100 °F, along with cough, shortness of breath, or difficulty breathing, in line with a working definition of CLI used for syndromic surveillance purposes beginning in early 2020. A second indicator, which we call “CLI in Community,” was based on responses to an item on the survey that asks whether respondents know someone personally in their community who is ill with COVID-like symptoms. We also compared reported diagnoses in CTIS to cumulative diagnoses in surveillance data.

To illustrate the utility of individual-level data to provide detailed information on characteristics that may be relevant to transmission risk, we compared reported diagnoses and reports of working outside of the home while symptomatic across occupational categories. To evaluate time trends in risk exposures and mitigating behaviors, we examined reported contacts, mask use, and use of public transit over time, across a range of stratifying variables, including counties grouped by levels on the Centers for Disease Control and Prevention (CDC) Social Vulnerability Index (SVI) (11), which is a composite measure constructed based on 15 social variables measured at the census tract level, age groups, and Census regions.

Finally, as an illustration of the information value of the survey at various levels of granularity, we assessed reported COVID-19 vaccination intent stratified by individual characteristics and across counties.

The study was approved by the CMU Institutional Review Board, under protocol STUDY2020_00000162. All respondents gave informed consent before participating in the survey.

As of April 5, 2021, a total of 20.2 million responses had been collected in the US CTIS. Table 2 summarizes characteristics of the survey respondents. Compared to the weighted sample, the unweighted sample had a higher proportion of women (66% vs. 52%) and a slightly higher proportion of respondents between ages 25 and 64 y (72% vs. 68%). Household size and prevalence of at least one comorbidity were similar in the unweighted and weighted samples.

Compared to 2019 ACS supplemental estimates, the weighted survey sample slightly overrepresented women, but had a broadly comparable age distribution and matched the ACS distribution across geographic regions. The weighted sample included a larger proportion of respondents with greater than a high school education, and a much smaller proportion with less than a high school education, suggesting the presence of a sampling or response bias correlated with education. This bias has remained consistent over time. As the weights provided by Facebook do not account for education, the weighting did not correct this bias.

A large fraction of daily respondents reported new or unusual symptoms consistent with COVID-19 (Fig. 1). The most common single new or unusual symptom among all respondents was “tiredness or exhaustion,” with a prevalence of 3.9% over Waves 4 through 10. Patterns of symptoms were notably different among the subset of respondents who reported testing positive for COVID-19, compared to all other respondents, including a substantially higher probability of reporting loss of smell or taste (34% compared to 1.2%).

Fig. 2 compares time series for three indicators from the CTIS (reported anosmia, CLI, and CLI in community) against the three main surveillance indicators that have been used to monitor trends in the epidemic (confirmed cases, hospitalizations, and deaths) stratified by Census region. Over the period April 6, 2020 to April 5, 2021, the three survey indicators tracked both broad temporal trends and regional patterns in the surveillance indicators, and several notable features are evident in the comparison. First, the survey-based indicators were less susceptible to the daily fluctuations and reporting anomalies that appeared in cases and deaths, including abrupt discontinuities around certain holiday periods. Second, trends and patterns in anosmia were similar to patterns in CLI, and the anosmia series provided a closer match than the other two survey indicators to the trends and patterns observed in COVID-19 hospitalizations, mirroring temporal peaks and comparative levels across regions over different waves of the epidemic. Third, CLI in community provided the most temporally stable signals while also expressing broad differences over time and space that were generally similar to those in other indicators. In a companion paper, we performed extensive correlation analyses between reported cases and various auxiliary indicators, including the survey-based CLI and CLI in community signals (12), showing strong correlations between cases and these two survey signals during much of the pandemic.

The CTIS includes questions about testing and diagnosis, which since September 8, 2020 have been asked of all respondents. Fig. 3 compares weekly CTIS estimates of the proportion of adults reporting that they have ever had a positive test for COVID-19 against cumulative diagnoses from surveillance reports by state in the same week. State surveillance reports were adjusted, using 2019 population estimates and CDC line-level demographic data on confirmed COVID-19 cases, to produce estimated diagnosis rates among the state’s population over age 18 y. As of April 5, 2021, reported diagnoses in the survey ranged from 3.1% in Hawaii to 19% in Idaho, and the correlation between survey reported diagnoses and surveillance reports at the state level was 0.83, indicating strong convergent validity.

Since Wave 4, the CTIS has included questions about occupation, which can offer valuable insights into exposures among essential workers and also supply signals of where transmission may be concentrated. Using responses from January 2021, we examined the probability of reporting a positive COVID-19 test across different occupation categories, as well as the probability of reporting working outside the home while having symptoms consistent with COVID-19 (Table 3). Substantial heterogeneity appeared across broad groups of occupation. The large proportion of people reporting never having been tested indicates the limitations of passive surveillance. Combining questions on symptoms, testing, and working into a single indicator, we examined the fraction of people who reported both working outside the home and currently having atypical symptoms; results ranged from more than 15% for respondents in food preparation and serving related occupations, to 4% of those in arts, design, entertainment, sports, and media.

A core set of questions included since the launch of the survey has addressed contacts and preventive behaviors. The survey has been amended over time to augment these items, with the addition of questions on mask use and specific high-risk behaviors in September 2020. In the context of recurrent surges in COVID-19 around the country over the course of 2020 and 2021, these items have illuminated how contacts and mitigating behaviors can shift in response to changes in local COVID-19 risk, sometimes preceding policy changes. For example, Fig. 4 shows selected variables relating to contacts and preventive behaviors over the period September 2020 to April 2021. Responses indicate sharp increases in risk-reducing behaviors during November and December as cases surged—including reduced contacts, increased use of masks, and reduced use of public transit—followed by relaxation of mitigating behaviors over the period January to April 2021 as cases fell.

Individual-level data allow for geographically detailed analysis that can also be disaggregated by demographic features. In Fig. 4, results are stratified in three different ways to illustrate this: by quartiles of the CDC SVI, by age, and by US Census region. There were minimal differences across counties grouped by the SVI in reported mask use, moderately higher contacts among those living in more vulnerable communities, and substantially higher use of public transit in more vulnerable counties. The second row shows age differences, which indicate a pronounced gradient of higher risk mitigation among older respondents, especially with respect to reduced contacts. The third row describes regional patterns that vary across indicators, with higher contacts and lower mask use in the South and Midwest regions compared to the Northeast and West, but greater use of public transit in the Northeast and West compared to South and Midwest.

Since December 19, 2020, the CTIS has included questions on vaccination intent, and since January 6, 2021, the survey has asked about vaccination status. The combination of geographic and demographic resolution in the survey allows a uniquely detailed view on vaccination acceptance and hesitancy across different United States population groups. Fig. 5A displays results by age group, race/ethnicity, gender, and Census region, pointing to high levels of acceptance among older respondents in all categories, but lower and more variable results at younger ages. (Respondents may identify as nonbinary or self-describe their gender, but this group was typically too small to break out and report reliable hesitancy estimates by region.) Fig. 5B shows the percentage of respondents indicating that they would probably not or definitely not get vaccinated across United States counties, indicating regional patterns but also high variability across counties within a given state. As the vaccination campaign slows across the country, high-resolution information on vaccine acceptance can inform policies that aim to increase uptake toward the goal of high levels of population immunity against COVID-19.