What grades and achievement tests measure

Achievement tests, like the Armed Forces Qualification Test (AFQT), are often used as proxies for cognitive ability (12–14). SI Appendix, Appendix S1 lists 50 papers that use AFQT scores as proxies for intelligence. Grades are also used as proxies for intelligence (1, 2).

Previous research studies relationships between IQ and personality*, between grades and IQ (a review of the literature is in ref. 18), and between personality and grades.^† Ref. 22 relates the High School Personality Questionnaire and the Culture Fair Intelligence Test to scores on standardized achievement tests and finds that conscientiousness and IQ predict scores on achievement tests. Ref. 23 surveys studies relating self-regulation and scores on standardized achievement tests, course grades, and high school achievement. It shows that self-regulation is more predictive of course grades than scores on standardized achievement tests and suggests that this may be the reason why course grades are more predictive of certain later-life outcomes than achievement tests. Ref. 24 reports that both self-discipline and IQ predict performance on achievement tests. Ref. 25 reports that self-control (a facet of Big Five conscientiousness) and IQ (measured by Raven Matrices) predict scores on the English/language arts and mathematics standardized achievement tests. Our analysis builds on and extends this research by analyzing the effects of cognition and personality on grades, achievement tests, and a variety of important life outcomes. We report results from samples pooled across genders.

Table 1 summarizes the availability of measures in the four datasets that we analyze.^‡ Although details and point estimates vary and some data contain only partial information, consistent patterns emerge across all four datasets.

Stella Maris is a Dutch high school at which we collected Raven’s IQ, scores on achievement tests [the Differential Aptitude Test (DAT)], grades, and measures of personality. For this sample, we have no measure of adult outcomes. The British Cohort Study (BCS) followed a cohort of children born in one week in April of 1970 until 2016. It has information on grades, IQ, scores on achievement tests, personality, and a variety of adult life outcomes. The National Longitudinal Survey of Youth 1979 (NLSY79) sampled American children aged 14–21 y old in 1979 and followed them ever since that time. It has an achievement test (the AFQT) and scores on different IQ tests across students, which we equate to produce a common IQ score. It has limited measures of personality but rich data on adult outcomes. The National Survey of Midlife Development in the United States (MIDUS) is a survey of adults aged 24–74 y old in 1995–1996 and 34–83 y old in 2004–2006. It has rich data on IQ, personality, and adult outcomes, but lacks information on achievement scores or grades. No single dataset produces definitive evidence. It is the consilience of the evidence across the diverse datasets that justifies the conclusions of this paper.^§

This section summarizes the correlations among the dimensions of human capabilities that we study. It also analyzes the extent to which personality predicts achievement test scores and grades above and beyond IQ.

Table 2 displays the correlations among the available measures of cognition and personality in our four datasets. Notice that the correlations between IQ and grades as well as between IQ and achievement tests are far from perfect. The same is true of the correlations between grades and achievement tests. Personality is positively correlated with grades and achievement test scores. Grades, achievement tests, and IQ capture different aspects of human capabilities.

Figs. 1, 2, and 3 display the predictive power of personality and IQ on grades and scores on achievement tests as measured by the adjusted R².^¶ The results from the Stella Maris data in Fig. 1 indicate that scores on the Raven’s Progressive Matrices test explain more of the variance in achievement scores (DAT) than the personality measures. However, personality traits explain a substantial fraction of the variance in the DAT, even when Raven IQ scores are included in regressions. In the Stella Maris data, grades are mostly related to personality traits. Scores on the Raven test do not predict overall grades.

Fig. 2 decomposes achievement tests and grades using data from the BCS. The results show that IQ and personality measured at age 10 y old predict scores on various achievement tests at ages 10 and 16 y old and grades at age 16 y old.

The NLSY data in Fig. 3 show that IQ explains more of the variance in the AFQT scores and grades than the only available personality variables—self-esteem and locus of control—but both personality measures are predictive. Note, however, that the measures of personality in the NLSY are only a subset of the wide array of personality traits typically used by psychologists (ref. 7 has a summary of these measures).

The predictive power of personality and IQ for grades and scores on achievement tests is considerably lower in the Stella Maris data compared with the other datasets. The predictive power of personality and IQ for grades and scores on achievement tests is considerably lower in the Stella Maris data compared with the other datasets, which is probably due to the restriction on range in that dataset. The sample is constructed from the two highest tracks (of three possible tracks) at that secondary school.

Some basic patterns emerge across all datasets. Personality predicts grades and scores on achievement tests. IQ is weighted more heavily in predicting achievement scores than in predicting grades. Note that most of the variance in both measures remains unexplained. The reason may be, in part, because of measurement error. However, it is also likely that important determinants of these measures are missing in our datasets.

Using the BCS, the NLSY, and the MIDUS, we determine how much of the variation in numerous important life outcomes is explained by IQ and personality traits. We also consider the relative predictive power of grades and scores on achievement tests compared with IQ. The outcomes studied include wages and measures of health among other items. We build on the analyses in refs. 4, 5, 7, and 26.

The results of our analysis of the BCS data plotted in Fig. 4 reveal that, for wages, years of schooling, body mass index, number of arrests, and life satisfaction, personality is at least as predictive as IQ.^# However, the variation explained by IQ and personality is relatively small. Consider, for example, the contribution to explained variance from a regression of log wages on IQ, personality, scores on achievement tests, and grades—reported in various combinations. Column 1 in Fig. 4 in the first block of columns (corresponding to wages) shows that IQ predicts wages, but the predictive power is small (around 1%). Column 2 in Fig. 4 shows that self-esteem, locus of control, antisocial behavior, and neuroticism, taken together, are more important determinants of wages. Both IQ and personality remain as important predictors in wage equations when both are included in a regression (column 3 in Fig. 4). The fourth column in Fig. 4 shows that achievement has more predictive power than IQ and personality alone. When IQ and personality are also included in a regression (column 5 in Fig. 4), achievement test scores remain an important predictor of wages, and IQ and personality also remain important predictors of wages. After controlling for scores on achievement tests, IQ loses around 60% of its predictive power. When grades are included, instead of achievement tests, the effect of IQ becomes negligible. A similar pattern arises across the other outcomes studied.

For the NLSY79, Fig. 5 parses the contributions of personality and IQ for a set of outcomes. Fig. 5 shows that IQ and personality only explain a small portion of the variance for all of the outcomes studied but that both are important predictors. IQ explains more of the variance than personality for log wages, any welfare, and physical health at age 40 y old, whereas personality explains more of the variance in mental health at age 40 y old and whether or not the individual voted in 2006. Achievement tests are better predictors of important life outcomes than IQ.

An analysis of the MIDUS data allows us to consider the predictive power of the Big Five personality traits for economic and health outcomes. Fig. 6 shows that the Big Five personality measures in the MIDUS data explain a much larger percentage of the variance than IQ for both wage and health outcomes.

The relative importance of IQ and personality measures varies across datasets. This variation is likely driven by differences in the measures used, the choice of measures, the populations considered, and the circumstances under which tests are taken. For example, in the NLSY79, IQ is a better predictor of log wages than personality, but in the BCS and the MIDUS data, personality measures are better predictors. The better and more comprehensive personality measures in the BCS and the MIDUS data compared with those available in the NLSY data likely explain why personality is more predictive of outcomes in those data. The differences may also be driven by the availability of outcomes in each dataset, because different outcomes most likely place relatively more or less importance on IQ and personality. For example, in both the NLSY79 and the MIDUS, mental health depends relatively more on personality than physical health.^||

Despite variation across datasets, consistent patterns emerge. Personality is a powerful predictor for most life outcomes across all datasets. Grades and achievement test scores are more predictive of adult outcomes than IQ. In regression analyses reported in SI Appendix, Appendix S8, adding grades and test scores to models with IQ and personality produces greater predictive power for the outcomes studied. This larger explained variance is additional evidence that they capture relevant dimensions of human capability not captured by IQ and personality. A general message from our analysis is that additional dimensions of achievement remain to be discovered.

Cognitive skills predict life outcomes. This paper reinterprets the evidence on the relationship between cognitive skills and a variety of important life outcomes by analyzing the constituent components of widely used proxies for cognitive skills—grades and achievement tests. Measures of personality predict achievement test scores and grades above and beyond IQ scores. Analyses using scores on achievement tests and grades as proxies for IQ conflate the effects of IQ with the effects of personality. Both measures have greater predictive power than IQ and personality alone, because they embody extra dimensions of personality not captured by our measures.

Why do these findings matter? Achievement tests are widely used to measure the traits required for success in school or life. It is important to know what they measure to design effective policy and use these measures to evaluate schools and teachers (evidence of teacher effectiveness on personality and its consequences for high school graduation is in ref. 28). Understanding the sources of differences in the test scores and grades used to explain the black–white achievement gap (29), the male–female wage gap (30), and other gaps by social class directs attention to what factors might be remediated (5). For example, personality or noncognitive skills are more malleable at later ages than IQ, and there are effective adolescent interventions that promote personality but are much less successful in boosting IQ (31, 32). The predictive power of grades shows the folly of throwing away the information contained in individual teacher assessments when predicting success in life.**

We received valuable comments from two referees; Anja Achtziger; Thomas Dohmen; Angela Lee Duckworth; Brent Roberts; Wendy Johnson; Tim Kautz; Anna Sjögren; seminar participants at Institutet för Arbetsmarknads- och Utbildningspolitisk Utvärdering (2010), Swedish Institute for Social Research (2010), Institute for the Study of Labor (2009), Institute for Fiscal Studies (2009), Geary Institute (2009), Tilburg University (2009), and the University of Konstanz (2009); and conference participants at the 2011 American Economic Association in Denver, the 2011 IZA Conference on Cognitive and Noncognitive Skills in Bonn, three Spencer conferences at the University of Chicago (2009 and 2010), the Second Conference on Non-Cognitive Skills in Konstanz, Germany (2009), the 2009 European Summer Symposium in Labour Economics in Buch am Ammersee, Germany, the 2009 Meeting of the Association for Research in Personality in Evanston, IL, and the 2010 Society of Labor Economists/European Association of Labour Economists in London. An appendix for this paper can be found at https://heckman.uchicago.edu/what-do-grades-measure. This research was supported by the American Bar Foundation; the Pritzker Children’s Initiative; the Buffett Early Childhood Fund; NIH Grants National Institute of Child Health and Human Development (NICHD) R37HD065072, NICHD R01HD54702, and National Institute on Aging R24AG048081; an anonymous funder; Successful Pathways from School to Work, an initiative of the University of Chicago’s Committee on Education; the Hymen Milgrom Supporting Organization; the Human Capital and Economic Opportunity Global Working Group, an initiative of the Center for the Economics of Human Development; the Institute for New Economic Thinking; a Tore Browaldh Grant from the Handelsbanken Research Foundation, and a VIDI grant from The Netherlands Organization for Scientific Research.