Superhuman artificial intelligence can improve human decision-making by increasing novelty

The top two panels of Fig. 1 present the time trends of human decision quality from 1950 to 2021. Specifically, Panels A and B show the estimated fixed effect of each year and month, respectively, on decision quality; player fixed effects were controlled for and the respective first periods were used as the baseline (i.e., 1950 for Panel A and January 1950 for Panel B). Both these panels reveal that human experts began to make significantly better decisions after the advent of superhuman AI, as evidenced by greater fixed effects of years (Panel A) and months (Panel B) after the 2016 to 2017 period than before it. Contrary to our expectations, both these panels also show that human decision-making improved comparatively little before the advent of superhuman AI (but SI Appendix, section 4.2 for minor exceptions upon closer examinations). In sum, human decision-making in Go improved substantially after AlphaGo’s victory in 2016 and release of superhuman AI programs in 2017, in stark contrast to the comparatively flat trend observed in the preceding 66 y (1950 to 2016).

Next, we examined the novelty of human decisions over the same period. One way to gauge novelty in decision-making is to examine the novelty of move sequences. As is commonly done in the chess community (19), we can find in each game the first move that makes the game’s move sequence historically novel (20, 21). For example, if a particular sequence of the first 9 moves in a game was observed in some game in the past, but the 10th move makes the sequence of 10 moves historically novel (i.e., never observed in any of the previous games in the dataset), then we may say that Move 10 was a novel move. If such a novel move occurs at an earlier point in one game than in another, then the earlier departure from previously observed decision-making would indicate greater novelty in the former game than in the latter. We thus used the idea of a novel move to construct a measure of novelty. Specifically, we identified the novel move in each game and subtracted its move number from the maximum move number observed in the dataset (i.e., 60^†) to form a Novelty Index for the novel move. (SI Appendix, section 2.3 for details about the Novelty Index.) We then conducted our novelty analyses using only the data on these novel moves.

We thus estimated the novelty of human decisions over time using this Novelty Index in the same way we estimated decision quality over time using the DQI. We present the results in Fig. 1, Panels C and D. As expected, later years and months put greater downward pressure on the Novelty Index (i.e., more negative fixed effects) because the observed set of unique move sequences grew over time and pushed novel moves later. Most importantly, however, we find a sharp increase in the yearly (Panel C) and monthly (Panel D) fixed effects on the Novelty Index following the advent of superhuman AI, suggesting that players broke away from previously observed move sequences earlier in their games. An additional analysis with an alternative operationalization of novelty showed a similar increase in novelty after the advent of superhuman AI (SI Appendix, Fig. S1).

Finally, we test whether such an increase in novelty can explain the improvement in decision-making. Specifically, we regressed the DQI of each move on i) the binary variable of whether the move was made before versus after the advent of superhuman AI (After AI Dummy), ii) the binary variable of whether the move was a novel move (Novelty Dummy), and iii) their interaction. Simultaneously, we controlled for fixed effects of months (only for Model 2), move numbers, and players. Our coefficient of interest was that of the interaction term which signifies how much more novel move decisions improved as compared with nonnovel move decisions, following the advent of superhuman AI. The first column in Table 1 shows the regression results not controlling for monthly fixed effects, whereas the second column shows the regression results controlling for monthly fixed effects without estimating the effect of After AI Dummy (to avoid multicollinearity).

In each of the two regression models, the interaction term was significant and positive, $\widehat{{\beta }_3}=0.515$. This suggests that the post-AI increase in decision quality was greater for novel moves than for nonnovel moves. In other words, novel and nonnovel moves together contributed to the increase in decision quality (because all moves had to be either novel or not novel, by definition), whereas novel moves on average contributed significantly more to the increase in decision quality than nonnovel moves on average.

One way to test the memorization hypothesis is to examine the move decisions that differ from the counterfactual (optimal) AI decisions. If memorization of the optimal AI decisions fully explains the increase in decision quality, then human decisions that differed from the optimal AI decisions would not show the same increase in quality after the advent of superhuman AI. On the other hand, if memorization cannot fully explain the post-AI increase in decision quality, then human decisions that differed from the optimal AI decisions could still show the post-AI increase in quality.

With this reasoning, we estimated the time trends of decision quality only for the decisions that differed from the optimal AI decisions. In our dataset, about 40% of all human decisions matched the optimal AI decisions, leaving us with about 60% of all human decisions for this analysis.

Fig. 2 shows the time trends of decision quality estimated from only the move decisions that differed from counterfactual (optimal) AI move decisions. As with the previous analysis (Fig. 1, Panel A), we still find a similar increase in the yearly fixed effects on decision quality after the advent of superhuman AI, even when our analysis focused on the move decisions that differed from those of AI. Estimating the monthly fixed effects for this restricted set of decisions shows the same post-AI increase in decision quality (SI Appendix, Fig. S2).

As this analysis involved only the decisions that differed from the optimal AI decisions, memorization of the optimal AI decisions cannot explain the still-evident increase in decision quality following the advent of superhuman AI.

As another way to test the memorization hypothesis, we analyzed the quality of decisions at different stages of the game, focusing especially on relatively later stages of the game (e.g., Moves 21 to 60). Because memorization of the optimal AI decisions in these later stages would be practically impossible (due to the complexity of Go), a post-AI increase in decision quality also in these stages would be evidence against the memorization hypothesis.

Prior to conducting these analyses, we found that more than half of all novel moves in our entire dataset occur before Move 10 and that 99% of opening move sequences become historically novel by Move 21. This meant that moves after Move 20 would likely present a completely novel state of the board to human players (i.e., different from every state they have ever faced), and that memorizing the optimal move for the completely novel state would not have been feasible. Based on this knowledge of novel moves, we split our dataset into six subsets of 10 moves and examined the time trends of decision quality for each set of 10 moves.

If the human decision quality increased because the humans simply memorized series of optimal decisions generated by an AI program, then the post-AI increase in decision quality would likely be observed for the earliest stages of the game (e.g., Moves 1 to 10 and Moves 11 to 20) but would unlikely be observed for relatively later stages of the game (e.g., Moves 21 to 60). On the other hand, if the human decision quality increased because the humans were making better decisions themselves, then the post-AI increase in decision quality would be observed in both types of stages (i.e., not only for Moves 1 to 20 but also for Moves 21 to 60). Indeed, our data are more consistent with this latter case (SI Appendix, Fig. S3). That is, human decision quality increased not only for the earliest stages of the game (Moves 1 to 10^‡ and Moves 11 to 20) but also for the relatively later stages of the game (Moves 21 to 50); SI Appendix, section 4.2 for more discussion on this analysis. These results provide additional evidence against the memorization hypothesis.

We further test the memorization hypothesis by examining the move decisions made after opponents deviate from a series of optimal AI decisions. Suppose players and their opponents memorized sequences of optimal AI decisions and such memorization fully explains the post-AI increase in decision quality. If so, in cases where an opponent first deviates from the series of optimal AI decisions, the focal player’s decision in response would not be part of the memorized sequence and would be unlikely to show the post-AI increase in quality. Thus, examining such decisions allows another test of the memorization hypothesis.

To illustrate, consider the following move sequence: dd (Black) → pp (White) → dp (Black) → pd (White). If the first two decisions (dd and pp) matched the optimal AI decisions, but the next decision by the opponent, dp, did not match the optimal AI decision, then the decision made in response to the opponent’s deviation, namely pd, would be analyzed.

We thus repeated the analysis of decision quality as in the Results section but focusing only on move decisions that come immediately after the opponent deviates from the optimal AI decision. Indeed, we do find that decisions in response to the opponent’s deviation also show an increase in quality following the advent of superhuman AI (SI Appendix, Fig. S4) and SI Appendix, section 4.3 for more discussion on this analysis. This is inconsistent with the memorization hypothesis, suggesting that the increase in human decision quality following the advent of superhuman AI is unlikely to be a result of human players simply memorizing sequences of AI decisions.

In this and the following subsections, we test whether the memorization hypothesis can fully account for the post-AI increase in novelty of human decisions. Recall that after the advent of superhuman AI, the yearly and monthly fixed effects on Novelty Index increased (Fig. 1 C and D). In other words, the historically novel move in each game tended to occur earlier in the game after the advent of superhuman AI. So, if novel moves occurring earlier in the game is simply a result of humans memorizing and replicating the optimal AI decisions, then examining only the novel moves that differed from the optimal AI decisions would not show such evidence for the increase in novelty.

With the reasoning above, we estimated the yearly fixed effects on the Novelty Index analyzing only the novel move decisions that differed from the corresponding optimal AI decisions (Fig. 3). A comparison with the temporal pattern from the move decisions that match the optimal AI decisions is also provided (SI Appendix, Fig. S5). In both analyses, we find the upward shift in the time trends of the Novelty Index following the advent of superhuman AI. Even when we examine only the novel move decisions that differed from the optimal AI decisions (i.e., decisions in which the corresponding optimal AI decisions were not memorized), the upward shift in Novelty Index is readily observable. This suggests that the increase in novelty cannot be explained solely by humans memorizing and replicating the optimal AI decisions.

As yet another test of the memorization hypothesis regarding the post-AI increase in decision novelty, we repeated the analyses of novelty in Results section (Fig. 1 C and D) after adding 600k move decisions made by a superhuman AI program into the dataset. For the previous analyses of the Novelty Index, we had identified the historically novel move in each game by considering only the previously observed sequences of human decisions. As a result, if humans witnessed historically novel decisions in their private studies with AI programs and then they themselves made the same decisions, such decisions would be considered historically novel (human) decisions in our analyses, even though they were initially discovered by AI programs. If such memorization occurred often enough, a substantial proportion of novel moves may have been misattributed to novelty by humans rather than novelty by AI programs.

To test for this possibility, we generated sequences of AI move decisions (i.e., AI versus AI) and inserted them in our dataset, temporally positioning them immediately before the advent of superhuman AI. We then redetermined novel moves with this expanded set of decision sequences artificially inserted and positioned prior to the advent of superhuman AI. If humans were merely memorizing novel decisions initially discovered by the AI programs, such decisions would now be less likely to be counted as historically novel because the same decisions would likely already exist in the dataset as having been made before the advent of superhuman AI. Thus, if we find the same post-AI increase in novelty (as measured with the Novelty Index) even after adding the simulated AI decisions, it would present further evidence against the hypothesis that memorization of AI decisions increased novelty.

Indeed, that is exactly what we find (SI Appendix, Fig. S6). Even when the 600k newly simulated AI decisions were added to the dataset as having been made before the advent of superhuman AI, and the Novelty Index of each game was recalculated accordingly, the time trends of the Novelty Index hardly change. In other words, the increase in novelty is still evident even when accounting for the possibility that human players may have used AI programs to generate and memorize promising game patterns.

Raw data came from Games of Go on Disk (“GoGoD”), a copyrighted collection of Go data (29) on move-by-move decisions. We purchased the database (i.e., zipped sgf files) and created a dataset for our analyses (30–33). We then used this dataset to construct the Decision Quality Index (DQI) and Novelty Index. Calculating the DQI for each move decision required a superhuman AI program (18) to generate the AI-optimal move decision and estimate the difference between the win rate of the counterfactual (optimal) AI decision and that of the actual human decision. Calculating the Novelty Index for each game did not require any superhuman AI program. More details about both measures are presented in SI Appendix, section 2.

To use observational data to track the quality of human decisions over time, it is essential to establish an objective measure that quantifies the quality of human decisions. However, the large state space of Go makes it difficult to determine which decisions are objectively better than others. We thus leverage a superhuman AI program KataGo (18) that peers into the complex relationship between a decision made in a given state and the final outcome of the game (i.e., win or lose) associated with the decision.

To construct the Decision Quality Index (DQI), we take advantage of two features of the AI program: 1) It makes decisions of superhuman quality, and 2) it can evaluate the quality of any decision, whether the decision was made by a human player or itself. We first enter into the AI program the game log data for all games between professional Go players. The log data contain a series of move decisions made by human players. Then, for each move decision, we have the AI program make its own optimal decision in the same state. This AI-generated decision serves as a counterfactual (optimal) decision (Step 1). Using the same AI program, we compare the win rate associated with the AI-generated decision with the win rate associated with the human decision (Step 2). The DQI of a given human decision, then, equals [100% - (win rate of the Counterfactual AI Decision - win rate of the actual human decision)]. SI Appendix, section 2.2 has a more technical definition.

M.S., J.K., B.v.O., and T.L.G. designed research; M.S. and J.K. performed research; M.S. and J.K. analyzed data; and M.S., J.K., B.v.O., and T.L.G. wrote the paper.

The authors declare no competing interest.