Latent structure in random sequences drives neural learning toward a rational bias

Neural Model of Temporal Integration

Our neural model is extremely simple (Fig. 2A). It consists of a sensory input layer that scans nonoverlapping binary digits of H vs. T and an internal prediction layer that attempts to predict the next input, while the prior inputs in the sequence are encoded in the temporal context. This model is based on a biologically motivated computational framework that has been used to explain the neural basis of cognition in a wide range of different domains (6), with the benefit of integrating prior temporal context information according to the properties of the deep neocortical neurons (layers 5b and 6) (7).

• Download figure
• Open in new tab
• Download powerpoint

Fig. 2.

Neural model of temporal integration to capture the statistics of pattern times in random sequences. (A) Architecture of the neural model. A single sensory input layer scans through a sequence of binary digits one digit at a time (input at t−1 is for illustration only). An internal prediction layer, with bidirectional connections from the input layer and its own temporal context representation, attempts to predict the next input. (B) Neural model behavior depicted by the ratio between repetition and alternation detectors in response to the actual probability of alternation (pA) in the input sequences. At pA=1/2, the model showed R/A≈0.70 (i.e., fewer repetition detectors than alternation detectors). Error bars (± SEM) represent the variability of model predictions. The dotted line is the squared total time ratio between alternation and repetition patterns (Eq. 2).

Our main hypothesis is that the predictive learning and temporal integration properties of this model, which reflect corresponding features of the neocortex, will produce representations that incorporate both the waiting time and mean time statistics of the input sequences (despite the inability of the model to accurately predict the next input in these random sequences). In other words, we predict a systematic interaction between these basic learning mechanisms and the surprisingly rich statistical structure of the input. This is a principled prediction based on the well-established sensitivity of these kinds of neural learning mechanisms to the statistical structure of inputs (e.g., ref. 11), and extensive parameter exploration demonstrates that our results hold across a wide range of parameters and that the model’s behavior is systematically affected by certain parameters in sensible ways (SI Text). Thus, despite the emergent nature of learning in our model, it nevertheless reflects systematic behavior and is not the result of a random parameter-fitting exercise. Moreover, we show below that the model’s behavior can be largely explained by a simple equation as a function of the mean time and waiting time statistics, further establishing the systematicity of the model’s behavior and also establishing a direct connection to more abstract Bayesian models as elaborated subsequently.

The model was trained with binary sequences generated at various levels of the probability of alternation (pA), each sequence consisting of 10,000 coin tosses (although learning occurred quickly within a few hundred trials). Crucially, learning was concerned with only reconstructing the input sequence but not pattern discrimination, as no teaching signals were provided regarding the underlying pA values and pattern time statistics. After training, the model was tested with a sequence of 1,000 tosses generated at the same pA level as in the training sequence. We then decoded these sequence representations through a reverse correlation technique. Based on the sensitivity of the unit activations to the temporal patterns of length 2, we classified and then counted the number of the units on the internal prediction layer as either repetition detectors (R, being sensitive to either HH or TT) or alternation detectors (A, being sensitive to either HT or TH). (See SI Text for the method of detector classification.)

Most intriguingly, at pA=1/2 (i.e., independent tosses of a fair coin), the model produced a ratio of R/A≈0.70—despite the equal frequency of pattern occurrences, repetition detectors were significantly less likely than alternation detectors. Such alternation bias is in the same direction of the representativeness bias, where people perceive alternation patterns as more representative of a random process than repetition patterns (2, 3). Effectively, this result demonstrates the gambler’s fallacy emerging naturally as a consequence of the alternation bias, due to the model’s sensitivity to the waiting time advantage of alternations compared with repetitions.

We then used the R/A ratio to compute the subjective probability of alternation, pA′, as the model’s internal representation of its actually experienced pA. With R/A≈0.70, we havepA′=AR+A=11+R/A≈0.59 .[1]This pA′ value is consistent with the empirical findings on subjective randomness. From a comprehensive review of the studies on random sequence perception and generation, it was found that the subjective probability of alternation was around 0.58∼0.63 (9).

To further characterize the nature of the alternation bias, we systematically varied the probability of alternation (pA) in generating the training sequence (i.e., departures from tossing a fair coin independently) and then measured the effects on the R/A ratio. We found a smooth curve, where the R/A ratio increased (more repetition detectors) as pA decreased (less frequent occurrences of alternations). At pA=3/7, the model reached an equilibrium point with equal numbers of repetition and alternation detectors, R/A=1 (Fig. 2B). That is, alternations have to be this much less frequent (i.e., greater mean time) to cancel out their waiting time advantage. This corresponds exactly to the equilibrium point where repetitions and alternations have the same sum of mean and waiting times (Fig. 1B).

Overall, the model’s behavior can be mostly replicated by a simple equation that averages the effects of the mean time and waiting time statistics (dotted line in Fig. 2B),RA≈(E[TA]+E[TA∗]E[TR]+E[TR∗])2,[2]where E[T] is the mean time, E[T∗] is the waiting time, and subscripts R and A represent repetitions and alternations, respectively. This establishes a clear higher-level explanation for the emergent behavior of the model, allowing us to summarize its behavior as simply averaging the effects of these two relevant statistics over the random sequences.

Bayesian Models of Random Sequence Production

A unifying perspective on human statistical learning requires bridging the gap between the implicit learning without instruction and the generalization of the learned patterns under structured and expressive rules (12). On the one hand, our neural model shows that through mere exposure to a set of input stimuli, a systematic bias was developed toward the alternation pattern in random sequences. On the other hand, recent Bayesian accounts for probabilistic learning suggest that the human mind performs rational inferences at both neural and behavioral levels (13, 14). Thus, we asked whether it was possible to relate the emergent behavior of the neural model to an existing Bayesian model of randomness judgments, specifically whether we could demonstrate a quantitative connection between the bias for local patterns at the neural level and the behavior of generating longer random sequences governed by the rules of Bayesian inference.

Let f(H,T) denote the degree of the belief that a sequence of coin tosses consisting of H heads and T tails is generated by a fair coin, where the probability of heads in any single toss is p=1/2. By Bayes’ theorem, assuming a uniform prior distribution p∈[0,1], f(H,T) can be formulated as the posterior probability density,f(H,T)=2−(H+T)∫01pH(1−p)Tdp=2−(H+T)(H+T+1)(H+TH).[3]

Because of the binomial coefficient (H+TH), Eq. 3 is maximized when H=T. That is, governed by the belief function f(H,T), the optimal solution to generating a random sequence is to always seek a balance between the numbers of heads and tails (10). Based on this belief function, Griffiths and Tenenbaum (8) proposed a Bayesian model of random sequence production. They first defined a likelihood function, Lk, to represent the local representativeness that choosing a head instead of a tail as the outcome of the kth toss would result in a more random sequence,Lk=∑i=1k−1log⁡f(Hi+1,Ti)−log⁡f(Hi,Ti+1)=log∏i=1k−1(Ti+1Hi+1), k≥2,[4]where Hi and Ti were, respectively, the numbers of heads and tails counting back i steps in the sequence. Then, with a free parameter (λ) to scale the contribution of Lk, the probability of choosing a head at each response (Rk) was obtained by a logistic function:P(Rk=H)=11+e−λLk.[5]

This Bayesian model was then used to fit a massive database from the “Zenith radio experiment,” where 20,099 participants attempted to produce five random binary symbols one at a time (15). It was found that a λ value of around 0.60 produced the optimal fit to 15 of the 16 data points from the human data (Fig. 3A).

• Download figure
• Open in new tab
• Download powerpoint

Fig. 3.

Bayesian models fitting to human data in random sequence production. (A) Probabilities of the generated random sequences, collapsed over the first choice (e.g., HHHHH is combined with TTTTT). Human data represent the responses of 20,099 participants (15). In the model by Griffiths and Tenenbaum (G&T 2001; ref. 8) (Eq. 5), the bias-gain parameter λ≈0.60 was obtained by best fitting the model to 15 of 16 human data points (excluding “HTHTH”). In our augmented model (Eq. 7), λ≈0.51 can be derived from the emergent behavior of the neural model. (B) Best-fitting λ values for the model by Griffiths and Tenenbaum (8) and the augmented model, with either the partial or the full dataset. In both datasets, the optimal λ value for the augmented model remained the same at 0.51 as predicted by the neural model.

The data point that Griffiths and Tenenbaum (8) did not predict well was the sequence HTHTH, which was judged by people to not be a very good random sequence, but Eq. 5 ranked it highly. It seems that in generating random sequences, people were seeking a balance not only between the heads and tails but also between higher-order pattern events (e.g., alternation itself is repeated four times in HTHTH). We can add this mechanism into the Bayesian model with an additional term, Mk,Mk=log(OT+1OH+1), k≥3,[6]where Mk performs a similar function to that of Lk, except being based on the numbers of the second-order pattern events, OH and OT (either alternation or repetition, depending the choice at Rk−1).

Applying the same scaling factor λ to both Lk and Mk, Eq. 5 becomesP(Rk=H)=11+e−λ(Lk+Mk).[7]This augmented model now produces an excellent fit to the full set of sequence data points (Pearson’s R2≈0.86), with λ≈0.51 as the optimal parameter value (Fig. 3A). In addition to improving the prediction on the sequence HTHTH, Eq. 7 also consistently makes better predictions than Eq. 5 on other data points, and the same value λ≈0.51 also produces the best fit to the partial dataset excluding the sequence HTHTH (Pearson’s R2≈0.89) (Fig. 3B).

The Bayesian model provides a formalization of the representativeness heuristic (2, 3, 8, 14). It captures the idea that when generating random sequences, people are seeking a balance between heads and tails and between repetitions and alternations not only in the global sequence, but also in the local subsequences (10). Apparently, the extent to which this balance is adjusted is determined by the free scaling parameter λ in Eqs. 5 and 7. However, beyond parameter fitting, the Bayesian model does not have any independent basis for specifying this parameter.

In the light of the neural model’s behavior (i.e., the alternation bias in Eq. 1), we predict that the scaling parameter λ should have originated naturally from people’s actual experiences of random sequences in the learning environment. Specifically, we can deduce from either Eq. 5 or Eq. 7 that λ actually serves as a bias-gain parameter that modulates the strength of the alternation bias:pA′=P(R2=T|R1=H)=11+2−λ.[8]When λ=0, Eq. 8 produces unbiased responses with the subjective probability of alternation pA′=1/2, corresponding to the process of independent coin tossing; and higher values λ>0 produce an increasing alternation bias with pA′>1/2. In other words, λ>0 corresponds to the tendency of avoiding repetitions, which applies to both the first- and second-order events (Eq. 7).

Then, we are able to show that λ can actually be derived from the behavior of the neural model. Substituting pA′ in Eq. 8 with Eq. 1, λ can be computed directly by the neural model’s R/A ratio (repetitions over alternations):λ=−log2RA.[9]For independent fair coin tossing (i.e., pA=1/2), the neural model showed R/A≈0.7, resulting in λ≈0.51—precisely the value that optimizes the fit to the human data for the augmented model (Fig. 3B).

The implication of Eq. 9 is that the naturally emergent properties of the neural model can in effect provide an independent anchor to the previously free parameter in the Bayesian model. Specifically, it shows that the bias-gain parameter λ is anchored to the alternation bias, which has been learned by the neural model through mere exposure to random sequences of fair coin tossing. Moreover, Eq. 9 is in accord with both the subjective probability of alternation pA′ (Eq. 1) and the normative measure of pattern mean time and waiting time statistics (Eq. 2). Most significantly, the derivation of the λ value demonstrates a quantitative connection between the implicit learning without instruction and the generalization of the learned patterns under structured and expressive rules, supporting a unified perspective on these two different learning mechanisms (12). This represents a remarkable convergence across multiple levels of analysis and further bolsters the validity of our understanding of the nature and origin of the systematic preference for alternating sequences and against repeating ones.