Robust identification of investor beliefs

Prices in asset markets reflect a combination of investor beliefs and their risk preferences. Researchers, as well as policymakers, look to asset market data as a barometer of public beliefs. Derivative claims prices potentially enrich what we can infer about conditional probability distributions of future events, but events of interest often entail components of macroeconomic uncertainty for which there will be a paucity of information along some dimensions. Moreover, since a central tenet of asset pricing is that investors must be compensated for exposure to macroeconomic shocks that are not diversifiable, beliefs about future macroeconomic performance are of paramount importance to understanding asset prices.

To disentangle the contributions of risk aversion from beliefs, many empirical approaches in the last few decades have focused on models of investor preferences by assuming rational expectations. Using the implied moment conditions of the investor’s portfolio choice problem in conjunction with this restriction gives a directly applicable and tractable approach for estimating and testing alternative model specifications. This approach, however, often leads to risk prices in some time periods that are attributed to an arguably extreme level of investor risk aversion or a rejection of the model. Risk aversion and belief formulation are intertwined. Rational expectation as a model of belief formation is meant to be a simplifying approximation. It can serve as an elegant and powerful modeling choice when appropriate. In a complex environment, however, it can be challenging to make statistical inferences pertinent to forward-looking decision making. In such settings, we find the presumption that model-dwelling investors and entrepreneurs know the true data-generating process to be tenuous and worthy of relaxation. We are not alone in this view.

Some researchers have explored mechanisms that could account for this evidence via a different channel, namely beliefs which differ from rational expectations. It is sometimes argued, but typically not justified formally, that these alternatives are small departures from rational expectations. These “belief distortions” relative to rational expectations alternatively could reflect the lack of investor confidence about the assignment of probabilities to future events. This has been modeled and captured formally as ambiguity aversion or concerns about model misspecification.

This paper proposes a formal methodology for analyzing models that imply conditional moment restrictions where the restrictions are presumed to hold under a distorted probability measure. It extends a previous econometrics literature that represents the statistical implications of asset pricing implications as conditional moment restrictions under rational expectations. Rational expectations on the part of individuals or enterprises can be motivated by a law of large numbers approximation used to pin down the beliefs of these economic agents “inside the model.” Once we relax the rational expectations, there are typically many choices of investor beliefs that satisfy the conditional moment restrictions. Rather than imposing a specific alternative to rational expectations, we restrict the family of investor probabilities to satisfy discrepancy bounds, which gives us a way of relaxing the rational expectations hypothesis based on pushing back from a law of large numbers approximation. We then use the conditional moment restrictions along with statistical discrepancy bounds, to characterize families of probabilities that satisfy the conditional moment restrictions.

Our approach provides a version of “bounded rationality” when assessing empirical evidence. Not only are the bounds we deduce of direct interest; they also can be used as diagnostics for specific models of belief distortions. Additionally, we show how to include survey data on subjective beliefs. Such data are typically sparse and not sufficient to pin down full probabilistic characterizations of beliefs. Given these data limitations, we bound probabilities even when certain features of the data may be known through direct evidence.

A common way to represent a probability distribution of a random vector is through how it assigns expectations to functions of that random vector. Since we have multiple probability distributions in play, we represent our bounds by building what is called a “nonlinear expectation” that minimizes expectations over members of the family of probability distortions that we identify. This gives us a formal way to characterize properties of probability distributions that are consistent with model-implied conditional moment restrictions. The choice of minimization is essentially a normalization as we bound expectations of functions of observable variables as well as the negative of such functions. Given the flexibility in our choice of what functions of observables we use when forming expectation bounds, our analysis provides a rich characterization of implications for the belief distortions.

While we use asset pricing applications for motivation, our analysis is more generally applicable to economic models with forward-looking agents. These agents may be groups of individuals making investment or portfolio choices when facing production or financial opportunities that are exposed to uncertainty in different ways. Alternatively, they may be forward-looking enterprises, making decisions today that have important consequences for the future.

In summary, our methodology gives a way to extract information on investor beliefs from asset market and survey data pertinent for both external analysts and policymakers who are looking for evidence to gauge private sector sentiments. In addition, our computations provide revealing diagnostics for model builders that embrace specific formulations of belief distortions as is common in the behavioral economics and finance literatures.

1. Asset Pricing with Distorted Beliefs

In standard economic applications, moment conditions are justified via an assumption of rational expectations. This assumption equates population expectations with those used by economic agents inside the model. These expectations are therefore presumed to be revealed by the law of large numbers applied to time series data.

Let (Ω,G,P) denote the underlying probability space and I⊂G represent information available to investors. The original moment equations under rational expectations are of the formEf(X,θ)∣I = 0,[1]where the function, f, captures the parameter dependence, θ, of either the payoff or the stochastic discount factor along with a random vector, X, of variables observed by the econometrician and used to construct the payoffs, prices, and the stochastic discount factor.

A typical asset pricing example is as follows: Let R denote an n-dimensional vector of gross returns corresponding to payoffs on financial or physical assets over some investment horizon, let S denote the corresponding stochastic discount factor for this horizon, and let I denote the investor information set. The stochastic nature of the stochastic discount factor captures the market compensations for exposure to uncertainty.

The underlying asset pricing equation isE[SR−1n|I] = 0,where 1n is an n-dimensional vector of ones. Both the stochastic discount factor and the return vector R may depend on unknown parameters, giving rise to Eq. 1. The vector of returns can be parameter dependent when the investment is in a physical asset with an unobserved return. While we posed this as an asset-pricing relation, restrictions of the same form can be derived from investment or asset demand equations with potentially differential exposures to uncertainty. Even in such models, there is a counterpart to stochastic discount factors that are perhaps different across agent types. More generally, these equations feature the forward-looking behavior of individuals or enterprises as they depend on the perceptions of the future.

2. Data Generation and Probability Divergence

In this section we construct and use the dynamic counterpart to the statistical divergence measure. We focus initially on relative entropy as a measure of statistical divergence, a measure which frequently arises in the analysis of large deviations of stochastic processes with temporal dependence (see, for instance, refs. 27 or 28). While we use relative entropy as a starting point, we go much farther by extending the so-called ϕ divergence measures in ways that have a recursive structure that is very similar to that of relative entropy.

While the applications that interest us use Markov formulations, we relax this assumption to entertain non-Markov distortions. For this reason, we initially consider a stationary and ergodic formulation that nests stationary, ergodic Markov processes. This is the same environment used by ref. 29 to study bounds on statistical efficiency and ref. 30 to study testable implications of asset pricing models, both in the presence of conditional moment restrictions. The ref. 30 analysis imposes rational expectations in contrast to the analysis in this paper.

We start with a baseline probability triple (Ω,G,P) and a measurable one-to-one transformation U which is measure preserving and ergodic under P. We use U to construct stochastic processes and filtrations.^†

Let I0⊂G depict information available at date zero. We use the transformation U to capture the information available at future dates via the recursionIt=Λ∈G:U−1Λ∈It−1=Λ∈G:U−tΛ∈I0.We presume that information accumulates,It⊂It+1,which in turn implies that It:−∞<t<+∞ is a filtration. Similarly, for any random variable B0 that is I0 measurable, we form Bt recursively:Bt(ω)=Bt−1U(ω)=B0Ut(ω).Thus for each initial random vector B0, there is a corresponding stochastic process {Bt:t≥0} that is adapted to the filtration {It:t≥0}. Since U is measure preserving, the process {Bt:t≥0} is stationary.

3. Moment Bounds

We next present the recursive approach we use for computing probability bounds. To represent these bounds, we entertain a rich family of functions g and compute sharp lower bounds on the expectation g(X1). As a special case, g(X1) could be an indicator of an event with its expectation being the probability of that event. We will be interested in other choices of g in our empirical illustration. We compute an upper bound on g(X1) by finding the negative of the lower bound on the expectation −g(X1). Formally, we are interested in solvinginfN1∈N∫EN1g(X1) | I0dQ0subject to the constraintsR(N1)≤κE[N1f(X1)∣I0] = 0.To compute a bound for a given choice of g, we will borrow an idea from the robust control literature. See, for instance, refs. 15 and 37. We will initially solve a problem with a discrepancy penalty indexed by a parameter ξ>0 and show how to solve a problem given ξ. We will then treat ξ as a Lagrange multiplier and trace out the implied discrepancies for each such ξ. In this way, ξ may be chosen to enforce the constraint R(N1)≤κ. For notational simplicity we suppress the parameter dependence in the moments of f(Xt) and g(Xt).

In much of what follows, we aim to solve the following:

4. Illustration

We illustrate these methods using a familiar asset pricing model with recursive utility investors as in refs. 44 and 45. While much of the asset pricing literature appeals to an arguably large risk aversion in conjunction with rational expectations when confronting data, we constrain risk aversion and instead explore belief distortions as an alternative expectation. We follow ref. 46 by allowing for market segmentation and avoid the direct use of consumption data. We then ask what implications asset market data have for predicted consumption growth rates.

Much of the macroasset pricing literature imposes risk aversion that is arguably large at least for some states of the macroeconomy. Refs. 47 and 48 give alternative rationales for substantially restricting the risk aversion coefficient. While we do not view as a settled issue what precise bounds should be imposed on risk aversion, we assume a unit risk aversion to feature the role of belief distortions when confronting asset pricing evidence. Our choice of unity is admittedly for convenience. As we will see, this choice leads to some particularly simple asset pricing implications.

Let Rtw denote a presumed observable return on wealth. As noted by ref. 45, the one-period stochastic discount factor under rational expectations is the reciprocal of the gross return on wealth when risk aversion is unity. Thus, under distorted beliefs represented by Nt,St=Nt(Rtw)−1,where St is the one-period stochastic discount factor under rational expectations. We use this setup for our illustration.

As ref. 45 notes, the consumption Euler equation for an investor impliesENt⁡log⁡Rtw∣It−1 = −logβ+(1−ρ)ENt⁡logGt∣It−1where Gt is the ratio of consumption growth over two adjacent time periods.^§§ The parameter β is the subjective discount factor, and ρ is the reciprocal of the intertemporal elasticity of substitution. By deducing bounds on the left-hand side, we may infer bounds on ρ times the market expectation of consumption growth of equity market participants expressed in logarithms. Recursive utility preferences are specified in terms of continuation values that determine the rankings of prospective consumption processes. As a rough approximation, when ρ<1, the wealth is positively related to the continuation value, where both are relative to current consumption. Conversely, they are negatively related when ρ>1. Thus, for this model of investor preferences, whether ρ is larger or smaller than one impacts how we interpret the evidence based on conditioning information.

In our illustration, we draw on the literature that suggests returns can be predicted from dividend–price ratios. While there have been debates on how fragile this evidence is, we step aside from that discourse and take the predictability evidence at face value to illustrate our method. Given our direct use of dividend–price measures, we purposefully choose a very coarse conditioning of information and split the dividend–price ratios into three bins using the three empirical terciles. We take the dividend–price terciles to be a three-state Markov process. Dividend–price ratios are known to be persistent, and this will be evident in our calculations.^¶¶

We implement our approach using quarterly data from 1954 to 2016. We use the return on the CRSP (Center for the Research on Security Prices) value-weighted index to proxy for the return on wealth. For asset returns, we use the return on a 3-mo treasury bill and the three Fama–French factor excess returns. We impose moment conditions for each return implied by Eq. 3, each scaled by three indicator functions for the terciles of the dividend–price ratio, giving a total of 12 moment conditions. All returns are converted from nominal to real returns using the deflator for nondurables consumption obtained from the Bureau of Labor Statistics. We then apply the methods described in 3. Moment Bounds to bound functions of the return on wealth as measured by the value-weighted return.

In Fig. 1, we report the bounds on the beliefs about the expected log return, which under the assumption of the unitary risk aversion coefficient are approximately proportional to the consumption growth rate belief when the subjective discount factor β is very close to one. The conditional expectation of log returns and the unconditional counterpart are all lower than their empirical counterparts. This observation follows by comparing the •s with the boxes in Fig. 1, where the top and bottom of the boxes are the upper and lower bounds with a relative entropy constraint imposed at a magnitude that is 20% higher than the minimum. The minimum relative entropy rate implies a half-life of about 24 quarters for reducing the probability by 50% of mistakenly rejecting the rational expectations. Increasing this by 20% reduces the half-life by the same percentage to about 20 quarters.^## While our choice of a 20% increase is a bit arbitrary and used for illustration purposes, it is straightforward to compute bounds with other choices of divergence thresholds. Across alternative applications, a choice of 20%, independent of magnitude of the minimal possible divergence, would be hard to defend. One nice aspect of relative entropy is that there is an explicit statistical interpretation that we find to be revealing, and thus the magnitude of the divergence has meaning.

• Download figure
• Open in new tab
• Download powerpoint

Fig. 1.

Expected log market return. The •s are empirical averages and the boxes give the imputed bounds when we inflated the minimum relative entropy by 20%. The minimum relative entropy is 0.0284 with a half-life of 24.4 quarters.

Interestingly, it is when we condition on the low value of the dividend–price ratio that we find the box with the largest height (biggest difference between the upper and lower bounds). Also, the bounds on the unconditional distorted expectations are very similar to those we found for the low dividend–price ratio. As a robustness check, we repeated these calculations for a quadratic conditional divergence and found only very modest differences (SI Appendix, section F).

Not only are conditional means distorted, but so are the transition probabilities as reported in Table 1. While the implied stationary probabilities are fairly evenly distributed over the three dividend states, essentially by construction, the minimal entropy probabilities down-weight substantially the high dividend–price ratio state and up-weight the low dividend–price state. The high dividend–price state, in particular, has a very small stationary probability under the minimum distorted stationary distribution. Consistent with this, the transition probabilities into this state are lower under the distortion and they are higher for exiting this state. The opposite happens for transitions in and out of the low dividend–price state. Thus, a hypothetical process that behaves in accordance with the minimum entropy distorted Markov transition matrix is likely to spend substantially more time in the low expected log-return state and much less time in the high expected log-return state. When we increase the relative entropy bound by 20%, the implied distorted transition matrices are quite similar to the implied transition matrix recovered by the minimizing relative entropy and depart from the empirical transition matrix in comparable ways.

An alternative approach would be to solve static versions of our analysis for each of the three different specifications of the conditioning information. Under this approach, there would be no reason for distorting the transition probabilities as the dynamical evolution of the conditioning information is ignored. Not surprisingly, this approach does lead to notable differences in bounds. The minimized entropy over the alternative configurations of the conditioning information using the undistorted transition probabilities is 0.047 in comparison to the much smaller 0.028 that we found using our method. We view belief distortions in the transition probabilities are of particular interest in behavioral models and in models with ambiguity aversion and see this as a virtue over methods that analyze the conditional problems separately.

As we mentioned at the outset of this section, there is a substantial asset pricing literature that studies time-varying risk compensation, often appealing to high values of risk aversion. We illustrate how belief distortions can imitate large risk compensations. Thus, consider the bounds on the implied risk compensations when we restrict the risk aversion parameter to be one. We report proportional risk premium using the ex post real return on treasury bills, Rf, as our riskless benchmark in Fig. 2. To construct these results, we compute bounds on logERw−logERf by extending the approach of 3. Moment Bounds as described in SI Appendix, section E. Restricting investor risk aversion allows for belief distortions to capture the fluctuating empirical compensations for exposure to uncertainty.

• Download figure
• Open in new tab
• Download powerpoint

Fig. 2.

Proportional risk compensations computed as logERw−logERf scaled to an annualized percentage. The •s are the empirical averages and the boxes give the imputed bounds when we inflated the minimum relative entropy by 20%.

Finally, since our empirical method looks at other information from two other Fama–French excess returns, we also report bounds on other risk compensations that we included in our analysis. We convert the excess returns into returns by adding the gross returns on bonds. We report these findings in SI Appendix, section E. Again, the risk compensations are greatly reduced relative to the empirical counterpart, and in most instances they are less sensitive to conditioning information.

Putting aside the empirical debate on return predictability, we see two possible conclusions from these results. One possibility is that the statistical divergence (measured as relative entropy) for the distortions is high enough to challenge a bounded rationality view of the recursive utility model with a unitary risk aversion. The other possibility is that this divergence is defensible, in which case our dynamic implementation reveals the most statistically plausible distortions on the evolution of the dividend–price ratios. It remains a judgment call as to when the resulting statistical bounds we find here are implausible. Researchers that embrace rational expectations do not consider belief distortions, while behavioral finance researchers seldom consider the implied statistical divergence of their modeled beliefs. Neither practice uses tools for assessing statistical approximation as we have done in this paper. We view these computations as providing a valuable empirical complement to the assessment of specific models of belief distortions or ambiguity aversion. Moreover, we have noted how survey evidence can be included within our framework, and we view this as a potentially valuable extension of the illustration presented here.

5. Bounding Other Probabilities

We have motivated our analysis as a method for extracting expectation bounds for subjective beliefs or restricted “worst-case” beliefs that support valuation under ambiguity aversion. These same methods provide expectation bounds for two other probability measures that are of interest in asset valuation. These measures are the one-period risk-neutral probabilities and the long-term forward probabilities. While surveys cease to provide information about these probabilities, even sparsely observed asset values, as we assume here, are revealing. We now comment on how to apply our method for both of these applications.

6. Conclusions

In this paper, we developed methods designed to extract information on investor beliefs from data on asset prices and investor surveys. Our approach presumes an econometric model of investors or enterprises that could be misspecified under rational expectations. We illustrated how limiting the statistical discrepancy between investor beliefs and rational expectations implies bounds on investors’ expectations. Formally, we represented this relationship through a nonlinear expectation function and derived its dual representation.

Going forward, we see two types of applications of our methods. Deducing market expectations about the future from forward-looking asset prices is a common practice in both the public and private sectors. But this is typically done either informally or by targeting so-called risk-neutral probabilities that confound beliefs and risk preferences. Our method provides a formal way to compute and represent information on investor beliefs constrained by a model of risk aversion along with a measure of statistical divergence.

Alternatively, we could use our approach to provide diagnostics for model misspecification under rational expectations. The bounds we deduce will help assess alternative models of subjective beliefs or ambiguity aversion. Implied belief bounds for small or moderate restrictions on the statistical divergence can give suggestive results for model builders as to how to repair potentially misspecified models. By comparing models of subjective beliefs or ambiguity aversion supported by belief distortions to the implied bounds, applied researchers could assess whether such departures from rational expectations could be easily discerned from limited data.

Future applications of our methodology could incorporate information from survey data on investor beliefs. One approach would be to include survey data directly as additional moment conditions when constructing expectation bounds. Another approach would be to compare survey-implied expectations to expectation bounds on the corresponding variables formed without using the survey data as information. The latter approach would provide a check on how plausible the survey data are as a representation of investor beliefs used in decision making.

While we focused on constraining probabilities based on intertemporal measures of divergence, these bounds could be used formally in the design of economic policy. For instance, refs. 58 and 59 pose dynamic policy problems in which a government policymaker fails to have precise knowledge of the beliefs of the private sector when designing a prudent course of action. While these papers explore what impact this imprecision could have on policy, our work looks more systematically at how to extract credible information about the beliefs.