Measurement invariance explains the universal law of generalization for psychological perception

Basic Problem and Notation

Chater and Vitányi (ref. 2, p. 346) state the law as “the probability of perceiving similarity or analogy between two items, a and b, is a negative exponential function of the distance d(a,b) between them in an internal psychological space.”

Let the notation P(Rb|Sa) describe the probability of a positive response, Rb, to the event b, given an initial stimulus, Sa, by the event a. A positive response expresses the perceived similarity of b to a, which may also be thought of as expressing the generalization that b and a belong to the same category.

The goal here is to understand how the perceived similarity of b to a, observed as Rb|Sa, translates into a continuous psychological measurement scale, Tb|a, so thatP(Rb|Sa)≡f(Tb|a)[1]for a suitably defined mapping Rb|Sa↦Tb|a and probability distribution function, f. We seek the characteristics of the mapping and the associated function, f.

Invariant Properties of Measurement

There are many different suggestions in the literature for how to define a perceptual scale, Tb|a (1, 2, 4). I focus on the minimal properties that any reasonable measurement scale must have, rather than on detailed assumptions motivated by external theories (5⇓–7). I express the minimal properties as simple invariances. Before listing the invariances, consider two simple examples.

First, suppose we wish to analyze the perception of temperature for event b, given that event a is at the freezing point for water. If we choose to measure the temperature on the Celsius scale, then Ta|a=0 and Tb|a=C. It would make sense to assume that perceptual generalization would be identical if we assigned numerical values on a Fahrenheit scale, T̃, which we obtain by T̃b|a=32+1.8Tb|a.

Second, suppose we wish to measure the perception of separation between two potentially dangerous prey items, such as noxious butterflies (4, 8). We begin by exposing a noxious butterfly, a, to a predator. After the predator tastes butterfly a, we then expose butterfly b to the same predator. For the exposure to b, we measure the tendency for the predator to attack the potential prey item. Data may include the directions of movements relative to the butterfly, attacks per minute, or the probability of attack over repeated experiments. We now wish to find a scale, Tb|a, that is a function of the data we have for the response to various butterflies, b, relative to an initial stimulus butterfly, a.

However we choose that scale, it makes sense to suppose that the information in Tb|a about the perceptual separation between b and a is the same as the information in α+βTb|a for some constants α and β. If that were not so, it would be equivalent to saying that the analogs of Celsius and Fahrenheit scalings would provide different information about the perceptual separation between the two butterflies.

For example, we may wish to set Ta|a=0 to describe a zero separation between identical butterflies, or we may wish to let Ta|a express the amount of the baseline predator perception of the separation between identical stimuli. In either case, our scale T should contain the same information with respect to the probabilities of response given in Eq. 1. Here, similarity associates with the probability of avoidance response. We may also wish to express our scale standardized with respect to a unit response, Tb*|a, to b*, or with respect to a unit response, Tb†|a, to b†. The constant multiplications required to transform between units of measure should not alter the information in the perceptual scale, T, about the probabilities of response.

Affine and Rotational Invariance

In other words, the way in which we measure perceptual distance between two stimuli should be independent of a shift and stretch of the scale by constant values. Formally, the scale should be shift invariant with respect to any constant, α, such thatf(Tb|a)=kαf(Tb|a+α)[2]for some constant of proportionality, kα. The scale should also be stretch invariant to any constant, β, such thatf(λTb|a)=f(λββTb|a),[3]for which I show below that λ=λββ is an invariant constant that is conserved in any particular application, set by the fact that 1/λ is the average value on the perceptual scale for positive responses to varying events b for a given stimulus a.

Thus, the scale Tb|a has the property that the associated probability pattern is invariant to the affine transformation of shift and stretch, Tb|a↦α+βTb|a. I will show that affine invariance by itself determines the exponential form for the universal law of generalization in perception.

In some cases, it makes sense to assume that the perceptual scale should also obey rotational invariance, such that the Pythagorean partitionTb|a=y12(θ)+y22(θ)[4]splits the measurement into components that add invariantly to Tb|a for any value of the parameter θ. The invariant quantity Tb|a defines a circle in the (y1,y2) plane with a conserved radius Rb|a=Tb|a that is invariant to θ, the angle of rotation around the circle, circumscribing a conserved area πRb|a2=πTb|a.

Rotational invariance partitions a conserved quantity into additive components, for which the order may be exchanged without altering the invariant quantity. When rotational invariance holds, the universal law takes on a Gaussian form, which we will see to be a special case of the general exponential form.

The following sections develop the three invariances of shift, stretch, and rotation. I show that essentially all of the common properties of perceptual generalization follow from these invariances. The analysis here briefly summarizes the detailed development described in Frank (9). The novelty in this article concerns the simple understanding of widely observed psychological patterns.

Shift Invariance Implies the Exponential Form

To simplify notation, denote the perceptual scale by x≡Tb|a and the associated probability distribution by f(x)≡f(Tb|a). If we assume that the functional form for the probability distribution, f, is invariant to a constant shift of the perceptual scale, x+α, then by the conservation of total probability∫k0f(x) dx=∫kαf(x+α) dx=1[5]holds for any magnitude of the shift, α, in which the proportionality constant, kα, changes with the magnitude of the shift, α, independently of the value of x, to satisfy the conservation of total probability.

From this equality for total probability, which holds for any shift α by adjustment of the constant, kα, the condition for x≡Tb|a to be a shift-invariant scale is equivalent tof(x+α)=καf(x),[6]in which κα depends only on α and is independent of x. Because the invariance holds for any shift, α, it must hold for an infinitesimal shift, α=ϵ. We can write the Taylor series expansion for an infinitesimal shift asf(x+ϵ)=f(x)+ϵf′(x)=κϵf(x),with κϵ=1−λϵ, because ϵ is small and independent of x, and κ0=1. Thus,f′(x)=−λf(x)is a differential equation with solutionf(x)=ke−λx,[7]in which k is determined by the conservation of total probability. When the perceptual scale ranges over positive values, x>0, then k=λ.

The assumption that a perceptual scale must be shift invariant is, by itself, sufficient to explain the exponential form of the universal law of generalization.

The Exponential Form Implies Shift Invariance

The previous section showed that if the perceptual scale, x, is shift invariant, then the exponential form of the universal law of generalization follows. This section shows that if the universal law of generalization takes on the exponential form, then the underlying perceptual scale must be shift invariant. Thus, shift invariance is necessary and sufficient for the exponential form. Any assumptions about the perceptual scale beyond shift invariance must be superfluous with respect to the exponential form.

Begin with the assumption of the exponential form in Eq. 7 and write the consequence of a shift of the scale x by α asf(x+α)=kαe−λ(x+α)=kαe−λαe−λx=ke−λxin which kα=keλα because the constant multiplier of the exponential must be chosen to satisfy the conservation of total probability, in other words, to normalize the total probability to be one. Thus, the exponential form implies shift invariance of the perceptual scale, x.

Stretch Invariance and Rate of Perceptual Change

If we assume that the perceptual scale is defined for positive values, x>0, then the average value of λx is always one, because∫0∞λxf(x)dx=λ∫0∞λxe−λxdx=1.Thus, for average value, x¯, the value of λ is 1/x¯. We can think of x¯ as the average discrimination of various events, b, relative to an initial stimulus, a, in which the set of events b corresponds to a uniform continuum along the perceptual scale, x.

It makes sense to assume that the average discrimination would not change if we arbitrarily multiplied our numerical scale for perception, x, by a constant, β. The conservation of average value and stretch invariance are equivalent, becauseλ∫0∞λββxe−λββxdx=1when we allow λβ to adjust to satisfy the conservation of average value so that λ=λββ or, equivalently, we assume stretch invariance of the scale, x≡Tb|a.

The constant λ=1/x¯ can be thought of as the slope or rate of change in the logarithm of discrimination, becauselog⁡f(x)=−λx.Stretch invariance, or the conservation of average value, is sufficient to set the rate of change in the logarithm of discrimination. The average value of −logf(x) is a common definition of information or entropy and is related to many interpretations in terms of information theory (4, 10).

Rotational Invariance and Gaussian Patterns

The scale, x, measures the perceptual difference between two entities or events. In some cases, the total difference, x, depends on the perceived differences along several distinct underlying dimensions. With two underlying dimensions, we may writex=z1(θ)+z2(θ).For a particular value of x, the parameter θ describes all of the combinations of the two underlying dimensions that add invariantly to x. If we let x=r2 and let the dependence of z on θ be implicit, we can write the prior expression equivalently asx=r2=z12+z22,which defines a circle with coordinates along the positive and negative values of (z1,z2), with a constant radius r that is rotationally invariant with respect to the parametric angle, θ. Traditionally, one uses yi=zi, so that the radius, r, of a sphere has the familiar definition of a Euclidean distancer2=∑yi2.For each radial value, r=x, we can write {yi(θ)} as the sets indexed by the parameter θ for which the individual dimensional measures combine to the same invariant radius. If the angles of rotation with equivalent radius occur with equal probability or without prior bias, then radial values are rotationally invariant with respect to probability or prior likelihood.

I now show that rotational invariance leads to the Gaussian pattern as a special case of the general exponential form. In the exponential form derived in earlier sections, λx described the stretch-invariant perceptual scale. To express that scale in terms of a rotationally invariant radial measure, r, we note that x=r2 and we let λ=πv2. Thus, we can write the stretch-invariant incremental perceptual measure asλdx=πv2dr2=2πv2rdr.The general exponential form isf(x)dx=λe−λxdx=2πv2re−πv2r2dr.At a given radius, vr, if, by rotational invariance, all combinations of values for the underlying measurement dimensions occur without bias or prior information, then the total probability in a radial increment, vdr, is spread uniformly over the circumferential path with length 2πvr.

A radial vector intersects a fraction of the total probability density in the circumferential path in proportion to 1/2πvr. Thus, the probability along an increment vdr of the radial vector is(1/2πvr)f(x)dx=ve−πv2r2dr=ve−λr2dr,invariantly with respect to the angle of orientation of the radial vector. This expression is the Gaussian distribution, with r2 as the squared deviation from the mean or central location and with parameters commonly written as λ=1/2σ2 and v=1/2πσ2 for variance σ2. The variance is simply the average value of the squared radial deviations, r2=x.

We can also write the Gaussian in terms of the standard perceptual scale, x, asg(x)dx=ve−λxdx.When we consider the standard perceptual scale, x, with respect to the incremental square-root scale, dx, we obtain a Gaussian. The incremental square-root scale makes sense when we consider x as an aggregate measure of the sum of underlying perceptual dimensions. Each dimension naturally takes on a square-root scaling relative to the invariant total distance, because of the Euclidean measure of squared distance as the sum of squares along each dimension.