## New Research In

### Physical Sciences

### Social Sciences

#### Featured Portals

#### Articles by Topic

### Biological Sciences

#### Featured Portals

#### Articles by Topic

- Agricultural Sciences
- Anthropology
- Applied Biological Sciences
- Biochemistry
- Biophysics and Computational Biology
- Cell Biology
- Developmental Biology
- Ecology
- Environmental Sciences
- Evolution
- Genetics
- Immunology and Inflammation
- Medical Sciences
- Microbiology
- Neuroscience
- Pharmacology
- Physiology
- Plant Biology
- Population Biology
- Psychological and Cognitive Sciences
- Sustainability Science
- Systems Biology

# Generalized entropies and logarithms and their duality relations

Contributed by Murray Gell-Mann, September 28, 2012 (sent for review August 6, 2012)

## Abstract

For statistical systems that violate one of the four Shannon–Khinchin axioms, entropy takes a more general form than the Boltzmann–Gibbs entropy. The framework of superstatistics allows one to formulate a maximum entropy principle with these generalized entropies, making them useful for understanding distribution functions of non-Markovian or nonergodic complex systems. For such systems where the composability axiom is violated there exist only two ways to implement the maximum entropy principle, one using escort probabilities, the other not. The two ways are connected through a duality. Here we show that this duality fixes a unique escort probability, which allows us to derive a complete theory of the generalized logarithms that naturally arise from the violation of this axiom. We then show how the functional forms of these generalized logarithms are related to the asymptotic scaling behavior of the entropy.

The concept of “superstatistics” (1⇓–3) provides a formal framework for a wide class of generalizations of statistical mechanics that were introduced recently. Within this framework it is possible to formulate a maximum entropy principle, even for nonergodic or non-Markovian systems, including many complex systems. From an axiomatic point of view, nonadditive systems are characterized by the fact that the fourth Shannon–Khinchin (SK) axiom governing composability of statistical systems is violated.^{†} For systems where all four SK axioms hold, the entropy is uniquely determined as the Boltzmann–Gibbs–Shannon (BGS) entropy (4, 5), . In the case where only the first three axioms are valid (e.g., non-Markovian systems), the entropy has a more general form (6). In the thermodynamic limit, which captures the asymptotic behavior for small values of the *p*_{i}, the entropy is given by the formulawhere Γ is the incomplete gamma function and (*c*, *d*) are constants that are uniquely determined by the scaling properties of the statistical system in its thermodynamic limit. In previous work (7) we were able to show that for systems where the first three SK axioms hold, there exist only two ways to formulate a consistent maximum entropy principle. Starting with an entropy of “trace form,”the maximization condition becomes δ Φ = 0, withwhere the last two terms are the constraints. The first of the two possible approaches [Hanel–Thurner (HT) approach] (8, 9) uses a generalized entropy and the usual form of the constraint, . The other approach, suggested in Tsallis and Souza (10) (TS approach), uses a generalized entropy and a more general way to impose constraints:

*P*_{i} is a so-called escort probability and *ν* is a real number. Though in the HT case the constraint has the usual interpretation as an energy constraint, we do not attempt to give a physical interpretation of the escort probabilities. The two approaches have been shown to be connected by a duality map *: with ** (meaning applying * twice) being the identity (7). A special case of this duality has been observed in Ferri et al. (11).

Entropies can be conveniently formulated using their associated generalized logarithms. We first specify the space ℒ of proper generalized logarithms Λ ∈ ℒ. We consider a generalized logarithm to be proper if the following properties hold: (*i*) Λ is a differentiable function Λ: ℝ_{+} → ℝ. This is necessary for a finite second derivative of the entropy; (*ii*) Λ is monotonically increasing, which is a consequence of the second SK axiom; (*iii*) Λ(1) = 0, which captures the requirement that the entropy of single-state systems is 0; and (*iv*) Λ′(1) = 1, is needed to fix the units of entropy.

In both approaches (HT and TS) there exist proper generalized logarithms Λ_{HT} and Λ_{TS} such thatandwith *x*_{0} a constant. If both approaches predict the same distribution function as a result of the maximization of Eq. **3**, then it can be shown that the two entropic functions *s*_{HT} and *s*_{TS} are one-to-one related by

In the following, we set *k* = 1; this can be achieved either by choosing physical units accordingly, or by simply absorbing *k* into *ν*, so that *ν* becomes a dimensionless parameter.

The full implication of Eq. **7**, which is related to the essence of this paper, can be summarized as follows. The statistical properties of a physical system—for instance, a superstatistical system as discussed in Hanel et al. (7)—uniquely determine the entropy *S*_{HT}. A priori, there exists a spectrum of TS entropies, *S*_{TS,ν}, whose boundaries are determined by the properties of the generalized logarithm associated with *S*_{HT}. Moreover, these properties determine a particular value *ν**, so that and *S*_{HT} become a pair of dual entropies. This unique duality allows us to derive a complete theory of generalized logarithms naturally arising as a consequence of the fourth SK axiom being violated. We present a full understanding of how the TS and HT approaches are interrelated and derive the most general form of families of generalized logarithms that are compatible with a maximum entropy principle and the first three SK axioms. Finally, we demonstrate how these logarithms can be classified according to their asymptotic scaling properties, following the results presented in Hanel and Thurner (6).

## Duality

In contrast to the images of generalized logarithms, which need not span ℝ completely and can differ from one another, the domain of generalized logarithms themselves is always all of ℝ_{+}. For these reasons, one may classify generalized logarithms according to the minimum and maximum values of their images and consider the group *G* of order-preserving automorphisms on ℝ_{+} that keep an infinitesimal neighborhood of 1 ∈ ℝ_{+} invariant, as the means to generate these classes. In the following, we call the elements *g* of this automorphism group scale transformations. More precisely, *g* ∈ is a scale transformation if *g* is differentiable and maps ℝ_{+} to ℝ_{+} one-to-one, *g*′ > 0, *g*(1) = 1, and *g*′(1) = 1. From these properties it follows that *g*(0) = 0 and lim_{x→∞}*g*(*x*) = ∞. Finally, we use the notation *f* ○ *g*(*x*) = *f* (*g*(*x*)).

Scale transformations leave the image of a generalized logarithm invariant, which allows us to parameterize classes in the following way. Given a proper generalized logarithm Λ ∈ ℒ, we write for its maximum and minimum valuesand define two functionalswhich associate numbers *ν*_{+} and *ν*_{−} to any Λ. For their sum we write *ν** = *ν*_{+} + *ν*_{−}. Next, we define sets of proper generalized logarithms,

Members of all have the same maximum and minimum values. In fact, the are exactly the equivalence classes in ℒ generated by *G*: Two generalized logarithms Λ^{(A)} and Λ^{(B)} are considered equivalent if there exists a scale transformation *g* ∈ *G* such that Λ^{(B)} = Λ^{(A)} ○ *g*. The space of generalized logarithms can be written as the union of these sets, .

With these definitions we now analyze the relation between the HT and TS approaches. Assuming that Λ_{HT} is given, Eq. **7** implies

*T*_{ν} is a shift operator with the property *T*_{ν} ○ *T*_{μ} = *T*_{ν}_{+μ}. We have of course Λ_{TS,0} = Λ_{HT}. The fact that Λ_{HT} is a proper generalized logarithm does not imply that Λ_{TS} is also proper for all choices of *ν*.

In fact, given that Λ ∈ ℒ, it can be shown (*SI Materials and Methods*) that *T*_{ν} ○ Λ ∈ ℒ if and only if *ν*_{−}[Λ] ≤ *ν* ≤ *ν*_{+}[Λ]. Moreover, for and for *T*_{ν} ○ Λ being a proper generalized logarithm, it follows that . As a consequence Λ_{TS,ν} (*x*) = *T*_{ν} ○ Λ_{HT} (*x*) is proper only for *ν*_{−}[Λ_{HT}] ≤ *ν* ≤ *ν*_{+}[Λ_{HT}], and

This equation does not uniquely determine a duality relation * on ℒ, yet by imposing the condition that * commute with scale transformations *g* ∈ , it can be shown (*SI Materials and Methods*) that * is given bywith the property

Thus, for each Λ_{HT} there exists a unique value *ν** = *ν*_{+}[Λ_{HT}] + *ν*_{−}[Λ_{HT}] such that Λ_{TS,ν*} is a proper generalized logarithm. The duality map * gives . Furthermore, because * and *g* commute ((Λ ○ *g*)* = Λ* ○ *g*), any proper generalized logarithm Λ can be decomposed into a specific representative , and a scale transformation *g*, so thatwhich implies that any Λ_{HT} or Λ_{TS,ν} can be decomposed in this way, and that the dual logarithms Λ_{HT} and transform identically under scale transformations.

## Functional Form of the Generalized Logarithms

Eq. **14** implies the existence of transformations that map members of to members of . These maps can be used to represent the duality * on specific families . Λ(*x*) → −Λ(1/*x*) is exactly such a map, because . The same holds for , which allows us to construct with the properties

By using Eq. **13** and inserting into Eq. **7**, we get

This equation may have many solutions , but we can restrict ourselves to finding a particular one; all of the others can be obtained by scale transformations, which is seen as follows: Suppose and are both solutions of Eq. **17**; then according to Eq. **15** for any pair (*ν*_{+}, *ν*_{−}) there exists a scale transformation such that . Because must leave Eq. **16** invariant (this is not the case for arbitrary scale transformations *g* ∈ ), these scale transformations have two properties. The first property is , which makes them members of a subgroup of all possible scale transformations *g* ∈ . The second property is and follows from the fact that * commutes with scale transformations.

A particular solution of Eq. **17** is given bywith *h*: ℝ → [−1, 1] a continuous, monotonically increasing, odd function, with lim_{x→∞} *h*(*x*) = 1 and *h*′(0) = 1. It can easily be verified that this solution has all of the required properties: is a proper logarithm with (correct minimum and maximum), , Λ_{ν,−ν}(*x*) = *h*(*ν* log(*x*))/*ν* is self-dual, and .

The above argument means that we can generate a specific family of logarithms , following Eq. **16**, by choosing one particular function *h* [e.g., *h*(*x*) = tanh(*x*)] and then using scale transformations to reach all other possibilities. In particular, some family with the property can be reached by a family of scale transformations , where are generalized exponential functions (inverse functions of logarithms). Moreover, if also follows Eq. **16**, then .

The family of dual logarithms discussed in Hanel et al. (7) is obtained in the framework presented here by setting either *ν*_{+} = 0 or *ν*_{−} = 0. These classes correspond to logarithms that are unbounded either from below or from above, whereas the duality maps . Moreover, in Hanel et al. (7), only pairs of dual logarithms have been considered such that Λ*(*x*) = −Λ(1/*x*), and the part that scale transformations play in the unique definition of * had not yet been described.

We are now in a position to understand all observable distribution functions emerging from the two approaches in terms of a single two-parameter family of generalized logarithms and a scale transformation. This result now raises the question of how is related to the two-parameter logarithms associated with the (*c*, *d*) entropies in Eq. **1** (6), and will further clarify the role of the scale transformations.

## Logarithm and (*c*, *d*) Entropy

Generalized entropies can be classified with respect to their asymptotic scaling behavior in terms of two scaling exponents, *c* and *d*, where 0 < *c* ≤ 1 and *d* is a real number (6); they are obtained from the scaling relationswhere *s* is the summand in Eq. **2**. Using Eq. **19**, de l’Hôpital’s rule, and the fact that *s*′(*x*) = −Λ(*x*), we find the exponents (*c*, *d*) for a given Λ ∈ ℒwhere we represent Λ as . In this way we get the dependence of (*c*, *d*) as a function of (*ν*_{+}, *ν*_{−}), *h*, and the scale transformation *g*. We first compute the asymptotic properties of *h* and *g*, defining the exponents *c*_{h,g} and *d*_{h,g} bywhere *φ*_{h,g} = 1 + *h* ○log ○*g*(*x*). Note that log ○*g* ∈ ℒ_{0,0}. By defining Λ_{0} ≡ log ○*g*, we compute its scaling exponents *c*_{0} and *d*_{0}

With these preparations one can derive the resultswhich demonstrate clearly that, given a fixed *h*, *c* is controlled by *ν*_{−}, (for *ν*_{+} = 0 and *c*_{0} = 1) and *d* is determined by the scale transformation.

## Examples

### Example 1.

#### A simple choice for h:

For example, fix *h*(*x*) = tanh(*x*). From Eq. **18** we get for the generalized logarithm

The associated generalized exponential (inverse of the generalized logarithm) is

### Example 2.

#### Power laws:

By setting *h*(*x*) = tanh(*x*) and *ν*_{+} = 0, we get from Eq. **24** the so-called *q*-logarithm, with , where 0 ≤ *q* = 1 − *ν*_{−} ≤ 1. The dual is , and we recover the well-known duality for *q*-logarithms. It is also well known that log_{q} results from the use of escort distributions (12, 13, 10), whereas log_{2−q} is a natural result of the HT approach (8, 9).

An example of a generalized logarithm that is not a power is obtained by taking in Eq. **24**. One obtains , with the dual .

### Example 3.

#### Scale transformations:

Any proper generalized logarithm can be written as a composition of a representative logarithm from Eq. **18** and a scale transformation, with . For example, pick Λ_{0,0}(*x*) = log(*x*), and , where *d* > 0 is a parameter of *g*. The generalized logarithm then becomes

The associated generalized exponential is a stretched exponential, , which is the known result for (*c*, *d*) entropies with *c* = 1 and *d* > 0 (6, 14).

### Example 4.

#### Different choices for h:

Suppose that a physical situation demands a specific Λ, and two observers, *A* and *B*, choose to represent Λ differently. Observer *A* chooses to represent Λ, so that , and observer *B* chooses *h*_{B}(*x*) = tanh(*x*) to represent . Then and can only differ by a scale transformation with , and it follows that . For the particular functions *h*_{A} and *h*_{B} we have chosen, we get .

## Discussion

By studying the two types of entropies that are related to the two possible ways to formulate a maximum entropy principle for systems that explicitly violate the fourth SK axiom, we find that there exists a unique duality that relates the two entropies. Consequently, thermodynamic properties derived from those two entropies will also be related through the duality. We show that the maximum and minimum of Λ_{HT} determine a unique value *ν** for which is the dual of Λ_{HT}. In this way it is possible for an object such as Λ_{HT}, which does not explicitly carry an index *ν*, to become dual to an object that does, such as Λ_{TS,ν}. The existence of this duality opens the way to characterizing all possible generalized logarithms as compositions of a specific functional form and scale transformations *g*. We derive the explicit form of and show that these logarithms are one-to-one related to two asymptotic scaling exponents (*c*, *d*) that allow one to characterize strongly nonergodic or non-Markovian systems in their thermodynamic limit (6). *ν*_{−} is shown to be directly related to *c*, and the form of the scale transformation *g* determines *d*. In summary, we provide a complete theory of all generalized logarithms that can arise as a consequence of the violation of the fourth SK axiom.

## Acknowledgments

R.H. and S.T. thank the Santa Fe Institute and M.G.-M. thanks the Aspen Center for Physics for their hospitality. Support for this work was provided in part by National Science Foundation Grant 1066293, Insight Venture Partners, and the Bryan J. and June B. Zwan Foundation (M.G.-M.).

## Footnotes

- ↵
^{1}To whom correspondence should be addressed. E-mail: mgm{at}santafe.edu.

Author contributions: R.H., S.T., and M.G.-M. designed research, performed research, and wrote the paper.

The authors declare no conflict of interest.

This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.1073/pnas.1216885109/-/DCSupplemental.

↵

^{†}Shannon–Khinchin axioms: (*i*) Entropy is a continuous function of the probabilities*p*_{i}only, i.e.,*s*should not explicitly depend on any other parameters. (*ii*) Entropy is maximal for the equidistribution*p*_{i}= 1/*W*; from this, the concavity of*s*follows. (*iii*) Adding a state*W*+ 1 to a system with*p*_{W}_{+}_{1}= 0 does not change the entropy of the system; from this,*s*(0) = 0 follows. (*iv*) Entropy of a system composed of two subsystems,*A*and*B*, is*S*(*A*+*B*) =*S*(*A*) +*S*(*B*|*A*).

Freely available online through the PNAS open access option.

## References

- ↵
- ↵
- ↵
- ↵
- Shannon CE

*Bell Syst Tech J*27(3):379-423, 623-656. - ↵
- Khinchin AI

- ↵
- ↵
- Hanel R,
- Thurner S,
- Gell-Mann M

- ↵
- ↵
- Thurner S,
- Hanel R

- ↵
- ↵
- Ferri GL,
- Martinez S,
- Plastino A

- ↵
- ↵
- Tsallis C

- ↵

## Citation Manager Formats

## Sign up for Article Alerts

## Article Classifications

- Physical Sciences
- Physics