New Research In
Physical Sciences
Social Sciences
Featured Portals
Articles by Topic
Biological Sciences
Featured Portals
Articles by Topic
- Agricultural Sciences
- Anthropology
- Applied Biological Sciences
- Biochemistry
- Biophysics and Computational Biology
- Cell Biology
- Developmental Biology
- Ecology
- Environmental Sciences
- Evolution
- Genetics
- Immunology and Inflammation
- Medical Sciences
- Microbiology
- Neuroscience
- Pharmacology
- Physiology
- Plant Biology
- Population Biology
- Psychological and Cognitive Sciences
- Sustainability Science
- Systems Biology
Generalized entropies and logarithms and their duality relations
Contributed by Murray Gell-Mann, September 28, 2012 (sent for review August 6, 2012)

Abstract
For statistical systems that violate one of the four Shannon–Khinchin axioms, entropy takes a more general form than the Boltzmann–Gibbs entropy. The framework of superstatistics allows one to formulate a maximum entropy principle with these generalized entropies, making them useful for understanding distribution functions of non-Markovian or nonergodic complex systems. For such systems where the composability axiom is violated there exist only two ways to implement the maximum entropy principle, one using escort probabilities, the other not. The two ways are connected through a duality. Here we show that this duality fixes a unique escort probability, which allows us to derive a complete theory of the generalized logarithms that naturally arise from the violation of this axiom. We then show how the functional forms of these generalized logarithms are related to the asymptotic scaling behavior of the entropy.
The concept of “superstatistics” (1⇓–3) provides a formal framework for a wide class of generalizations of statistical mechanics that were introduced recently. Within this framework it is possible to formulate a maximum entropy principle, even for nonergodic or non-Markovian systems, including many complex systems. From an axiomatic point of view, nonadditive systems are characterized by the fact that the fourth Shannon–Khinchin (SK) axiom governing composability of statistical systems is violated.† For systems where all four SK axioms hold, the entropy is uniquely determined as the Boltzmann–Gibbs–Shannon (BGS) entropy (4, 5), . In the case where only the first three axioms are valid (e.g., non-Markovian systems), the entropy has a more general form (6). In the thermodynamic limit, which captures the asymptotic behavior for small values of the pi, the entropy is given by the formula
where Γ is the incomplete gamma function and (c, d) are constants that are uniquely determined by the scaling properties of the statistical system in its thermodynamic limit. In previous work (7) we were able to show that for systems where the first three SK axioms hold, there exist only two ways to formulate a consistent maximum entropy principle. Starting with an entropy of “trace form,”
the maximization condition becomes δ Φ = 0, with
where the last two terms are the constraints. The first of the two possible approaches [Hanel–Thurner (HT) approach] (8, 9) uses a generalized entropy and the usual form of the constraint,
. The other approach, suggested in Tsallis and Souza (10) (TS approach), uses a generalized entropy and a more general way to impose constraints:
Pi is a so-called escort probability and ν is a real number. Though in the HT case the constraint has the usual interpretation as an energy constraint, we do not attempt to give a physical interpretation of the escort probabilities. The two approaches have been shown to be connected by a duality map *: with ** (meaning applying * twice) being the identity (7). A special case of this duality has been observed in Ferri et al. (11).
Entropies can be conveniently formulated using their associated generalized logarithms. We first specify the space ℒ of proper generalized logarithms Λ ∈ ℒ. We consider a generalized logarithm to be proper if the following properties hold: (i) Λ is a differentiable function Λ: ℝ+ → ℝ. This is necessary for a finite second derivative of the entropy; (ii) Λ is monotonically increasing, which is a consequence of the second SK axiom; (iii) Λ(1) = 0, which captures the requirement that the entropy of single-state systems is 0; and (iv) Λ′(1) = 1, is needed to fix the units of entropy.
In both approaches (HT and TS) there exist proper generalized logarithms ΛHT and ΛTS such thatand
with x0 a constant. If both approaches predict the same distribution function
as a result of the maximization of Eq. 3, then it can be shown that the two entropic functions sHT and sTS are one-to-one related by
In the following, we set k = 1; this can be achieved either by choosing physical units accordingly, or by simply absorbing k into ν, so that ν becomes a dimensionless parameter.
The full implication of Eq. 7, which is related to the essence of this paper, can be summarized as follows. The statistical properties of a physical system—for instance, a superstatistical system as discussed in Hanel et al. (7)—uniquely determine the entropy SHT. A priori, there exists a spectrum of TS entropies, STS,ν, whose boundaries are determined by the properties of the generalized logarithm associated with SHT. Moreover, these properties determine a particular value ν*, so that and SHT become a pair of dual entropies. This unique duality allows us to derive a complete theory of generalized logarithms naturally arising as a consequence of the fourth SK axiom being violated. We present a full understanding of how the TS and HT approaches are interrelated and derive the most general form of families of generalized logarithms that are compatible with a maximum entropy principle and the first three SK axioms. Finally, we demonstrate how these logarithms can be classified according to their asymptotic scaling properties, following the results presented in Hanel and Thurner (6).
Duality
In contrast to the images of generalized logarithms, which need not span ℝ completely and can differ from one another, the domain of generalized logarithms themselves is always all of ℝ+. For these reasons, one may classify generalized logarithms according to the minimum and maximum values of their images and consider the group G of order-preserving automorphisms on ℝ+ that keep an infinitesimal neighborhood of 1 ∈ ℝ+ invariant, as the means to generate these classes. In the following, we call the elements g of this automorphism group scale transformations. More precisely, g ∈ is a scale transformation if g is differentiable and maps ℝ+ to ℝ+ one-to-one, g′ > 0, g(1) = 1, and g′(1) = 1. From these properties it follows that g(0) = 0 and limx→∞g(x) = ∞. Finally, we use the notation f ○ g(x) = f (g(x)).
Scale transformations leave the image of a generalized logarithm invariant, which allows us to parameterize classes in the following way. Given a proper generalized logarithm Λ ∈ ℒ, we write for its maximum and minimum valuesand define two functionals
which associate numbers ν+ and ν− to any Λ. For their sum we write ν* = ν+ + ν−. Next, we define sets of proper generalized logarithms,
Members of all have the same maximum and minimum values. In fact, the
are exactly the equivalence classes in ℒ generated by G: Two generalized logarithms Λ(A) and Λ(B) are considered equivalent if there exists a scale transformation g ∈ G such that Λ(B) = Λ(A) ○ g. The space of generalized logarithms can be written as the union of these sets,
.
With these definitions we now analyze the relation between the HT and TS approaches. Assuming that ΛHT is given, Eq. 7 implies
Tν is a shift operator with the property Tν ○ Tμ = Tν+μ. We have of course ΛTS,0 = ΛHT. The fact that ΛHT is a proper generalized logarithm does not imply that ΛTS is also proper for all choices of ν.
In fact, given that Λ ∈ ℒ, it can be shown (SI Materials and Methods) that Tν ○ Λ ∈ ℒ if and only if ν−[Λ] ≤ ν ≤ ν+[Λ]. Moreover, for and for Tν ○ Λ being a proper generalized logarithm, it follows that
. As a consequence ΛTS,ν (x) = Tν ○ ΛHT (x) is proper only for ν−[ΛHT] ≤ ν ≤ ν+[ΛHT], and
This equation does not uniquely determine a duality relation * on ℒ, yet by imposing the condition that * commute with scale transformations g ∈ , it can be shown (SI Materials and Methods) that * is given by
with the property
Thus, for each ΛHT there exists a unique value ν* = ν+[ΛHT] + ν−[ΛHT] such that ΛTS,ν* is a proper generalized logarithm. The duality map * gives . Furthermore, because * and g commute ((Λ ○ g)* = Λ* ○ g), any proper generalized logarithm Λ can be decomposed into a specific representative
, and a scale transformation g, so that
which implies that any ΛHT or ΛTS,ν can be decomposed in this way, and that the dual logarithms ΛHT and
transform identically under scale transformations.
Functional Form of the Generalized Logarithms
Eq. 14 implies the existence of transformations that map members of to members of
. These maps can be used to represent the duality * on specific families
. Λ(x) → −Λ(1/x) is exactly such a map, because
. The same holds for
, which allows us to construct
with the properties
By using Eq. 13 and inserting into Eq. 7, we get
This equation may have many solutions , but we can restrict ourselves to finding a particular one; all of the others can be obtained by scale transformations, which is seen as follows: Suppose
and
are both solutions of Eq. 17; then according to Eq. 15 for any pair (ν+, ν−) there exists a scale transformation
such that
. Because
must leave Eq. 16 invariant (this is not the case for arbitrary scale transformations g ∈
), these scale transformations have two properties. The first property is
, which makes them members of a subgroup
of all possible scale transformations g ∈
. The second property is
and follows from the fact that * commutes with scale transformations.
A particular solution of Eq. 17 is given bywith h: ℝ → [−1, 1] a continuous, monotonically increasing, odd function, with limx→∞ h(x) = 1 and h′(0) = 1. It can easily be verified that this solution has all of the required properties:
is a proper logarithm with
(correct minimum and maximum),
, Λν,−ν(x) = h(ν log(x))/ν is self-dual, and
.
The above argument means that we can generate a specific family of logarithms , following Eq. 16, by choosing one particular function h [e.g., h(x) = tanh(x)] and then using scale transformations to reach all other possibilities. In particular, some family
with the property
can be reached by a family of scale transformations
, where
are generalized exponential functions (inverse functions of logarithms). Moreover, if
also follows Eq. 16, then
.
The family of dual logarithms discussed in Hanel et al. (7) is obtained in the framework presented here by setting either ν+ = 0 or ν− = 0. These classes correspond to logarithms that are unbounded either from below or from above, whereas the duality maps . Moreover, in Hanel et al. (7), only pairs of dual logarithms have been considered such that Λ*(x) = −Λ(1/x), and the part that scale transformations play in the unique definition of * had not yet been described.
We are now in a position to understand all observable distribution functions emerging from the two approaches in terms of a single two-parameter family of generalized logarithms and a scale transformation. This result now raises the question of how
is related to the two-parameter logarithms associated with the (c, d) entropies in Eq. 1 (6), and will further clarify the role of the scale transformations.
Logarithm and (c, d) Entropy
Generalized entropies can be classified with respect to their asymptotic scaling behavior in terms of two scaling exponents, c and d, where 0 < c ≤ 1 and d is a real number (6); they are obtained from the scaling relationswhere s is the summand in Eq. 2. Using Eq. 19, de l’Hôpital’s rule, and the fact that s′(x) = −Λ(x), we find the exponents (c, d) for a given Λ ∈ ℒ
where we represent Λ as
. In this way we get the dependence of (c, d) as a function of (ν+, ν−), h, and the scale transformation g. We first compute the asymptotic properties of h and g, defining the exponents ch,g and dh,g by
where φh,g = 1 + h ○log ○g(x). Note that log ○g ∈ ℒ0,0. By defining Λ0 ≡ log ○g, we compute its scaling exponents c0 and d0
With these preparations one can derive the resultswhich demonstrate clearly that, given a fixed h, c is controlled by ν−, (for ν+ = 0 and c0 = 1) and d is determined by the scale transformation.
Examples
Example 1.
A simple choice for h:
For example, fix h(x) = tanh(x). From Eq. 18 we get for the generalized logarithm
The associated generalized exponential (inverse of the generalized logarithm) is
Example 2.
Power laws:
By setting h(x) = tanh(x) and ν+ = 0, we get from Eq. 24 the so-called q-logarithm, with , where 0 ≤ q = 1 − ν− ≤ 1. The dual is
, and we recover the well-known duality for q-logarithms. It is also well known that logq results from the use of escort distributions (12, 13, 10), whereas log2−q is a natural result of the HT approach (8, 9).
An example of a generalized logarithm that is not a power is obtained by taking in Eq. 24. One obtains
, with the dual
.
Example 3.
Scale transformations:
Any proper generalized logarithm can be written as a composition of a representative logarithm from Eq. 18 and a scale transformation, with . For example, pick Λ0,0(x) = log(x), and
, where d > 0 is a parameter of g. The generalized logarithm then becomes
The associated generalized exponential is a stretched exponential, , which is the known result for (c, d) entropies with c = 1 and d > 0 (6, 14).
Example 4.
Different choices for h:
Suppose that a physical situation demands a specific Λ, and two observers, A and B, choose to represent Λ differently. Observer A chooses to represent Λ, so that
, and observer B chooses hB(x) = tanh(x) to represent
. Then
and
can only differ by a scale transformation
with
, and it follows that
. For the particular functions hA and hB we have chosen, we get
.
Discussion
By studying the two types of entropies that are related to the two possible ways to formulate a maximum entropy principle for systems that explicitly violate the fourth SK axiom, we find that there exists a unique duality that relates the two entropies. Consequently, thermodynamic properties derived from those two entropies will also be related through the duality. We show that the maximum and minimum of ΛHT determine a unique value ν* for which is the dual of ΛHT. In this way it is possible for an object such as ΛHT, which does not explicitly carry an index ν, to become dual to an object that does, such as ΛTS,ν. The existence of this duality opens the way to characterizing all possible generalized logarithms as compositions of a specific functional form
and scale transformations g. We derive the explicit form of
and show that these logarithms are one-to-one related to two asymptotic scaling exponents (c, d) that allow one to characterize strongly nonergodic or non-Markovian systems in their thermodynamic limit (6). ν− is shown to be directly related to c, and the form of the scale transformation g determines d. In summary, we provide a complete theory of all generalized logarithms that can arise as a consequence of the violation of the fourth SK axiom.
Acknowledgments
R.H. and S.T. thank the Santa Fe Institute and M.G.-M. thanks the Aspen Center for Physics for their hospitality. Support for this work was provided in part by National Science Foundation Grant 1066293, Insight Venture Partners, and the Bryan J. and June B. Zwan Foundation (M.G.-M.).
Footnotes
- ↵1To whom correspondence should be addressed. E-mail: mgm{at}santafe.edu.
Author contributions: R.H., S.T., and M.G.-M. designed research, performed research, and wrote the paper.
The authors declare no conflict of interest.
This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.1073/pnas.1216885109/-/DCSupplemental.
↵†Shannon–Khinchin axioms: (i) Entropy is a continuous function of the probabilities pi only, i.e., s should not explicitly depend on any other parameters. (ii) Entropy is maximal for the equidistribution pi = 1/W; from this, the concavity of s follows. (iii) Adding a state W + 1 to a system with pW + 1 = 0 does not change the entropy of the system; from this, s(0) = 0 follows. (iv) Entropy of a system composed of two subsystems, A and B, is S(A + B) = S(A) + S(B|A).
Freely available online through the PNAS open access option.
References
- ↵
- ↵
- ↵
- ↵
- Shannon CE
- ↵
- Khinchin AI
- ↵
- ↵
- Hanel R,
- Thurner S,
- Gell-Mann M
- ↵
- ↵
- Thurner S,
- Hanel R
- ↵
- ↵
- Ferri GL,
- Martinez S,
- Plastino A
- ↵
- ↵
- Tsallis C
- ↵
Citation Manager Formats
Sign up for Article Alerts
Article Classifications
- Physical Sciences
- Physics