# From understanding of color perception to dynamical systems by manifold learning

^{a}Computer Science Department, Technion-Israel Institute of Technology, Haifa 32000, Israel

See allHide authors and affiliations

## Comprehending Color

Let us start with a seemingly unrelated field to that described in the article by Yair et al. (1) in PNAS. The field of psychophysics deals with the relationships between physical stimuli and mental phenomena. An excellent example is the scientific community’s early efforts to study the human perception of color. Scientists have been intrigued by visual awareness of colors, trying to understand our interpretation of colors and attempting to quantify human perception with simple equations. Roughly speaking, one could divide these efforts into axiomatic ones that gave birth to the Young, Maxwell, Helmholtz, and, later on, Schrödinger so-called “inductive color line elements” and the empirical color arc-lengths that reflected the effort to virtually embed measurements of human color perception into a simple, often Euclidean, domain. In fact, the latter school of thought, of treating the problem empirically rather than axiomatically, is probably one of the earliest attempts to apply a manifold learning technique to study a psychophysical phenomenon. The outcome was the insightful observation that human color perception is 3D, while most birds probably have (and most dinosaurs probably had) a color perception manifold of higher dimensions and most other mammals share a lower dimensional space for the (lack of) perception of color. One of the analysis tools used to arrive at this important observation is known as multidimensional scaling (MDS), and is related to the famous principal component analysis machinery that is commonly used in big data representation, for which various modern generalizations exist. While axiomatic realizations of studying the color receptors in the eye lead Maxwell (2) to the understanding that color images could be synthesized by a linear combination of three monochromatic colors, the space in which one should operate and the ways by which anchor (basis) colors should be selected has been the topic of many scientific and industrial explorations leading to the design of modern mobile, computer, and television screens, as well as almost all printing devices. The early empirical analysis of color perception by manifold learning is indeed a remarkable step in our ability to model human behavior and harness this understanding to our benefit.

When trying to process images so as to enhance them and improve their quality, the color line element should obviously come into play. Geometry modeling of image formation indeed led researchers to the introduction of a new manifold that marries the color line element with the image coordinates, giving rise to a 2D manifold (the image) embedded in a 5D space, where three of these dimensions are an exact result of our understanding of color perception. Indeed, these geometric observations are incorporated, in one way or another, into most modern color image processing tools. The Beltrami filter, bilateral filter, and, in fact, most color processing methodologies exploit our understanding of the fundamental geometry behind color perception in one way or another.

## Image Understanding

Once our perception of color was well understood, automatic identifying and classifying of the content in an image became the next problem the academic community was trying to resolve. This effort gave birth to a field known as robot vision, computer vision, or image understanding. The notion of so-called “deep learning” without “understanding” has very recently substantially changed the way in which fields like computer vision are addressed. Within the context of deep learning, we can now deliberately avoid the term understanding, as deep learning methods try to interpolate between given observations, fitting a nonlinear interpolation mechanism with a multiparameter fixed architecture into a huge corpus of annotated observations. In these research arenas, “big data” of annotated objects are usually publicly available, which make the process of tuning the large number of parameters of the interpolation mechanism (often referred to as a neural network) feasible. At the other end, representation of empirical observations of physical phenomena for which there is no apparent explanation through a nonlinear multiparameter generic mechanism without an Occam’s razor simple theoretical or axiomatic reasoning is what the article by Yair et al. (1) is about. A couple of centuries after the manifold learning way of deciphering the psychophysical modeling of color perception, the model of Yair et al. (1) comes in a timely fashion, introducing novel concepts and ideas by which some understanding could still be extracted from the learned objective. In this case, the learning is not deep by its modern definition but rather belongs to the class of approaches known as manifold learning, or geometric reasoning.

## Study of Shapes and Forms

Another interesting and related geometric reasoning mechanism for analyzing the structure of the cortical surface by mapping its geometric structure into a Euclidean one was introduced by Schwartz et al. (3). Treating surfaces as metric spaces was an interesting new line of thought that came about from the effort of Schwartz et al. (3) to find a unified parametrization for the surface of the brain. The idea was to first embed the geometric structure of the surface into a 2D Euclidean space invoking, yet again, the MDS procedure. This effort, in a sense, is a generalization of the celebrated mapmakers’ cartography problem.

In an attempt to refine the shape-matching problem, the Gromov–Hausdorff distance, which was thought to be restricted to theoretical analysis of abstract metric spaces, has also been adopted by the shape analysis and manifold learning communities. It was suggested as a candidate for measuring the discrepancy between two deformable shapes (4), followed by the generalized MDS (GMDS) (5) that numerically approximates that measure to compute the map between two surfaces that best preserves corresponding intergeodesic distances. In the GMDS framework, the Hausdorff distance was replaced by the Wasserstein norm. Other efforts that try to find a universal parametrization for surfaces suggested conformal mapping to a disk (6⇓–8) and even using the earth mover's distance (EMD) between corresponding conformal factors on the disk (9) in a fashion related to that suggested in the model of Yair et al. (1).

Distances measured on surfaces are smooth functions for which the gradient magnitude is equal to 1 almost everywhere. As such, they are suited for compact spectral representation (10), allowing us to translate the Gromov–Hausdorff framework into the spectral domain. In fact, resorting to a truncated spectral domain as an alternative for computing the Gromov distance explicitly for metric analysis was probably first suggested by Bérard et al (11) and more recently adopted to surface analysis by Ovsjanikov et al. (12).

## Deciphering Dynamical Systems

In their paper, Yair et al. (1) are applying geometric manifold learning techniques to find compact representations of empirical observations of physical phenomena through a geometry known as diffusion maps. That is, distances between events are described A couple of centuries after the manifold learning way of deciphering the psychophysical modeling of color perception, the model of Yair et al. comes in a timely fashion, introducing novel concepts and ideas by which some understanding could still be extracted from the learned objective.

by kernels defined through some basis functions. On top of that, Wasserstein distance is efficiently approximated using recent developments with an even smoother version of measuring discrepancies, treating distances between events as ones between probability distributions that relate to the Monge–Kantorovich problem or the EMD. In the paper of Yair et al. (1), the EMD is computed in such a way that the coupling between the “ground distance” is constructed in a data-driven way from the observations. Although most of the ingredients and tools have been known for a while, their combination allows the authors to nicely apply them to find the governing behavior of a pendulum extracted from a video of its different positions in time as one example and to find a compact geometric description of even more complicated systems, like two pendula. Capturing the behavior of dynamical systems efficiently for which only the time axis is a common denominator poses a challenge that Yair et al. (1) seem to gracefully overcome. It is yet to be seen if this novel application of manifold learning to dynamical systems would lead to technological innovative inventions such as those resulting from deciphering human perception of color.

## References

- ↵.
- Yair O,
- Talmon R,
- Coifman RR,
- Kevrekidis IG

- ↵.
- Niven WD

- Maxwell JC

- ↵
- ↵.
- Memoli F,
- Sapiro G

- ↵.
- Bronstein AM,
- Bronstein MM,
- Kimmel R

- ↵
- ↵.
- Lipman Y,
- Funkhouser T

- ↵.
- Zeng Y, et al.

- ↵
- ↵.
- Aflalo Y,
- Kimmel R

- ↵
- ↵.
- Ovsjanikov M,
- Ben-Chen M,
- Solomon J,
- Butscher A,
- Guibas L

## Citation Manager Formats

## Article Classifications

- Physical Sciences
- Applied Mathematics

## See related content: