Induction as model selection
- Department of Psychology, University of California, Los Angeles, CA 90095-1563
All intelligent systems, whether children, scientists, or futuristic robots, require the capacity for induction, broadly defined to encompass all inferential processes that expand knowledge in the face of uncertainty (1). Any finite set of data is consistent with an infinite number of inductive hypotheses. The apparent accuracy of many everyday inferences therefore suggests that humans have, as the philosopher Charles Peirce put it, “special aptitudes for guessing right” (2). How can people, often restricted to sparse and noisy data, achieve some significant degree of success in discerning the underlying regularities in the world? The answer seems to require specifying inductive constraints. The report by Kemp and Tenenbaum in this issue of PNAS (3) represents an important advance in understanding the constraints that guide successful induction across a broad set of domains.
Overview of hierarchical Bayesian approach to learning structural form proposed by Kemp and Tenenbaum (3), using examples of similarities among a set of animals. (A) The data at the bottom, in the form of a feature vector for each animal, can potentially be produced by alternative forms (ring, partition, tree, order, hierarchy) that can take on many different structures (defined by nodes and edges in graph). Likelihoods constrain the possible structural forms to those consistent with the data of feature vectors (blue background), but the set of possibilities may remain large. (B) The set of possible structural forms is further constrained by the prior probability of each form and by the prior conditional probability of each structure given a form. The priors for structures conditional on forms favor simpler structures (those with fewer nodes). Bayesian inference identifies the specific structure (hierarchy in green) that has maximal probability as determined by the product of the likelihood and prior knowledge: P(S, F|D) ∝ …
*E-mail: holyoak{at}lifesci.ucla.edu
