## New Research In

### Physical Sciences

### Social Sciences

#### Featured Portals

#### Articles by Topic

### Biological Sciences

#### Featured Portals

#### Articles by Topic

- Agricultural Sciences
- Anthropology
- Applied Biological Sciences
- Biochemistry
- Biophysics and Computational Biology
- Cell Biology
- Developmental Biology
- Ecology
- Environmental Sciences
- Evolution
- Genetics
- Immunology and Inflammation
- Medical Sciences
- Microbiology
- Neuroscience
- Pharmacology
- Physiology
- Plant Biology
- Population Biology
- Psychological and Cognitive Sciences
- Sustainability Science
- Systems Biology

# Considerations for parameter optimization and sensitivity in climate models

Contributed by James C. McWilliams, October 26, 2010 (sent for review July 1, 2010)

## Abstract

Climate models exhibit high sensitivity in some respects, such as for differences in predicted precipitation changes under global warming. Despite successful large-scale simulations, regional climatology features prove difficult to constrain toward observations, with challenges including high-dimensionality, computationally expensive simulations, and ambiguity in the choice of objective function. In an atmospheric General Circulation Model forced by observed sea surface temperature or coupled to a mixed-layer ocean, many climatic variables yield rms-error objective functions that vary smoothly through the feasible parameter range. This smoothness occurs despite nonlinearity strong enough to reverse the curvature of the objective function in some parameters, and to imply limitations on multimodel ensemble means as an estimator of global warming precipitation changes. Low-order polynomial fits to the model output spatial fields as a function of parameter (quadratic in model field, fourth-order in objective function) yield surprisingly successful metamodels for many quantities and facilitate a multiobjective optimization approach. Tradeoffs arise as optima for different variables occur at different parameter values, but with agreement in certain directions. Optima often occur at the limit of the feasible parameter range, identifying key parameterization aspects warranting attention—here the interaction of convection with free tropospheric water vapor. Analytic results for spatial fields of leading contributions to the optimization help to visualize tradeoffs at a regional level, e.g., how mismatches between sensitivity and error spatial fields yield regional error under minimization of global objective functions. The approach is sufficiently simple to guide parameter choices and to aid intercomparison of sensitivity properties among climate models.

Interest in systematic parameter sensitivity and optimization has been developing both in the context of global average climate sensitivity associated with increased greenhouse gases and the effort to improve the model climatology (1–7). Some of this work has focused on variations with parameter of a climate sensitivity defined by the change of global average surface temperature under doubled CO_{2}, some on optimizing the simulation of current climate features by tuning parameter values. Here we examine related questions in sensitivity and optimization, with a particular interest in precipitation. Despite capturing large-scale features, simulations of precipitation in current climate are subject to considerable regional-scale bias (8–12). A common experience is that the simulated climatology exhibits high sensitivity to parameterization changes in certain respects but nonetheless proves difficult to constrain toward observations. At the same time, predicted changes in seasonal precipitation under global warming exhibit striking disagreement among models (13–15). This manifestation of sensitivity is a critical limitation to confidence levels in regional-scale prediction of hydrological cycle changes, and is thus arguably more important for human and ecosystem impacts, on the timescale of a human lifetime, than the climate sensitivity for global average temperature.

The underlying nature of the system has an enormous impact on which of the large literature of strategies for evaluation of sensitivity and optimization are suitable. Is the parameter dependence reasonably smooth in at least some leading measures of the simulation, or does it exhibit discontinuities of derivative or multiple minima? A common dilemma is a model revision yielding improvement in one field or region, but degradation in another—is this tradeoff due to a fundamental lack of robustness (16) or to challenges in maneuvering in a space that is high dimensional in parameter inputs and very high dimensional in model output fields?

A brief listing of the setup of the problem helps to frame the approach. (*i*) A climate modeler generally has a priori information about the range through which it is plausible to vary each parameter, thus determining a feasible range for a constrained optimization problem. Whether parameter range boundary solutions are commonly encountered or whether optima primarily occur in the interior is of interest. (*ii*) The number of parameters *N* to be considered can easily be 10–30, although typically only a subset of these will have strong sensitivity. Brute-force sampling at density *s* gives an order-*s*^{N} problem, but optimization procedures can have much better scaling properties, especially if the relevant measures can be approximated as smoothly varying over parts of the parameter space. (*iii*) Objective functions that could be used for optimization procedures are poorly agreed on, although there has been considerable attempt to define scalar “metrics” to condense information (17, 18). (*iv*) Climate model simulations required to evaluate objective functions at different points in parameter space are costly. Shorter simulations incur substantial error in estimating climate quantities due to intrinsic variability. (*v*) There will typically be an a priori standard case based on plausible guesses of parameter values. The climate modeler must decide whether to update these values based on information from the optimization and assessment process.

## Multiobjective Approach

A standard optimization approach is to lump variables *ϕ*_{k} into a single weighted cost function, such as [1]where *f*_{k} is an objective function associated with a particular aspect of the climate system, such as the square error (*β* = 1) or rms error () in Eq. **1**, of a particular climate variable *ϕ*_{k} relative to the observed field . Typically, *ϕ*_{k} will be time averages (although they could include regression coefficients or other statistics) that depend on space and season, and 〈〉_{k} is a spatial average (or spatial and seasonal average) over a domain of interest. The domain might be global, but there may also be user-dependent preferences for particular averaging regions, hence the inclusion of subscript *k* on the spatial average. Optimization of a weighted cost function yields a single optimum (at considerable cost if the procedure involves functional evaluations using the climate model) that is a compromise among these variables. However, the preference vector of weights *w*_{k} is arbitrary, because it depends on how a particular user values accuracy in certain regions or in certain variables.

Adopting elements from multiobjective optimization (19–23) provides a natural fit for climate modeling in several respects. Treating *f*_{k} as multiple objective functions, optima for each of which are to be considered simultaneously, can identify tradeoffs among variables, the best case for each variable, and dominance relations among potential optima. In addition, much information relevant to decisions on parameter update are qualitative, e.g., indications from a field campaign that a parameterization is less trustworthy in a certain range. Such higher-level information can be subjectively combined in choosing among optima.

## Metamodel for Climate Field Parameter Dependence

The climate modeling problem has close parallels to high-dimensional design problems with computationally expensive black-box functions, optimization strategies for which are reviewed in ref. 24. Results below on smoothness in certain measures lead us to adopt a strategy of fitting the parameter dependence, termed metamodeling (25, 26), with a low-order polynomial. An important distinction is that the metamodel here is created for the spatial and seasonal vector field of the climate model output, not merely for the objective functions. Advantages of this approach include analytic insights that aid visualization of the potentially strong regional dependence of the sensitivity and optimization. To the extent that the metamodel is successful, optimization for multiple objective functions is relatively inexpensive. This approach addresses a nagging issue—that a parameter optimization choice based on one set of objective functions is always vulnerable to reevaluation by additional criteria. Here, alternative objective functions are easily added from the same set of climate model runs.

## Climate Model Setup

We examine these issues in the International Centre for Theoretical Physics climate model (27, 28), which has been used for a number of climate problems (29, 30). Here a triangular truncation at total wavenumber 30 (T30) version is used, roughly equivalent to a 3.75° × 3.75° spatial resolution. We examine both a set of runs with the atmospheric general circulation model (AGCM) forced by observed sea surface temperature (SST) and the AGCM coupled to a mixed-layer (ML) ocean, using a simple estimate of ocean heat flux transport divergence known as a Q-flux (31). The Q-flux is set such that observed SST is approximated at the standard parameter values and does not vary with parameter. For the imposed-SST case, ensembles of ten 25-y runs (1978–2002), differing in their initial conditions, are carried out for each parameter value considered. For the AGCM-ML runs, 260-y runs are carried out at each parameter value, omitting a 10-y spin up and using ten 25-y averages for analogous statistics.

## Parameter Dependence and Fits to Model Output Fields

Here we illustrate the parameter space dependence using four parameters that can have significant impact on the model solution: the subgrid scale wind gustiness that controls the minimum wind speed in the bulk formula for surface fluxes, a viscosity parameter measured as a damping time, a cloud albedo parameter, and the relative humidity parameter from the deep convective parameterization, RH_{conv}, which represents the observed inhibition of convection unless free troposphere relative humidity is sufficiently high.

### AGCM Forced by Observed SST Case.

Fig. 1 shows results for slices along the four selected parameter directions for the case of seasonal precipitation. Contrary to our initial expectations, the parameter dependence proves rather smooth in large-scale measures. We underline the caveat that this approximate smoothness will not necessarily hold for regional-scale measures or more exotic statistics. The fact that it tends to hold for measures such as global rms error or spatial correlation to observations for climatological fields can be of great utility in the estimation of sensitivity and optimization. This property motivates use of low-order polynomial fits below. Estimation error is small for the global rms error for the 10-member ensemble used here, but variation among 25-y averages for individual ensemble members is illustrated in Fig. 1*D*. A single 25-y run would suffice for a first estimate of sensitivity and nonlinearity but would have limitations for statements regarding smoothness.

### Fitting Strategy.

Consider a quadratic metamodel fit to a climate field *ϕ* on the space of *N* parameters, letting *μ*_{i} denote the parameter taken relative to the standard value [2]where ϕ can be a climatological field, anomaly regression, or other statistic from model output. Each coefficient *a*_{i}(**x**,*t*), *b*_{ij}(**x**,*t*) is a high-dimensional vector over space and season, with *b*_{ij} = *b*_{ji}. Any objective function, such as the rms error or spatial correlation to the observed field, can then be reconstructed from the metamodel .

The *N* diagonal values of *b*_{ij} can be fit along with the linear coefficients *a*_{i} from the 2*N* endpoints of the *μ*_{i} ranges, along with the standard case. Thus an order-*N* first-fit procedure yields an estimate of the importance of quadratic nonlinearity in addition to linear sensitivity. The off-diagonal *b*_{ij} coefficients can be evaluated from the corners of pairwise planes (or an equivalent number of suitably distributed points). Because the procedure is of order *N*^{2}, it should in practice be done for a pruned subset of parameter directions. Here results with off-diagonal *b*_{ij} use rms fits to values from all four corners of each pairwise plane. Specific algorithm implementation and seasonal dependence is further discussed in the *SI Appendix*. Here the aim is to illustrate the broad system behavior.

Fig. 1 shows the rms error along four parameter axes constructed from the quadratic metamodel (Eq. **2**). The fit uses only the endpoints and the standard case so AGCM values in the interior constitute independent validation points. Also shown is the rms error constructed from the linear term of Eq. **2**, equivalent to a linear metamodel constructed from the same points and required to pass through the standard case. For the first two parameters shown, gustiness and viscosity, the linear metamodel works fairly well over the entire parameter range, with the quadratic term a small correction. For the second two, the cloud albedo parameter and RH_{conv}, the linear fit works over some neighborhood, but the quadratic terms become important over the full range. For both these cases, optima occur at or near the limit of the feasible parameter range, an uncomfortable situation for the climate modeler that occurs fairly often in our results. In both cases, positive curvature associated with the linear term is opposed by negative curvature associated with the quadratic effects. For RH_{conv}, these actually reverse the curvature of the objective function, with the striking implication that no interior minimum can occur.

### Mixed-Layer Case and Sensitivity Under Change of CO_{2}.

These properties tend to be inherited in the mixed-layer run, as seen in Fig. 2*A* for the case of RH_{conv}. The smoothness of the solution for large-scale measures and the reasonable fit of the quadratic metamodel are reproduced, as is the negative curvature of the rms error.

For the case of changes between doubled CO_{2} and preindustrial AGCM-ML runs, we cannot of course show rms error because no observations are available. To provide a rough sense of the parameter dependence, Fig. 2*B* shows rms differences of the model spatial field at each parameter point relative to the spatial field at a reference parameter value, here using the current climate optimum along the RH_{conv} axis. The metamodel is the quadratic fit as a function of parameter as in Eq. **2** but to the spatial field Δ*ϕ* = *ϕ*_{2×} - *ϕ*_{pre}, where *ϕ*_{2×} and *ϕ*_{pre} are 250-y averages from the doubled CO_{2} and preindustrial cases, respectively, and the two-standard error range is estimated from ten 25-y intervals. Again the rms difference evolves smoothly, exhibiting substantial nonlinearity that is well fit by the quadratic metamodel.

Fig. 3 shows the linear and quadratic contributions of the metamodel (Eq. **2**) in the RH_{conv} direction (fit from endpoints only) for the AGCM-ML preindustrial CO_{2} case. The spatial pattern is that of the coefficients *a*_{i}, *b*_{ii}, and the magnitude is given at parameter endpoint *μ*_{i max} (i.e., at RH_{conv} = 0.9 relative to standard value). The sum of the two fields gives the precipitation change from the standard case at *μ*_{i max}. Both the linear sensitivity and the quadratic nonlinearity are sufficiently strong to produce substantial modifications of the precipitation pattern across the parameter range, especially in the tropics where convective precipitation is the dominant contribution. These spatial patterns provide a view of where changes are occurring at the regional scale and of the regional importance of nonlinearity. Analytic results that further exploit these properties are discussed below.

Fig. 4 shows corresponding linear and quadratic contributions to the parameter sensitivity of the precipitation change under CO_{2} doubling. The strength of the sensitivity is comparable to the magnitudes of seasonal precipitation differences in the tropics seen across a multimodel ensemble in the Coupled Model Intercomparison Project 3 archive (e.g., refs. 14 and 15). The substantial differences in the pattern of change predicted under global warming are here seen in a single model, occurring relatively smoothly under parameter variations. A key feature is the importance of the nonlinear term, which is overall of comparable importance to the linear term and can exceed it in particular regions.

## Analytic Approximations and Regional Contributions

Climate model output comes in the form of high-dimensional vector fields, but these fields are easily interpreted by a climate modeler as geographic information. It can thus be highly advantageous to have analytic solutions that present aspects of the optimization problem in this form. A common class of question is the difference between an optimization based on a regional spatial average, as opposed to the global average. Analytic approximations to the optimization based on the metamodel permit examination of such differences. Recall that the quadratic fit (Eq. **2**) to the AGCM field ϕ gives rms or square-error objective functions *f* (here omitting subscript *k*) that have fourth-order terms in μ. Using square error (which has the same extrema as rms), expanding in μ about a reference value, here the standard case, retaining second-order in μ in *f*, and differentiating yields [3][4][5]where *g* is the gradient in μ and *A* the Hessian matrix from the curvature, both evaluated at the standard case, and *ϕ*_{err} = *ϕ*_{std} - *ϕ*_{obs} is the error of the standard case compared to observations. A standard quadratic optimization problem (32) min _{μ} *f*, with constraints *μ*_{i min} ≤ *μ*_{i} ≤ *μ*_{i max} would give *Aμ* = -*g* for interior solutions that fall within the feasible μ-range, if additional conditions on the curvature are satisfied. For *f* associated with seasonal precipitation, Figs. 1*D* and 2*A* showed that the problem is nonconvex in the RH_{conv} direction, corresponding to *A* not being positive semidefinite, when 〈〉 is a global average. When a boundary solution occurs, interior solutions for other variables can be found with the same *A*_{ij} and a slight adaptation of *g* (see *SI Appendix*). Thus considerable insight into the optimization problem can be gleaned from the approximation (Eq. **3**) (noting that this approximation is not used for numerical multiobjective optimization below, which carries all terms).

The off-diagonal terms in *A* act to change the orientation of the basin in which the optimization occurs. Off-diagonal contributions come from both the linear sensitivity via 〈*a*_{i}*a*_{j}〉 and from 〈*b*_{ij}*ϕ*_{err}〉. These off-diagonal terms are often small compared to the diagonal (i.e., parameter pairs often do not interact strongly), so focusing on the diagonally dominated problem ∂_{μi}*f* = *g*_{i} + *A*_{ii}*μ*_{i} yields a sense of basic behavior. The form of *g*_{i} is a spatial projection of the sensitivity *a*_{i} with the error of the standard case, *ϕ*_{err}. Thus regional sensitivity is only effective at yielding a reduction in error to the extent that the spatial pattern of *a*_{i} matches the error pattern. For an interior solution, if *b*_{ii} is small, the *μ*_{i} optimum would simply be . This expression implies that the distance moved in *μ*_{i} is a tradeoff between the part of the sensitivity that projects on the error, and thus can reduce it, and the part of the sensitivity that is orthogonal to the error (under the inner product corresponding to the average used in the objective function). When using a global average measure, this tradeoff has substantial implications for regional errors. Suppose *a*_{i} has a large value in a small region that does not project on the error. The optimization will yield a solution that reduces global error, at the cost of introducing a large error in that region—a likely explanation for this common modeling experience.

The linear contribution has positive curvature, consistent with the possibility of having an interior minimum. However the quadratic term 2〈*b*_{ii}*ϕ*_{err}〉 can reverse this curvature if sufficiently negative, as in Fig. 2*a*. Again only the projection on *ϕ*_{err} matters. Recall that the quantity acting like an inner product, 〈〉, is typically a spatial average, but can be user dependent, e.g., a regional measure instead of a global average. For a quick view of whether objective functions defined for spatial averages on different regions will tend to have properties similar to the global average or strongly differing, one can examine maps of the spatial structure of these contributions such as those shown in Fig. 5. The quantity (*a*_{i}*ϕ*_{err}) normalized by the diagonal contribution (Fig. 5*A*) is proportional to contributions to the objective function gradient *g*_{i} in RH_{conv} (at *μ* = 0). The fact that most points on the map are negative indicates that optimization using various regional averages in the objective function will most commonly yield optima that move in the same, positive direction in RH_{conv} as occurs when the global average was used. The quantity (Fig. 5*B*) is the contribution of nonlinearity in the model field to the diagonal term. The global average of this term is negative, with magnitude larger than one, so it reverses the curvature. The fact that this term has large negative contributions almost everywhere indicates that this influence of nonlinear terms in the RH_{conv} direction tends to hold consistently for regional averages. For other parameters, regional properties need not be so consistent with those of the global optimization, for instance containing regions of opposite sign. Examination of the spatial patterns in the analytic solution can provide a low-cost way of identifying such regional issues and can be easier to interpret than a large number of regional objective functions.

## Multiobjective Optimization

The success of the metamodel over the feasible parameter range, permitting the construction of various objective functions, facilitates multiobjective optimization, with separate objective functions for each major climate variable. Repeated optimizations using standard packages for constrained optimization (here the Nonlinear Interior point Trust Region Optimization package, ref. 32) prove quite practical, although advanced multiobjective techniques (20, 21) could be applied.

Fig. 6 quantifies the contradiction among objective functions, i.e., the tradeoffs in optimizing for one variable versus another, as the location of the optima in parameter space for different climate variables, illustrating with the June–August season. The spread of the optima in parameter space is substantial in two of the parameter directions shown. In the RH_{conv} direction, the optima all occur at or near the same boundary of the feasible range. This agreement favors an update of the parameter value, but the fact that it occurs at the end of the feasible range suggests careful scrutiny of the physical processes involved in the parameterization in this range. The sensitivity to high values of water vapor in the free troposphere required for convective onset is closely related to other work pointing to the importance of a moist environment in the lower troposphere for development of deep convective plumes (33–38). In the other parameters in Fig. 6, boundary optima are also common. Boundary optima are likewise found in other measures we have examined. In selecting parameter updates, these boundary optima would tend to be distrusted to a degree that depends on the climate modeler’s confidence in the parameterization at the end of the range.

The optima in Fig. 6 are a nondominated set, i.e., comparing the objective functions *f*_{k} between any two optima, one cannot improve one *f*_{k} without making others worse. Additional information about the tradeoffs among optima are provided by constructing from the proxy the set of values of *f*_{k} when each is at its own optimum (the ideal objective vector or utopia point) (19, 22). These values provide a useful normalization for the objective function *f*_{k}. Examining normalized *f*_{k} at each of the other optima indicates that some tradeoffs are only a few percent worse than the optimum value and highlights certain others. For instance, a significant tradeoff in this model involves land surface temperature, which is roughly 30% off its optimum value at the optima for precipitation, vertical velocity, and low-level wind.

Also shown in Fig. 6 is a comparison of a metamodel based on order-*N* climate model evaluations versus a procedure that uses order *N*^{2}. The cheaper order-*N* procedure, using the approximation that parameters do not interact strongly in the quadratic terms, works well enough for most variables that examination of the properties would be useful prior to investing in the additional runs, and could guide the choice of these.

## Discussion

Examination of the properties of a climate model parameter dependence suggests that simple strategies for creating metamodels can provide considerable utility. The relative smoothness in parameter space of large-scale measures of model climatological error and of corresponding sensitivity under greenhouse forcing—even for notoriously nonlinear quantities such as precipitation—is good news for many optimization or sensitivity estimation strategies. Departures from smooth behavior may be anticipated, so it is encouraging that useful results can be obtained by adapting entry-level methods from different areas; extensions to deal with potential complications exist in each.

### Spatial Field Metamodel Benefits and Applicability.

Fitting a metamodel, here quadratic, to *spatial and seasonal fields* of climate model statistics as a function of parameter yields several desirable properties. One is not locked into a particular objective function or set of objective functions from the outset. From a sufficient set of climate model runs, different users can evaluate metamodels for various fields and derive user-dependent objective functions. Furthermore, linearity versus nonlinearity in the fields themselves is very much of interest for evaluation of strategies like multimodel ensemble averages. The linear terms in the metamodel yield quadratic contributions in typical objective functions, which can be directly compared to contributions of nonlinear terms. For further insight into the occurrence of smoothness, consider the contrast to linearization of a climate model about a known trajectory or stationary point. Although parameterizations, notably for precipitation, have locally strong nonlinearities, large internal variability tends to have a smoothing effect on the statistics typically used in objective functions. There may well exist quantities from the same climate simulations that would not have smooth parameter dependence. The quality of the metamodel fit is best evaluated for objective functions based on regional or large-scale averages, rather than at the climate model grid scale, to avoid distracting exceptions that do not affect the larger picture.

### Implications for Optimization Strategy.

Despite relative smoothness of large-scale measures, nonlinearity can be important over the range of certain parameters for quantities like precipitation. In a parameter affecting the relative humidity associated with deep convection, nonlinear terms are large enough to reverse the curvature of the rms error objective function for seasonal precipitation climatology. This curvature reversal yields a sharp minimum in the objective function at the boundary of the feasible range. Nonlinearity contributes to occurrence of boundary optima in other parameters and quantities as well. For the climate model setup of the constrained optimization problem, optima on the boundary typically imply a solution at the limit of permissible values. Rather than blind numerical optimization of a weighted sum, a preferable use of multiobjective information is to supply feedback to the modeler on key tradeoffs and parameterization ranges, and on physical processes for which further scrutiny and development is likely to yield bias reduction.

Analytic results show clearly how large sensitivity does not necessarily lead to reduction in model error relative to an a priori standard parameter setting. When the sensitivity is dominated by linear terms, interior optima come from a tradeoff between the part of the sensitivity that projects on the error spatial pattern and the part that does not, which tends to create new errors in sensitive regions when a global objective function is used. Nonlinear terms likewise enter the optimization at leading order through their projection on the error spatial pattern.

Overall, these methods contribute to a language of assessing and reporting parameter tuning, compactly quantifying the dilemmas. For the question of robustness versus sensitivity in climate models, the results highlight the contribution of having more processes and spatial degrees of freedom exhibiting substantial sensitivity than constraints upon these.

### Implications for Multimodel Ensemble Mean and Model Intercomparison.

Nonlinearity of parameter dependence has a particularly important role in the sensitivity of precipitation under greenhouse gas increases. In the face of strong model disagreement, current practice is to base projections on multimodel ensemble means. The magnitude and spatial distribution of the parameter sensitivity of doubled-CO_{2} precipitation changes seen here suggests that parameter dependence within a single model can provide a useful prototype for thinking about a multimodel ensemble. The analogue of the multimodel ensemble would be an ensemble of randomly chosen parameter values within the feasible range. First, consider a case where a certain parameter setting represents the true system and the model ensemble consists of parameter values randomly distributed about this true value without bias, i.e., the expected value of the parameter settings is the correct value. In this excessively optimistic scenario, would a multimodel ensemble average give an accurate estimate of the true precipitation change? The multimodel average of the linear contributions to the parameter sensitivity would indeed have an expected value of zero—but this error calculation would not hold for the nonlinear contributions. The size of the nonlinearity found here suggests this effect is likely to be a substantial issue. Furthermore, the common occurrence of the best match of current climate simulations to observations at the end of the a priori parameter range suggests that a model ensemble whose parameters are symmetrically distributed through such a range is likely to be biased.

The simple strategy for simulations used here is close to common modeling practice while providing an economical means of extracting more information. A recent extraordinary effort (5), investigating sensitivity for 10 parameters, invested on the order of 10^{4} simulations—a quadratic metamodel could be estimated with less than 100 (see *SI Appendix*) making such estimations more widely feasible. The approach could thus form the basis for a Sensitivity Model Intercomparison Project (SensMIP) in which different models would provide a comparable sampling of their respective most dangerous parameter set. SensMIP questions would include whether the space of possible regional climate change patterns spanned by each model’s feasible range compares to that spanned by the set of models at their standard values, the degree of nonlinearity associated with leading physical processes that impact these, and whether optimization information from the current climate metamodel could help constrain the range of possibilities for climate change. If so, and if smoothness properties prove comparable to those here, metamodeling of the parameter dependence across a set of leading models could yield more quantitative assessment of regional climate change.

## Acknowledgments

We thank F. Kucharski, M. Chekroun, M. Ghil, C. Covey, K. Taylor, and P. Gleckler for discussions. This work was supported in part by National Science Foundation ATM-0645200, National Oceanic and Atmospheric Administration NA08OAR4310882, and Department of Energy DE-FG02-07ER64439.

## Footnotes

^{1}To whom correspondence should be addressed. E-mail: jcm{at}atmos.ucla.edu.Author contributions: J.D.N., A.B., H.L., J.C.M., and J.E.M. performed research; and J.D.N. wrote the paper.

The authors declare no conflict of interest.

This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.1073/pnas.1015473107/-/DCSupplemental.

Freely available online through the PNAS open access option.

## References

- ↵
- ↵
- ↵
- ↵
- ↵
- Knight CG,
- et al.

- ↵
- ↵
- ↵
- ↵
- Covey C,
- et al.

- ↵
- ↵
- ↵
- ↵
- ↵
- Li W,
- Fu R,
- Dickinson RE

- ↵
- Neelin JD,
- Munnich M,
- Su H,
- Meyerson JE,
- Holloway C

- ↵
- McWilliams JC

- ↵
- ↵
- ↵
- Burke EK,
- Kendall G

- Deb K

- ↵
- Coello CAC,
- Lamont GB,
- Veldhuizen DAV

- ↵
- Branke J,
- Deb K,
- Slowinski KMR

- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- Molteni F

- ↵
- ↵
- ↵
- ↵
- ↵
- Nocedal J,
- Wright SJ

- ↵
- ↵
- ↵
- ↵
- ↵
- ↵

## Citation Manager Formats

## Sign up for Article Alerts

## Article Classifications

- Physical Sciences
- Environmental Sciences