Blurb::
An empirical model that is created from data or the results of a submodel
Description::
Surrogate models are inexpensive approximate models that are intended
to capture the salient features of an expensive high-fidelity model.
They can be used to explore the variations in response quantities over
regions of the parameter space, or they can serve as inexpensive
stand-ins for optimization or uncertainty quantification studies (see,
for example, the surrogate-based optimization methods,
\ref method-surrogate_based_global and \ref method-surrogate_based_local).

Surrogate models supported in Dakota are 
categorized as Data Fitting or Hierarchical, as shown below.
Each of these surrogate types provides an approximate representation
of a "truth" model which is used to perform the parameter to response
mappings. This approximation is built and updated using results from the
truth model, called the "training data".

<ul>
  <li> Data fits:
  
    <em>Known Issue: When using discrete variables, there have been
    sometimes significant differences in surrogate behavior observed
    across computing platforms in some cases.  The cause has not yet
    been fully diagnosed and is currently under investigation.  In
    addition, guidance on appropriate construction and use of
    surrogates with discrete variables is under development.  In the
    meantime, users should therefore be aware that there is a risk of
    inaccurate results when using surrogates with discrete
    variables.</em>

    Data fitting methods involve construction of an
    approximation or surrogate model using data (response values,
    gradients, and Hessians) generated from the original truth model.
    Data fit methods can be further categorized as local, multipoint,
    and global approximation techniques, based on the number of points
    used in generating the data fit.
    <ol>
    <li> Local: built from response data from a single point in parameter space
      <ul>
      <li> Taylor series expansion: \ref model-surrogate-local-taylor_series

           Training data consists of a single point, plus gradient and Hessian information.
      </ul>
    <li> Multipoint:  built from two or more points in parameter space, often involving the current and previous iterates of a minimization algorithm.
      <ul>
      <li> TANA-3: \ref model-surrogate-multipoint-tana

           Training Data comes from a few previously evaluated points 
      </ul>
    <li> Global full space response surface methods: 
      <ul>
      <li> Polynomial regression: \ref model-surrogate-global-polynomial
      <li> Gaussian process (Kriging): \ref model-surrogate-global-gaussian_process
      <li> Artifical neutral network: \ref model-surrogate-global-neural_network
      <li> MARS: \ref model-surrogate-global-mars
      <li> Radial Basis Functions: \ref model-surrogate-global-radial_basis
      <li> Orthogonal polynomials (only supported in PCE/SC for now): \ref method-polynomial_chaos and \ref method-stoch_collocation
      </ul>

      Training data is generated using either a design of
      experiments method applied to the truth model ( specified by
      \ref model-surrogate-global-dace_method_pointer), or from saved
      data (specified by \ref model-surrogate-global-reuse_points ) 
      in a restart database, or an import file.
  </ol>
  <li> Multifidelity/hierarchical:
  
    Multifidelity modeling involves the use of a
    low-fidelity physics-based model as a surrogate for the original
    high-fidelity model. The low-fidelity model typically involves a
    coarser mesh, looser convergence tolerances, reduced element order,
    or omitted physics.
  
    See \ref model-surrogate-hierarchical.
<!-----
  <li> Reduced-order model surrogates: none currently active in Dakota


    <b> Reduced Order Models </b>: A reduced-order model (ROM) is
    mathematically derived from a high-fidelity model using the
    technique of Galerkin projection. By computing a set of basis
    functions (e.g., eigenmodes, left singular vectors) that capture
    the principal dynamics of a system, the original high-order system
    can be projected to a much smaller system, of the size of the
    number of retained basis functions.

---->
</ul>

The global and hierarchal surrogates have a correction feature 
in order to improve the local accuracy of the surrogate models.
The
correction factors force the surrogate models to match the true
function values and possibly true function derivatives at the center
point of each trust region.
Details can be found on global \ref model-surrogate-global-correction
or hierarchical \ref model-surrogate-hierarchical-correction.


Topics::	
Examples::
Theory::
Surrogate models are used extensively in the
surrogate-based optimization and least squares methods, in
which the goals are to reduce expense by minimizing the number of
truth function evaluations and to smooth out noisy data with a global
data fit. However, the use of surrogate models is not restricted to
optimization techniques; uncertainty quantification and optimization
under uncertainty methods are other primary users.

<b> Data Fit Surrogate Models </b>

A surrogate of the {\em data fit} type is a non-physics-based
approximation typically involving interpolation or regression of a set
of data generated from the original model. Data fit surrogates can be
further characterized by the number of data points used in the fit,
where a local approximation (e.g., first or second-order Taylor
series) uses data from a single point, a multipoint approximation
(e.g., two-point exponential approximations (TPEA) or two-point
adaptive nonlinearity approximations (TANA)) uses a small number of
data points often drawn from the previous iterates of a particular
algorithm, and a global approximation (e.g., polynomial response
surfaces, kriging/gaussian_process, neural networks, radial basis 
functions, splines)
uses a set of data points distributed over the domain of interest,
often generated using a design of computer experiments.

Dakota contains several types of surface fitting methods that can be
used with optimization and uncertainty quantification methods and
strategies such as surrogate-based optimization and optimization under
uncertainty. These are: polynomial models (linear, quadratic, and
cubic), first-order Taylor series expansion, kriging spatial
interpolation, artificial neural networks, multivariate adaptive
regression splines, radial basis functions, and moving least squares. 
With the exception of Taylor series methods, all of the above methods 
listed in the previous sentence are accessed in Dakota through the 
Surfpack library. All of these surface fitting methods can be
applied to problems having an arbitrary number of design parameters.
However, surface fitting methods usually are practical only for
problems where there are a small number of parameters (e.g., a maximum
of somewhere in the range of 30-50 design parameters). The
mathematical models created by surface fitting methods have a variety
of names in the engineering community. These include surrogate models,
meta-models, approximation models, and response surfaces. For this
manual, the terms surface fit model and surrogate model are used.

The data fitting methods in Dakota include software developed by
Sandia researchers and by various researchers in the academic
community.



<b> Multifidelity Surrogate Models </b>

A second type of surrogate is the {\em model hierarchy} type (also
called multifidelity, variable fidelity, variable complexity, etc.).
In this case, a model that is still physics-based but is of lower
fidelity (e.g., coarser discretization, reduced element order, looser
convergence tolerances, omitted physics) is used as the surrogate in
place of the high-fidelity model. For example, an inviscid,
incompressible Euler CFD model on a coarse discretization could be
used as a low-fidelity surrogate for a high-fidelity Navier-Stokes
model on a fine discretization.

<!-----
<b> Reduced Order Models </b>

A third type of surrogate model involves {\em reduced-order modeling}
techniques such as proper orthogonal decomposition (POD) in
computational fluid dynamics (also known as principal components
analysis or Karhunen-Loeve in other fields) or spectral decomposition
(also known as modal analysis) in structural dynamics. These
surrogate models are generated directly from a high-fidelity model
through the use of a reduced basis (e.g., eigenmodes for modal
analysis or left singular vectors for POD) and projection of the
original high-dimensional system down to a small number of generalized
coordinates. These surrogates are still physics-based (and may
therefore have better predictive qualities than data fits), but do not
require multiple system models of varying fidelity (as required for
model hierarchy surrogates).
----->

<b> Surrogate Model Selection </b>

This section offers some guidance on choosing from among the available
surrogate model types.

<ul>
<li> For Surrogate Based Local Optimization, using the
  \ref method-surrogate_based_local method with a trust region:

  using the keywords: 
  <ol>
  <li>
    \ref model-surrogate \ref model-surrogate-local \ref model-surrogate-local-taylor_series or
  </li>
  <li>
    \ref model-surrogate \ref model-surrogate-multipoint \ref model-surrogate-multipoint-tana
  </li>
  </ol>
  will probably work best.

  If for some reason you wish or need to use a global 
  surrogate (not recommended) then the best of these options is likely 
  to be either:
  <ol>
  <li>
    \ref model-surrogate \ref model-surrogate-global 
    \ref model-surrogate-global-gaussian_process \ref model-surrogate-global-gaussian_process-surfpack or
  </li>
  <li>
    \ref model-surrogate \ref model-surrogate-global \ref model-surrogate-global-moving_least_squares.
  </li>
  </ol>
</li>
<li> For Efficient Global Optimization (EGO), the 
   \ref method-efficient_global method:

   the default surrogate is:
   \ref model-surrogate-global-gaussian_process \ref model-surrogate-global-gaussian_process-surfpack 
   which is likely to find a more optimal value and/or require fewer true 
   function evaluations than the alternative,
   \ref model-surrogate-global-gaussian_process \ref model-surrogate-global-gaussian_process-dakota. However, the 
   \ref model-surrogate-global-gaussian_process-surfpack will likely take more time to build 
   than the \c dakota version. Note that currently the 
   \ref method-efficient_global-use_derivatives keyword is not recommended for use with
   EGO based methods.
</li>
<li> For EGO based global interval estimation, the 
   \ref method-global_interval_est \ref method-global_interval_est-ego method: 

   the default \ref model-surrogate-global-gaussian_process \ref model-surrogate-global-gaussian_process-surfpack will
   likely work better than the alternative \ref model-surrogate-global-gaussian_process 
   \ref model-surrogate-global-gaussian_process-dakota.
</li>
<li> For Efficient Global Reliability Analysis (EGRA), the 
   \ref method-global_reliability method:

   the \ref model-surrogate-global-gaussian_process-surfpack and 
   \c dakota versions of the gaussian process tend to give 
   similar answers with the \c dakota version tending to use
   fewer true function evaluations. Since this is based on EGO, it
   is likely that the default \ref model-surrogate-global-gaussian_process-surfpack is more 
   accurate, although this has not been rigorously demonstrated.
</li>
<li> For EGO based Dempster-Shafer Theory of Evidence, i.e. the 
   \ref method-global_evidence \ref method-global_evidence-ego method, the default
   \ref model-surrogate-global-gaussian_process \ref model-surrogate-global-gaussian_process-surfpack often use
   significantly fewer true function evaluations than the 
   alternative \ref model-surrogate-global-gaussian_process \ref model-surrogate-global-gaussian_process-dakota.
</li>
<li> When using a global surrogate to extrapolate, any of the surrogates:
   <ul>
   <li>\ref model-surrogate-global-gaussian_process \ref model-surrogate-global-gaussian_process-surfpack
   </li>
   <li>\ref model-surrogate-global-polynomial \ref model-surrogate-global-polynomial-quadratic
   </li>
   <li>\ref model-surrogate-global-polynomial \ref model-surrogate-global-polynomial-cubic 
   </li>
   </ul>
   are recommended.
</li>
<li> When there is over roughly two or three thousand data points 
   and you wish to interpolate (or approximately interpolate) then 
   a Taylor series, Radial Basis Function Network, or Moving Least
   Squares fit is recommended. The only reason that the 
   \ref model-surrogate-global-gaussian_process 
   \ref model-surrogate-global-gaussian_process-surfpack 
   is not recommended is that it can take a considerable amount of time
   to construct when the number of data points is very large. Use 
   of the third party MARS package included in Dakota is generally 
   discouraged.
</li>
<li> In other situations that call for a global surrogate, the 
   \ref model-surrogate-global-gaussian_process
   \ref model-surrogate-global-gaussian_process-surfpack is generally 
   recommended. The \c use_derivatives keyword will 
   only be useful if accurate and inexpensive derivatives 
   are available. Finite difference derivatives are disqualified 
   on both counts. However, derivatives generated by analytical,
   automatic differentiation, or continuous adjoint techniques
   can be appropriate. Currently, first order derivatives, i.e.
   gradients, are the highest order derivatives that can be used
   to construct the \ref model-surrogate-global-gaussian_process
   \ref model-surrogate-global-gaussian_process-surfpack
   model; Hessians will not be used even if they are available.
</li>
</ul>




Faq::
See_Also::	model-single, model-nested
