Blurb::
Correction approaches for surrogate models
Description::

Some of the surrogate model types support the use of correction
factors that improve the local accuracy of the surrogate models.

The \c correction specification specifies that the approximation will
be corrected to match truth data, either matching truth values in the
case of \c zeroth_order matching, matching truth values and gradients
in the case of \c first_order matching, or matching truth values,
gradients, and Hessians in the case of \c second_order matching. For
\c additive and \c multiplicative corrections, the correction is local
in that the truth data is matched at a single point, typically the
center of the approximation region. The \c additive correction adds a
scalar offset (\c zeroth_order), a linear function (\c first_order),
or a quadratic function (\c second_order) to the approximation to
match the truth data at the point, and the \c multiplicative
correction multiplies the approximation by a scalar (\c zeroth_order),
a linear function (\c first_order), or a quadratic function (\c
second_order) to match the truth data at the point. The \c additive
\c first_order case is due to \cite Lew00
and the \c multiplicative \c first_order case is commonly known as
beta correction \cite Haftka1991. For the \c combined
correction, the use of both additive and multiplicative corrections
allows the satisfaction of an additional matching condition, typically
the truth function values at the previous correction point (e.g., the
center of the previous trust region). The \c combined correction is
then a multipoint correction, as opposed to the local \c additive and
\c multiplicative corrections. Each of these correction capabilities
is described in detail in \cite Eld04.


The
correction factors force the surrogate models to match the true
function values and possibly true function derivatives at the center
point of each trust region.
Currently, Dakota supports either zeroth-,
first-, or second-order accurate correction methods, each of which can
be applied using either an additive, multiplicative, or combined
correction function. For each of these correction approaches, the
correction is applied to the surrogate model and the corrected model
is then interfaced with whatever algorithm is being employed. The
default behavior is that no correction factor is applied.

The simplest correction approaches are those that enforce consistency
in function values between the surrogate and original models at a
single point in parameter space through use of a simple scalar offset
or scaling applied to the surrogate model. First-order corrections
such as the first-order multiplicative correction (also known as beta
correction \cite Cha93) and the first-order additive
correction \cite Lew00 also enforce consistency in the gradients and
provide a much more substantial correction capability that is
sufficient for ensuring provable convergence in SBO algorithms.
SBO convergence rates can be further
accelerated through the use of second-order corrections which also
enforce consistency in the Hessians \cite Eld04, where the
second-order information may involve analytic, finite-difference, or
quasi-Newton Hessians.

Correcting surrogate models with additive corrections involves
<!-- cannot reuse anchor in a DUPLICATE: \anchor eq-correct_val_add -->
\f{equation}
\hat{f_{hi_{\alpha}}}({\bf x}) = f_{lo}({\bf x}) + \alpha({\bf x}) 
\f}
where multifidelity notation has been adopted for clarity. For
multiplicative approaches, corrections take the form
<!-- cannot reuse anchor in a DUPLICATE: \anchor eq-correct_val_mult -->
\f{equation}
\hat{f_{hi_{\beta}}}({\bf x}) = f_{lo}({\bf x}) \beta({\bf x})
\f}
where, for local corrections, \f$\alpha({\bf x})\f$ and \f$\beta({\bf x})\f$
are first or second-order Taylor series approximations to the exact
correction functions:
<!-- cannot reuse anchor in a DUPLICATE: \anchor eq-taylor_a -->
<!-- cannot reuse anchor in a DUPLICATE: \anchor eq-taylor_b -->
\f{eqnarray}
\alpha({\bf x}) & = & A({\bf x_c}) + \nabla A({\bf x_c})^T 
({\bf x} - {\bf x_c}) + \frac{1}{2} ({\bf x} - {\bf x_c})^T 
\nabla^2 A({\bf x_c}) ({\bf x} - {\bf x_c})  \\
\beta({\bf x}) & = & B({\bf x_c}) + \nabla B({\bf x_c})^T 
({\bf x} - {\bf x_c}) + \frac{1}{2} ({\bf x} - {\bf x_c})^T \nabla^2 
B({\bf x_c}) ({\bf x} - {\bf x_c})
\f}
where the exact correction functions are
<!-- cannot reuse anchor in a DUPLICATE: \anchor eq-exact_B -->
<!-- cannot reuse anchor in a DUPLICATE: \anchor eq-exact_A -->
\f{eqnarray}
A({\bf x}) & = & f_{hi}({\bf x}) - f_{lo}({\bf x})    \\
B({\bf x}) & = & \frac{f_{hi}({\bf x})}{f_{lo}({\bf x})} 
\f}
Refer to \cite Eld04 for additional details on the derivations.

A combination of additive and multiplicative corrections can provide
for additional flexibility in minimizing the impact of the correction
away from the trust region center. In other words, both additive and
multiplicative corrections can satisfy local consistency, but through
the combination, global accuracy can be addressed as well. This
involves a convex combination of the additive and multiplicative
corrections:
\f[ \hat{f_{hi_{\gamma}}}({\bf x}) = \gamma \hat{f_{hi_{\alpha}}}({\bf x}) +
(1 - \gamma) \hat{f_{hi_{\beta}}}({\bf x})  \f]
where \f$\gamma\f$ is calculated to satisfy an additional matching
condition, such as matching values at the previous design iterate.

It should be noted that in both first order correction methods, the
function \f$\hat{f}(x)\f$ matches the function value and gradients of
\f$f_{t}(x)\f$ at \f$x=x_{c}\f$. This property is necessary in proving that
the first order-corrected SBO algorithms are provably convergent to a
local minimum of \f$f_{t}(x)\f$. However, the first order correction
methods are significantly more expensive than the zeroth order
correction methods, since the first order methods require computing
both \f$\nabla f_{t}(x_{c})\f$ and \f$\nabla f_{s}(x_{c})\f$. When the SBO
strategy is used with either of the zeroth order correction methods,
or with no correction method, convergence is not guaranteed to a local
minimum of \f$f_{t}(x)\f$. That is, the SBO strategy becomes a heuristic
optimization algorithm. From a mathematical point of view this is
undesirable, but as a practical matter, the heuristic variants of SBO
are often effective in finding local minima.

<b> Usage guidelines </b>

\li Both the \c additive zeroth_order and
 \c multiplicative zeroth_order correction methods are
 "free" since they use values of \f$f_{t}(x_{c})\f$ that are normally
 computed by the SBO strategy.

\li The use of either the \c additive first_order method or
 the \c multiplicative first_order method does not necessarily
 improve the rate of convergence of the SBO algorithm.

\li When using the first order correction methods, the
 gradient-related response keywords must be modified to allow either analytic or
 numerical gradients to be computed. This provides the gradient data
 needed to compute the correction function.

\li For many computationally expensive engineering optimization
 problems, gradients often are too expensive to obtain or are
 discontinuous (or may not exist at all). In such cases the heuristic
 SBO algorithm has been an effective approach at identifying optimal
 designs \cite Giu02.

Topics::
Examples::
Theory::
Faq::
See_Also::	
