Dakota Reference Manual: metrics

| Dakota Reference Manual Version 6.15 Explore and Predict with Confidence

- Main Page
- Related Pages

All Pages
metrics

Compute surrogate quality metrics

## Topics

This keyword is related to the topics:

- surrogate_models

## Specification

Alias: diagnostics

Argument(s): STRINGLIST

Default: No diagnostics

Child Keywords:

| Required/Optional | Description of Group | Dakota Keyword | Dakota Keyword Description
--- | --- | --- | --- | ---
| Optional | cross_validation | Perform k-fold cross validation
| Optional | press | Leave-one-out cross validation

## Description

Diagnostic metrics assess the goodness of fit of a global surrogate to its training data.

The default diagnostics are:

- `root_mean_squared`
- `mean_abs`
- `rsquared`

Additional available diagnostics include

- `sum_squared`
- `mean_squared`
- `sum_abs`
- `max_abs`

The keywords `press` and `cross_validation` further specify leave-one-out or k-fold cross validation, respectively, for all of the active metrics from above.

Usage Tips When specified, the `metrics` keyword must be followed by a list of quoted strings, each of which activates a metric.

## Examples

This example input fragment constructs a quadratic polynomial surrogate and computes four metrics on the fit, both with and without 5-fold cross validation. (Also see dakota/share/dakota/test/dakota_surrogate_import.in for additional examples.)

    model
      surrogate global
        polynomial quadratic
        metrics = "root_mean_squared" "sum_abs" "mean_abs" "max_abs"
        cross_validation folds = 5

## Theory

Most of these diagnostics refer to some operation on the residuals (the difference between the surrogate model and the truth model at the data points upon which the surrogate is built).

For example, `sum_squared` refers to the sum of the squared residuals, and `mean_abs` refers to the mean of the absolute value of the residuals. `rsquared` refers to the R-squared value typically used in regression analysis (the proportion of the variability in the response that can be accounted for by the surrogate model). Care should be taken when interpreting metrics, for example, errors may be near zero for interpolatory models or rsquared may not be applicable for non-polynomial models.