\<!-------------------------------------------->

\page hdf5_output Dakota HDF5 Output

Beginning with release 6.9, %Dakota gained the ability to write 
many method results such as the correlation matrices computed by 
\ref method-sampling studies and the best parameters 
discovered by optimization methods to disk in **HDF5**. In %Dakota
6.10 and above, evaluation data (variables and responses for 
each model or interface evaluation) may also be written. Many users 
may find this newly supported format more convenient than scraping or 
copying and pasting from Dakota's console output.

To enable HDF5 output, the \ref environment-results_output 
keyword with the \ref environment-results_output-hdf5 option 
must be added to the Dakota input file. In additon, Dakota must 
have been built with HDF5 support. Beginning with Dakota 6.10, HDF5
is enabled in our publicly available downloads.  HDF5 support is 
considered a somewhat experimental feature. The results of some Dakota 
methods are not yet written to HDF5, and in a few, limited situations, 
enabling HDF5 will cause Dakota to crash.

\section hdf5_output-description HDF5 Concepts

HDF5 is a format that is widely used in scientific software for 
efficiently storing and organizing data. The HDF5 standard and 
libraries are maintained by the [HDF Group](https://hdfgroup.org).

In HDF5, data are stored in multidimensional arrays called *datasets*. 
Datasets are organized hierarchically in *groups*, which also can 
contain other groups. Datasets and groups are conceptually similar 
to files and directories in a filesystem. In fact, every HDF5 file 
contains at least one group, the root group, denoted "/", and
groups and datasets are referred to using slash-delimited absolute or
relative paths, which are more accurately called *link names*.

\image html hdf5_layout.png "Example HDF5 Layout"



HDF5 has as one goal that data be "self-documenting" through the 
use of metadata. %Dakota output files include two kinds of
metadata.
- **Dimension Scales**. Each dimension of a dataset may have zero or more
  scales, which are themselves datasets. Scales are often used to 
  provide, for example, labels analogous to column headings 
  in a table (see the dimension scales that %Dakota applies to 
  \ref hdf5_results-sampling_moments "moments") or numerical values 
  of an indepenent variable (user-specified probability levels in 
  \ref hdf5_results-level_mappings "level mappings").
- **Attributes**. key:value pairs that annotate a group or dataset. A key
  is always a character string, such as \c dakota_version, and (in Dakota 
  output) the value can be a string-, integer-, or real-valued scalar. 
  %Dakota stores the number of samples that were requested 
  in a \c sampling study in the attribute 'samples'.

\section hdf5_output-accessing Accessing Results

Many popular programming languages have support, either natively or from
a third-party library, for reading and writing HDF5 files. The HDF Group
itself supports C/C++ and Java libraries. The Dakota Project suggests 
the \c h5py module for Python. Examples that demonstrate using \c h5py
to access and use Dakota HDF5 output may be found in the Dakota installation
at \c dakota/share/dakota/examples/official/hdf5.

\section hdf5_results Organization of Results

Currently, complete or nearly complete coverage of results from sampling, 
optimization and calibration methods, parameter studies, and stochastic 
expansions exists. Coverage will continue to expand in future releases 
to include not only the results of all methods, but other potentially useful
information such as interface evaluations and model tranformations.

Methods in Dakota have a character string Id and are executed by Dakota one or
more times. (Methods are executed more than once in studies that
include a \ref model-nested "nested model", for example.) The Id may be provided by 
the user in the input file using the \ref method-id_method keyword, or it may be
automatically generated by Dakota. Dakota uses the label \c NO_METHOD_ID for methods
that are specified in the input file without an \c id_method, and \c NOSPEC_METHOD_ID_\<N\> 
for methods that it generates for its own internal use. The \<N\> in the latter 
case is an incrementing integer that begins at 1.

The results for the \<N\>th execution of a method that has the label \c \<method Id\> 
are stored in the group

    /methods/<method Id>/results/execution:<N>/

The \c /methods group is always present in Dakota HDF5 files, provided at
least one method added results to the output. (In a future Dakota release, the top
level groups \c /interfaces and \c /models will be added.) The group \c execution:1
also is always present, even if there is only a single execution.

The groups and datasets for each type of result that Dakota is currently capable
of storing are described in the following sections. Every dataset is documented
in its own table. These tables include: 

- A brief *description* of the dataset.
- The *location* of the dataset relative to `/methods/<method Id>/execution:<N>`.
  This path may include both literal text that is always present and replacement text.
  Replacement text is \<*enclosed in angle brackets and italicized*\>. Two examples of 
  replacement text are \<*response descriptor*\> and  \<*variable descriptor*\>, which 
  indicate that the name of a Dakota response or variable makes up a portion of the path.
- Clarifying *notes*, where appropriate.
- The *type* (String, Integer, or Real) of the information in the dataset.
- The *shape* of the dataset; that is, the number of dimensions and the size of each 
  dimension.
- A description of the dataset's *scales*, which includes
  + The *dimension* of the dataset that the scale belongs to.
  + The *type* (String, Integer, or Real) of the information in the scale.
  + The *label* or name of the scale.
  + The *contents* of the scale. Contents that appear in plaintext are literal and will
    always be present in a scale. Italicized text describes content that varies.
  + *notes* that provide further clarification about the scale.
- A description of the dataset's *attributes*, which are key:value pairs that provide
  helpful context for the dataset.

The **Expected Output** section of each \ref method's keyword documentation 
indicates the kinds of output, if any, that method currently can write to HDF5. These 
are typically in the form of bulleted lists with clariying notes that refer 
back to the sections that follow.

\subsection hdf5_results-metadata Study Metadata

Several pieces of information about the Dakota study are stored as
attributes of the top-level HDF5 root group ("/"). These include:

\htmlinclude study_attributes_table.html

\subsection hdf5_results-variables A Note about Variables Storage

Variables in most Dakota output (e.g. tabular data files) and input (e.g. imported
data to construct surrogates) are listed in "input spec" order. (The \ref variables 
keyword section is arranged by input spec order.) In this ordering, they are sorted
first by function:
1. Design
2. Aleatory
3. Epistemic
4. State

And within each of these categories, they are sorted by domain:
1. Continuous
2. Discrete integer (sets and ranges)
3. Discrete string
4. Discrete real

A shortcoming of HDF5 is that datasets are homogeneous; for example, string- and 
real-valued data cannot readily be stored in the same dataset. As a result,
Dakota has chosen to flip "input spec" order for HDF5 and sort first by domain, then 
by function when storing variable information. When applicable, there may
be as many as four datasets to store variable information: one to store continuous 
variables, another to store discrete integer variables, and so on. Within each of these, 
variables will be ordered by function.

\subsection hdf5_results-sampling_moments Sampling Moments

\ref method-sampling produces moments (e.g. mean, standard deviation or
variance) of all responses, as well as 95% lower and upper confidence
intervals for the 1st and 2nd moments. These are stored as described below.
When \ref method-sampling is used in incremental mode by specifying
\ref method-sampling-refinement_samples, all results, including the \c moments group,
are placed within groups named \c increment:\<N\>, where \<N\> indicates
the increment number beginning with 1.

\htmlinclude sampling_moments.html
<br>
\htmlinclude moment_confidence_intervals.html

\subsection hdf5_results-correlations Correlations

A few different methods produce information about the correlations between
pairs of variables and responses (collectively: factors). The four tables
in this section describe how correlation information is stored. One important
note is that HDF5 has no special, native type for symmetric matrices, and so
the simple correlations and simple rank correlations are stored in dense 2D
datasets.

\htmlinclude simple_corr.html
<br>
\htmlinclude simple_rank_corr.html
<br>
\htmlinclude partial_corr.html
<br>
\htmlinclude partial_rank_corr.html

\subsection hdf5_results-pdf Probability Density

Some aleatory UQ methods estimate the probability density of resposnes.

\htmlinclude pdf.html

\subsection hdf5_results-level_mappings Level Mappings

Aleatory UQ methods can calculate level mappings (from user-specified probability,
reliability, or generalized reliability to response, or vice versa).

\htmlinclude prob_levels.html
<br>
\htmlinclude rel_levels.html
<br>
\htmlinclude gen_rel_levels.html
<br>
\htmlinclude response_levels.html
 
\subsection hdf5_results-vbd Variance-Based Decomposition (Sobol' Indices)
Dakota's \ref method-sampling method can produce main and total effects; stochastic
expansions (\ref method-polynomial_chaos, \ref method-stoch_collocation) additionally
can produce interaction effects.
\htmlinclude vbd_main.html
<br>
\htmlinclude vbd_total.html
Each order (pair, 3-way, 4-way, etc) of interaction is stored in a separate dataset. The
scales are unusual in that they are two-dimensional to contain the labels of the variables
that participate in each interaction.
\htmlinclude vbd_interactions.html


\subsection hdf5_results-se_moments Integration and Expansion Moments
Stochastic expansion methods can obtain moments two ways.
\htmlinclude int_moments.html
<br>
\htmlinclude exp_moments.html

\subsection hdf5_results-extreme_responses Extreme Responses
\ref method-sampling with epistemic variables produces extreme values (minimum and maximum)
for each response.
\htmlinclude extreme_responses.html

\subsection hdf5_results-pstudies Parameter Sets

All parameter studies (\ref method-vector_parameter_study, \ref method-list_parameter_study,
\ref method-multidim_parameter_study, \ref method-centered_parameter_study) record tables of
evaluations (parameter-response pairs), similar to Dakota's tabular output file. Centered 
parameter studies additionally store evaluations in an order that is more natural to intepret, 
which is described below.

In the tabular-like listing, variables are stored according to the scheme described in a 
\ref hdf5_results-variables "previous section".

\htmlinclude parameter_sets.html

\subsection hdf5_results-var_slices Variable Slices

Centered paramter studies store "slices" of the tabular data that make evaluating the
effects of each variable on each response more convenient. The steps for each individual 
variable, including the initial or center point, and corresponding responses are stored 
in separate groups.

\htmlinclude cps_var_slices_steps.html
<br>
\htmlinclude cps_var_slices_resps.html

\subsection hdf5_results-best_params Best Parameters

Dakota's optimization and calibration methods report the parameters at the best point
(or points, for multiple \ref method-final_solutions "final solutions") discovered. These
are stored using the scheme decribed in the \ref hdf5_results-variables "variables" section. When 
more than one solution is reported, the best parameters are nested in groups named \c set:\<N\>, 
where \<N\> is a integer numbering the set and beginning with 1.

State (and other inactive variables) are reported when using
\ref responses-objective_functions "objective functions" and for some 
\ref responses-calibration_terms "calibration" studies. However, when using configuration 
variables in a calibration, state variables are suppressed.

\htmlinclude best_parameters.html

\subsection hdf5_results-best_obj_fncs Best Objective Functions

Dakota's optimization methods report the objective functions at the best point (or points,
for multiple \ref method-final_solutions "final solutions") discovered. When 
more than one solution is reported, the best objective functions are nested in groups named \c set:\<N\>, 
where \<N\> is a integer numbering the set and beginning with 1.

\htmlinclude best_objective_functions.html

\subsection hdf5_results-best_constraints Best Nonlinear Constraints

Dakota's optimization and calibration methods report the nonlinear constraints at the best 
point (or points, for multiple \ref method-final_solutions "final solutions") discovered. When 
more than one solution is reported, the best constraints are nested in groups named \c set:\<N\>, 
where `N` is a integer numbering the set and beginning with 1.


\htmlinclude best_constraints.html

\subsection hdf5_results-calibration Calibration

When using \ref responses-calibration_terms "calibration terms" with an optimization method, or 
when using a nonlinear least squares method such as \ref method-nl2sol, Dakota reports residuals
and residual norms for the best point (or points, for multiple 
\ref method-final_solutions "final solutions") discovered.

\htmlinclude best_residuals.html
<br>
\htmlinclude best_residual_norm.html

\subsection hdf5_results-lsq_confidence_intervals Parameter Confidence Intervals

Least squares methods (\ref method-nl2sol, \ref method-nlssol_sqp, \ref method-optpp_g_newton)
compute confidence intervals on the calibration parameters.

\htmlinclude lsq_confidence_intervals.html

\subsection hdf5_results-calibration-no_config Best Model Responses (without configuration variables)

When performing calibration with experimental data (but no configruation variables),
Dakota records, in addition to the best residuals, the best original model resposnes.

\htmlinclude best_model_responses.html

\subsection hdf5_results-calibration-config Best Model Responses (with configuration variables)

When performing calibration with experimental data that includes configuration variables, Dakota
reports the best model responses for each experiment. These results include the configuration 
variables, stored in the scheme described in the \ref hdf5_results-variables "variables" section,
and the model responses.

\htmlinclude best_config_vars.html
<br>
\htmlinclude best_model_resps_config_vars.html

\subsection hdf5_results-ms_pareto Multistart and Pareto Set

The \ref method-multi_start and \ref method-pareto_set methods are meta-iterators that control
multiple optimization sub-iterators. For both methods, \c Dakota stores the results of the
sub-iterators (best parameters and best results). For \c multi_start, \c Dakota additionally 
stores the initial points, and for \c pareto_set, it stores the objective function weights.

\htmlinclude multistart_starting_points.html
<br>
\htmlinclude pareto_set_weights.html
<br>
\htmlinclude concurrent_best_parameters.html
<br>
\htmlinclude concurrent_responses.html

\section hdf5_output-hdf5_evaluations Organization of Evaluations

An *evaluation* is a mapping from variables to responses performed by a %Dakota model or interface.
Beginning with release 6.10, %Dakota has the ability to report evaluation history in HDF5 format. The 
HDF5 format offers many advantages over existing console output and 
\ref environment-tabular_data "tabular output". Requring no "scraping", it is more convenient for 
most users than the former, and being unrestricted to a two-dimensional, tabular arragnment of 
information, it is far richer than the latter.

This section begins by describing the %Dakota components that can generate evaluation data. It then 
documents the high-level organization of the data from those components. Detailed documentation of
the individual datasets (the "low-level" organization)  where data are stored follows. Finally,
information is provided concerning input keywords that control which components report evaluations.

\subsection hdf5_evaluations-hdf5_eval_sources Sources of Evaluation Data

Evaluation data are produced by only two kinds of components in %Dakota: **models** and **interfaces**.
The purpose of this subsection is to provide a basic description of models and interfaces for 
the purpose of equipping users to manage and understand HDF5-format evaluation data.

Because interfaces and models must be specified in even simple Dakota studies, most novice users 
of %Dakota will have some familiarity with these concepts. However, the exact nature of the
relationship between methods, models, and interfaces may be unclear. Moreover, the models and 
interfaces present in a Dakota study are not always limited to those specified by the user. Some 
input keywords or combinations of components cause Dakota to create new models or interfaces 
"behind the scenes" and without the user's direct knowledge. Not only can user-specified models 
and interfaces write evaluation data to HDF5, but also these auto-generated components. Accordingly, 
it may be helpful for consumers of Dakota's evaluation data to have a basic understanding 
of how Dakota creates and employs models and interfaces.

Consider first the input file shown here.
\verbatim
environment
  tabular_data
  results_output
    hdf5

method
  id_method 'sampling'
  sampling
    samples 20
  model_pointer 'sim'

model
  id_model 'sim'
  single
  interface_pointer 'tb'

variables
  uniform_uncertain 2
    descriptors 'x1' 'x2'
    lower_bounds 0.0 0.0
    upper_bounds 1.0 1.0

responses
  response_functions 1
    descriptors 'f'
  no_gradients
  no_hessians

interface
  id_interface 'tb'
  fork
    analysis_drivers 'text_book'
  
\endverbatim

This simple input file specifies a single method of type \ref method-sampling, which also has the Id 'sampling'.
The 'sampling' method possesses a \ref model of type \ref model-single (alias simulation) named 'sim', which it
uses to perform evaluations. (Dakota would have automatically generated a single model had one not been 
specified.) That is to say, for each variables-to-response mapping required by the method, it provides variables
to the model and receives back responses from it.

Single/simulation models like 'sim' perform evaluations by means of an interface, typically an interface to an 
external simulation. In this case, the interface is 'tb'. The model passes the variables to 'tb', which executes
the \c text_book driver, and receives back responses. 

It is clear that two components produce evaluation data in this study. The first is the single model 'sim', which 
receives and fulfills evaluation requests from the method 'sampling', and the second is the interface 'tb', which 
similarly receives requests from 'sim' and fulfills them by running the \c text_book driver.

Because \ref environment-tabular_data "tabular data" was requested in the environment block, a record of the model's
evaluations will be reported to a tabular file. The interface's evaluations could be dumped from
the restart file using \c dakota_restart_util.

If we compared these evaluation histories from 'sim' and 'tb', we would see that they are identical to one another.
The model 'sim' is a mere "middle man" whose only responsibility is passing variables from the method down 
to the interface, executing the interface, and passing responses back up to the method. However, this is not always
the case.

For example, if this study were converted to a gradient-based optimzation using \ref method-optpp_q_newton, and the user
specified \ref responses-numerical_gradients :
\verbatim

# model and interface same as above. Replace the method, variables, and responses with:

method
  id_method 'opt'
  optpp_q_newton

variables
  continuous_design 2
    descriptors 'x1' 'x2'
    lower_bounds 0.0 0.0
    upper_bounds 1.0 1.0

responses
   objective_functions 1
    descriptors 'f'
  numerical_gradients
  no_hessians

\endverbatim

Then the model would have the responsibility of performing finite differencing to estimate gradients of the 
response 'f' requested by the method. Multiple function evaluations of 'tb' would map to a single 
gradient evaluation at the model level, and the evaluation histories of 'sim' and 'tb' would contain different 
information. 

Note that because it is unwieldy to report gradients (or Hessians) in a tabular format, they are not written 
to the tabular file, and historically were avialable only in the console output. The HDF5 format provides convenient
access to both the "raw" evaluations performed by the interface and higher level model evaluations
that include estimated gradients.

This pair of examples hopefully provides a basic understanding of the flow of evaluation data between a method, model,
and interface, and explains why models and interfaces are producers of evaluation data.

Next consider a somewhat more complex study that includes a %Dakota model of type \ref model-surrogate. A surrogate model 
performs evaluations requested by a method by executing a special kind of interface called an *approximation interface*, 
which Dakota implicitly creates without the direct knowledge of the user. Approximation interfaces are a generic container 
for the various kinds of surrogates Dakota can use, such as \ref model-surrogate-global-gaussian_process 
"gaussian processes".

A Dakota model of type global surrogate may use a user-specified \ref model-surrogate-global-dace_method_pointer "dace method"
to construct the actual underlying model(s) that it evaluates via its approximation interface. The dace method will 
have its own model (typically of type single/simulation), which will have a user-specified interface.

In this more complicated case there are at least four components that produce evaluation data: (1) the surrogate model
and (2) its approximation interface, and (3) the dace method's model and (4) its interface. Although only components 
(1), (3), and (4) are user-specified, evaluation data produced by (2) may be written to HDF5, as well. (As 
\ref hdf5_evaluations-selection "explained below", only evaluations performed by the surrogate model and the dace 
interface will be recorded by default. This can be overriden using \ref environment-results_output-hdf5 sub-keywords.) 
This is an example where "extra" and potentially confusing data appears in Dakota's output due to an auto-generated 
component.

An important family of implicitly-created models is the *recast* models, which have the responsibility of transforming 
variables and responses. One type of recast called a *data transform model* is responsible for computing residuals when a 
user provides \ref responses-calibration_terms-calibration_data "experimental data" in a calibration study. *Scaling* recast 
models are employed when scaling is requested by the user for variables and/or responses.

Recast models work on the principle of function composition, and "wrap" a submodel, which may itself also be a recast model.
The innermost model in the recursion often will be the simulation or surrogate model specified by the user in the input file. 
%Dakota is capable of recording evaluation data at each level of recast.

\subsection hdf5_evaluations-high_level High-level Organization of Evaluation Data

This subsection describes how evaluation data produced by models and interfaces are organized at high level. A
detailed description of the datasets and subgroups that contain evaluation data for a specific model or interface
is given in the \ref hdf5_evaluations-low_level "next subsection".

Two top level groups contain evaluation data, \c /interfaces and \c /models.

\subsubsection hdf5_evaluations-high_level-interfaces Interfaces

Because interfaces can be executed by more than one model, interface evaluations are more precisely thought of as 
evaluations of an interface/model combination. Consequently, interface evaluations are grouped not only by interface
Id ('tb' in the example above), but also the Id of the model that requested them ('sim').

    /interfaces/<interface Id>/<model Id>/

If the user does not provide an Id for an interface that he specifies, Dakota assigns it the Id \c NO_ID. 
Approximation interfaces receive the Id \c APPROX_INTERFACE_\<N\>, where N is an incrementing integer 
beginning at 1. Other kinds of automatically generated interfaces are named \c NOSPEC_INTERFACE_ID_\<N\>.

\subsubsection hdf5_evaluations-high_level-models Models

The top-level group for model evaluations is \c /models. Within this group, model evaluations are grouped by type:
\c simulation, \c surrogate, \c nested, or \c recast, and then by model Id. That is:

    /models/<type>/<model Id>/    

Similar to interfaces, user-specified models that lack an Id are given one by Dakota. A single model is named 
\c NO_MODEL_ID. Some automatically generated models receive the name \c NOSPEC_MODEL_ID.

Recast models are a special case and receive the name \c RECAST_\<WRAPPED-MODEL\>_\<TYPE\>_\<N\>. 
In this string:
- \c WRAPPED-MODEL is the Id of the innermost wrapped model, typically a user-specified model
- \c TYPE is the specific kind of recast. The three most common recasts are:
  + \c RECAST: several generic responsibilities, including summing objective functions to present to a single-objective 
    optimizer
  + \c DATA_TRANSFORM: Compute residuals in a calibration
  + \c SCALING: scale variables and responses
- \c N is an incrementing integer that begins with 1. It is employed to distinguish recasts of the same type that 
  wrap the same underlying model.

The model's evaluations may be the result of combining information from multiple sources. A simulation/single model will 
receive all the information it requires from its interface, but more complicated model types may use information not only
from interfaces, but also other models and the results of method executions. Nested models, for instance, receive information 
from a submethod (the mean of a response from a sampling study, for instance) and potentially also an 
\ref model-nested-optional_interface_pointer "optional interface".

The sources of a model's evaluations may be roughly identified by examining the contents of that models' \c sources group. 
The \c sources group contains softlinks (note: softlinks are an HDF5 feature analogous to soft or symbolic links on many
file systems) to groups for the interfaces, models, or methods that the model used to produce its 
evaluation data. (At this time, Dakota does not report the specific interface or model evaluations or method executions that
were used to produce a specific model evaluation, but this is a planned feature.)

Method results likewise have a \c sources group that identifies the models or methods employed by that method. By following 
the softlinks contained in a method's or model's \c sources group, it is possible to "drill down" from a method to its ultimate 
sources of information. In the sampling example above, interface evaluations performed via the 'sim' model at the request 
of the 'sampling' method could be obtained at the HDF5 path: \c /methods/sampling/sources/sim/sources/tb/

\subsection hdf5_evaluations-low_level Low-Level Organization of Evaluation Data

Within each model and interface's "high-level" group, evaluation data are stored according to a "low-level" schema. This section desribes the "low-level" schema.

Data are divided first of all into \c variables, \c responses, and \c properties groups. In addition, if a
a user specifies metadata responses in his Dakota input, a \c metadata dataset will be present.

\subsubsection hdf5_evaluations-low_level-variables Variables

The \c variables group contains datasets that store the variables information for each evaluation.
Four datasets may be present, one for each "domain": \c continuous, \c discrete_integer, 
\c discrete_string, and \c discrete_real. These datasets are two-dimensional, with a row (0th dimension) 
for each evaluation and a column (1st dimension) for each variable. The 0th dimension has one dimension scale
for the integer-valued evaluation Id. The 1st dimension has two scales. The 0th scale contains descriptors
of the variables, and the 1st contains their variable Ids. In this context, the Ids are a 1-to-N ranking
of the variables in Dakota "input spec" order.

\htmlinclude evaluation_variables_table.html

\subsubsection hdf5_evaluations-low_level-responses Responses

The responses group contains datasets for functions and, when available, gradients and Hessians. 

\b Functions: The \c functions dataset is two-dimensional and contains function values for all responses. Like 
the variables datasets, evaluations are stored along the 0th dimension, and responses are stored along the 1st. 
The evaluation Ids and response descriptors are attached as scales to these axes, respectively.

\htmlinclude evaluation_functions_table.html

\b Gradients: The \c gradients dataset is three-dimensional. It has the shape 
 \f$ evaluations \times responses \times variables\f$. Dakota supports a specification of 
\ref responses-mixed_gradients, 
and the \c gradients dataset is sized and organized such that only those responses for which gradients are 
available are stored. When \c mixed_gradients are employed, a response will not necessarily have the same
index in the \c functions and \c gradients datasets.

Because it is possible that the gradient could be computed with respect to any of the continuous variables, active
or inactive, that belong to the associated model, the \c gradients dataset is sized to accomodate gradients taken
with respect to all continuous variables. Components that were not included in a particular evaluation will be set 
to NaN (not a number), and the \c derivative_variables_vector (in the matadata group) for that evaluation can be 
examined as well.

\htmlinclude evaluation_gradients_table.html

\b Hessians: Hessians are stored in a four-dimensional dataset, \f$ evaluations \times responses \times 
 \times variables \times variables \f$.
The \c hessians dataset shares many of the characteristics with the \c gradients: in the mixed_hessians case, it will 
be smaller in the response dimension than the \c functions dataset, and unrequested components are set to NaN.

\htmlinclude evaluation_hessians_table.html

\subsubsection hdf5_evaluations-low_level-properties Properties

The \c properties group contains up to four members.

<b>Active Set Vector:</b> The first is the \c active_set_vector dataset. It is two dimensional, with
rows corresponding to evaluations and columns corresponding to responses. Each element contains an integer in
the range 0-7, which indicates the request (function, gradient, Hessian) for the corresponding response for 
that evaluation. The 0th dimension has the evaluations Ids scale, and the 1st dimension has two scales: the 
response descriptors and the "default" or "maximal" ASV, an integer 0-7 for each response that indicates
the information (function, gradient, Hessian) that possibly could have been requested during the study.

\htmlinclude evaluation_asv_table.html

<b>Derivative Variables Vector:</b> The second item in the properties group is the \c derivative_variables_vector 
dataset. It is included only when gradients or Hessians are available. Like the ASV, it is two-dimensional. Each 
column of the DVV dataset corresponds to a continuous variable and contains a 0 or 1, indicating whether gradients 
and Hessians were computed with respect to that variaable for the evaluation. The 0th dimension has the evaluation 
Ids as a scale, and the 1st dimension has two scales. The 0th is the descriptors of the continuous variables. The 1st 
contains the variable Ids of the continuous variables.

\htmlinclude evaluation_dvv_table.html

<b>Analysis Components:</b> The third member of the properties group is the \c analysis_components dataset. It is a 1D 
dataset that is present only when the user specified analysis components, and it contains those components as strings.

\htmlinclude evaluation_an_comps_table.html

The final possible member of the properties group is the \c variable_parameters group. It is included only for models,
which possess variables, and is described in a separate section below.

\subsubsection hdf5_evaluations-low_level-metadata Metadata

Beginning with release 6.16, Dakota supports response \ref responses-metadata. If configured, metadata values 
are stored in the \c metadata dataset.

\htmlinclude evaluation_metadata_table.html

\subsection hdf5_evaluations-selection Selecting Models and Interfaces to Store

When HDF5 output is enabled (by including the \ref environment-results_output-hdf5 keyword), then by default 
evaluation data for the following components will be stored:

- The model that belongs to the top-level method. (Currently, if the top-level method is a metaiterator such as
  \c method-hybrid, no model evaluation data will be stored.)
- All simulation interfaces. (interfaces of type \ref interface-analysis_drivers-fork, 
  \ref interface-analysis_drivers-system, \ref interface-analysis_drivers-direct, etc).

The user can override these defaults using the keywords \ref environment-results_output-hdf5-model_selection and 
\ref environment-results_output-hdf5-interface_selection.

The choices for \c model_selection are:
- \ref environment-results_output-hdf5-model_selection-top_method : (*default*) Store evaluation data for the top method's
  model only.
- \ref environment-results_output-hdf5-model_selection-all_methods : Store evaluation data for all models that belong 
  directly to a method. Note that a these models may be recasts of user-specified models, not the user-specified
  models themselves.
- \ref environment-results_output-hdf5-model_selection-all : Store evaluation data for all models.
- \ref environment-results_output-hdf5-model_selection-none : Store evaluation data for no models.

The choices for \c interface_selection are:
- \ref environment-results_output-hdf5-interface_selection-simulation : (*default*) Store evaluation data for simulation
  interfaces.
- \ref environment-results_output-hdf5-interface_selection-all : Store evaluation data for all interfaces.
- \ref environment-results_output-hdf5-interface_selection-none : Store evaluation data for no interfaces.

If a model or interface is excluded from storage by these selections, then they cannot appear in the \c sources group
for methods or models.

\section hdf5_output-hdf5_dist_params Distribution Parameters


Variables are characterized by parameters such as the mean and standard
deviation or lower and upper bounds. Typically, users provide these parameters
as part of their input to Dakota, but Dakota itself may also compute them as it
scales and transforms variables, normalizes empirical distributions (e.g. for
\c histogram_bin_uncertain variables), or calculates alternative parameterizations
(lambda and zeta vs mean and standard deviation for a \c lognormal_uncertain).

Beginning with release 6.11, models write their variable’s parameters to HDF5.
The information is located in each model's \c properties/variable_parameters subgroup. 
Within this group, parameters are stored by Dakota variable type (e.g.
\c normal_uncertain), with one 1D dataset per type. The datasets have the same
names as their variable types and have one element per variable. Parameters are
stored by name.

Consider the following variable specification, which includes two normal and
two uniform variables:
\verbatim
  variables 
    normal_uncertain 2 
      descriptors 'nuv_1' 'nuv_2' 
      means 0.0 1.0
      std_devations 1.0 0.5
    uniform_uncertain 2
    descriptors 'uuv_1' 'uuv_2'
      lower_bounds -1.0 0.0
      upper_bounds  1.0 1.0
\endverbatim

Given this specification, and assuming a model ID of “tb_model”, Dakota will
write two 1D datasets, both of length 2, to the group
\c /models/simulation/tb_model/metadata/variable_parameters, the first named
\c normal_uncertain, and the second named \c uniform_uncertain. Using a JSON-like
representation for illustration, the \c normal_uncertain dataset will appear as:

\verbatim
  [
    {
      "mean": 0.0,
      "std_deviation": 1.0,
      "lower_bound": -inf, 
      "upper_bound": inf
    },
    { 
      "mean": 1.0,
      "std_deviation": 0.5,
      "lower_bound": -inf,
      "upper_bound": inf
    }
  ]
\endverbatim

The \c uniform_uncertain dataset will contain:
\verbatim
  [
    {
      "lower_bound": -1.0, 
      "upper_bound":  1.0
    },
    { 
      "lower_bound": 0.0,
      "upper_bound": 1.0
    }
  ]
\endverbatim

In these representations of the \c normal_uncertain and \c uniform_uncertain
datasets, the outer square brackets ([]) enclose the dataset, and each element
within the datasets are enclosed in curly braces ({}). The curly braces are
meant to indicate that the elements are dictionary-like objects that
support access by string field name. A bit more concretely, the following code
snippet demonstrates reading the mean of the second normal variable, nuv_2.

\code{.py}
import h5py 

with h5py.File("dakota_results.h5') as h:
    model = h["/models/simulation/tb_model/"]
    # nu_vars is the dataset that contains distribution parameters for
    # normal_uncertain variables
    nu_vars = model["variable_parameters/normal_uncertain"]
    nuv_2_mu = nu_vars[1]["mean"] # 1 is the 0-based index of nuv_2, and 
                                  # "mean" is the name of the field where 
                                  # the mean is stored; nuv_2_mu now contains
                                  # 1.0.
\endcode

The feature in HDF5 that underlies this name-based storage of fields is
compound datatypes, which are similar to C/C++ structs or Python dictionaries.
Further information about how to work with compound datatypes is available in
the h5py documentation.

\subsection hdf5_dist_params-conventions Naming Conventions and Layout

In most cases, datasets for storing parameters have names that match their
variable types. The \c normal_uncertain and \c uniform_uncertain datasets illustrated
above are examples. Exceptions include types such as \ref variables-discrete_design_set,
which has string, integer, and real subtypes. For these, the dataset name is the
top-level type with \c _string, \c _int, or \c _real appended:
\c discrete_design_set_string, \c discrete_design_set_int, and
\c discrete_design_set_real.

Most Dakota variable types have scalar parameters. For these, the names of the
parameters are generally the singular form of the associated Dakota keyword.
For example, \ref variables-triangular_uncertain variables are characterized in 
Dakota input using the plural keywords \c modes, \c lower_bounds, and 
\c upper_bounds. The singular field names are, respectively, "mode", 
"lower_bound", and "upper_bound". In this case, all three parameters are 
real-valued and stored as floating point numbers, but variable types/fields can
also be integer-valued (e.g. \c binomial_uncertain/num_trials) or string-valued.

Some variable/parameter fields contain 1D arrays or vectors of information.
Consider \ref variables-histogram_bin_uncertain variables, for which the user 
specifies not just one value, but an ordered collection of abscissas and 
corresponding ordinates or counts. Dakota stores the abscissas in the 
"abscissas" field, which is a 1D dataset of floating-point numbers. It
similarly stores the counts in the "counts" field. (In this case, only the 
normalized counts are stored, regardless of whether the user provided counts or
ordinates.)

When the user specifies more than one \c histogram_bin_uncertain variable, it
often is also necessary to include the 
\ref variables-histogram_bin_uncertain-pairs_per_variable keyword to divide the
\c abscissa/count pairs among the variables. This raises the question of how 
lists of parameters that vary in length across the variables ought to be stored.

Although HDF5 supports variable-length datasets, for simplicity (and due to
limitations in h5py at the time of the 6.11 release), Dakota stores vector
parameter fields in conventional fixed-length datasets. The lengths of these
datasets are determined at runtime in the following way: For a particular
variable type and field, the field for all variables is sized to be large
enough to accommodate the variable with the longest list of parameters. Any
unused space for a particular variable is filled with \c NaN (if the parameter
is real-valued), \c INTMAX (integer-valued), or an empty string (\c
string-valued). In addition, each variable has an additional field,
"num_elements", that reports the number of elements in the fields that contain
actual data and not fill values.

Consider this example, in which the user has specified a pair of
\c histogram_bin_uncertain variables. The first has 3 pairs, and the second has 4.

\verbatim
  variables
    histogram_bin_uncertain 2
      pairs_per_variable 2 3
      abscissas  0.0   0.5  1.0 
                -1.0  -0.5  0.5  1.0 
      counts     0.25  0.75 0.0 
                 0.2   0.4  0.2  0.0
\endverbatim

For this specification, Dakota will write a dataset named
\c histogram_bin_uncertain to the \c metadata/variable_parameters/ subgroup for the
model. It will be of length 2, one element for each variable, and contain the
following:
\verbatim
  [
    {
      "num_elements": 3,
      "abscissas": [0.0, 0.5, 1.0, NaN],
      "counts": [0.25, 0.75, 0.0, NaN]
    },
    {
      "num_elements": 4,
      "abscissas": [-1.0, -0.5, 0.5, 1.0],
      "counts": [0.2, 0.4, 0.2, 0.0]
    }
  ]
\endverbatim

\subsection hdf5_dist_params-h5py_examples h5py Examples

The fields available for a variable parameters dataset can be determined in
h5py by examining the datatype of the dataset.
\code{.py}
import h5py
with h5py.File("dakota_results.h5") as h:
    model = h["/models/simulation/NO_MODEL_ID/"]
    md = model["metadata/variable_parameters"]
    nu = md["normal_uncertain"]
    nu_param_names = nu.dtype.names 
    # nu_param_names is a tuple of strings: ('mean', 'std_deviation',
    # 'lower_bound', 'upper_bound')
\endcode

\subsection hdf5_dist_params-limitations Known Limitations

h5py has a known bug that prevents parameters for some types of variables from
being accessed (the Python interpreter crashes with a segfault). These include:
- \c histogram_point_uncertain string
- \c discrete_uncertain_set string

\subsection hdf5_dist_params-metadata Metadata

The variable parameter datasets have two dimension scales. The first (index 0)
contains the variable descriptors, and the second (index 1) contains variable
Ids.  Available Parameters

\subsection hdf5_dist_params-allparams Parameter Listing for All Types

The table below lists all Dakota variables and parameters that can be stored.

\htmlinclude dist_params_table.html
