Blurb::
Calculate the Kullback-Leibler Divergence between prior and posterior

Description::
The Kullback-Leibler (KL) Divergence, also called the relative entropy, 
provides a measure of the difference between two probability distributions. 
By specifying \c kl_divergence, the KL Divergence between the posterior 
\f$f(\boldsymbol{\theta} | \textbf{y}^{Data})\f$ and the prior 
\f$f(\boldsymbol{\theta})\f$ parameter distributions is calculated such that 

\f[
D_{KL} = \int f(\boldsymbol{\theta} | \textbf{y}^{Data} ) \log 
\frac{ f(\boldsymbol{\theta} | \textbf{y}^{Data}) }{ f(\boldsymbol{\theta}) }
d\boldsymbol{\theta}
\f]

This quantity can be interpreted as the amount of information gained about the 
parameters during the Bayesian update. 

<b> Expected Output </b>

If \c kl_divergence is specified, the calculated value will be reported to the
screen at the end of the calibration, following the sample statistics of the 
response functions. Example output is given below.

<b> Additional Discussion </b> 

The quantity calculated is a \f$k\f$-nearest neighbor approximation of the 
possibly multi-dimensional integral given above. Therefore, some applications
whose true KL Divergence is quite close to zero may report a negative KL 
Divergence. 

Examples::
Below is a \c method block of a Dakota input file that indicates the calculation
of the KL Divergence

\verbatim
method,
       bayes_calibration queso
          dram
          seed = 34785
          chain_samples = 1000
          posterior_stats kl_divergence
\endverbatim

The calculated KL approximation is indicated in the screen output by 
"Information gained from prior to posterior" as shown below

\verbatim
Sample moment statistics for each response function:
                            Mean           Std Dev          Skewness          Kurtosis
least_sq_term_1  3.9982462078e-01  4.7683816550e-04 -2.3448518080e+00  7.7381497770e+00

Information gained from prior to posterior = 1.0066819600e+01

<<<<< Iterator bayes_calibration completed.
<<<<< Environment execution completed. 
\endverbatim
