8. Run an uncertainty quantification

The uncertainty quantification procedure provides the mean and standard deviation of the quantity of interest and the Sobol’ indices by constructing a Polynomial Chaos Expansion (PCE). More information on PCE is available in Polynomial Chaos Expansion. The mean value and the corresponding uncertainty of the model parameters are characterized in design_space.csv and stochastic_space.csv, respectively. More information on characterizing these files is available in The design_space.csv file and The stochastic_space.csv file, respectively. The model returns the values for the quantity of interest through the function evaluate() defined in case_description. More information on this Python wrapper is discussed in Python wrapper.

8.1. The uncertainty quantification dictionary

To run the uncertainty quantification, first the uncertainty quantification module should be imported:

import rheia.UQ.uncertainty_quantification as rheia_uq

To characterize the uncertainty quantification, the following dictionary with parameters related to the case and uncertainty quantification should be completed:

dict_uq = {'case':                  case_name,
           'pol order':             pol_order,
           'objective names':       obj_names,
           'objective of interest': obj_of_interest,
           'results dir':           directory,

           'sampling method':       sampling_method,        #optional, default is 'SOBOL'
           'create only samples':   only_samples_bool,      #optional, default is False
           'draw pdf cdf':          [draw_bool, n_samples], #optional, default is [False]
           'n jobs':                n_jobs,                 #optional, default is 1
           }

The items of the uncertainty quantification dictionary are described in the following subsections. This dictionary is used as the argument for the :py:func`run_uq` function, which initiates the uncertainty quantification procedure:

rheia_uq.run_uq(dict_uq)

8.1.1. Necessary items

In the following subsections, the necessary items are described. If one of these items is not provided, the code will return an error.

8.1.1.1. ‘case’: case_name

The string case_name corresponds to the name of the case. This name should be equal to the name of the folder that comprises the case, which situates in the folder that contains the cases (i.e. CASES). To illustrate, if the case is defined in CASES\CASE_1, the dictionary includes the following item:

'case': 'CASE_1'

8.1.1.2. ‘pol order’: pol_order

The polynomial order corresponds to the maximum polynomial degree in the PCE. The polynomial order is characterized by an integer, e.g. for a polynomial order of 2:

'pol order': 2

8.1.1.3. ‘objective names’: obj_names

A PCE is constructed for only 1 quantity of interest. However, the statistical moments for several model outputs can be of interest. To avoid that for each model output, a new set of model evaluations needs to be performed, different model outputs can be stored for each model evaluation. The names of the different model outputs can be provided in the list 'objective_names'. These names are chosen freely by the user, formatted in a string. If the model returns 3 outputs, the list can be constructed as:

'objective names': ['output_1', 'output_2', 'output_3']

8.1.1.4. ‘objective of interest’: obj_of_interest

Despite that several outputs can be returned for each model evaluation, only one output can be selected as a quantity of interest for the PCE. The name of this quantity of interest (obj_of_interest) should be provided. This name should be present in the list of all the objective names. To illustrate, if the quantity of interest is 'output_2', then the item in the dictionary is configurated as:

'objective of interest': 'output_2'

8.1.1.5. ‘results dir’: directory

The results directory corresponds to the folder where the results are stored. For an illustrative case 'CASE_1', the UQ results are saved in the folder RESULTS\CASE_1\UQ\results_1 by initiating the following item in the dictionary:

'results dir': results_1

8.1.2. Optional items

The following items are optional items. If one of these items is not provided in the dictionary, a default value will be assigned to the key. If none of these are provided, the optional dictionary items are defined as follows:

'sampling method':       'SOBOL',
'create only samples':   False,
'draw pdf cdf':          [False],
'n jobs':                1,

8.1.2.1. ‘sampling method’: sampling_method

For the construction of a PCE, a number of model evaluation are required. The samples for model evaluation can be generated in two different ways: randomly, or through a Sobol’ sequence. The random generation is called through the string 'RANDOM', while the Sobol’ sequence is initiated through 'SOBOL'. The default configuration for generating the samples for PCE is through a Sobol’ sequence:

'sampling method': 'SOBOL'

8.1.2.2. ‘create only samples’: only_samples_bool

In some cases, the coupling of the system model with the framework is complex. To avoid this coupling, the samples required to determine the statistical moments can be generated without initiating a model evaluation. Hence, the framework should only generate the samples. To do so, the Boolean only_samples_bool can be set to True:

'create only samples': True

However, the default configuration sets the value of 'create only samples' to False:

'create only samples': False

Additional information on how to create just the samples is present in Create samples for an unconnected model.

8.1.2.3. ‘draw pdf cdf’: [draw_bool, n_samples]

In addition to the statistical moments, the data for generation the probability density function (pdf) and cumulative distribution function (cdf) can be generated. This information can be generated by setting the draw_bool to True and providing the number of samples evaluated on the PCE n_samples. To illustrate, to generate pdf and cdf datapoints based on a Monte Carlo evaluation on the PCE surrogate with 100,000 samples:

'draw pdf cdf': [True, 1000000]

In the default configuration, the pdf and cdf are not generated:

'draw pdf cdf': [False]

Reasonable results can be expected on the pdf and cdf when considering 100,000 samples or more. As these samples are evaluated on the PCE, the computational cost of generating the pdf and cdf is negligible.

8.1.2.4. ‘n jobs’: n_jobs

The number of parallel processes can be defined by the number of available cores on the CPU. The default value corresponds to linear processing:

'n jobs': 1

Alternatively, the number of parallel processes can be retreived through the cpu_count function from the multiprocessing package. After importing multiprocessing, the item can be defined by:

'n jobs': int(multiprocessing.cpu_count()/2)

8.1.3. Example of a dictionary for uncertainty quantification

When combining the examples in the previous section, a configurated uncertainty quantification dictionary with the necessary items looks as follows:

import rheia.UQ.uncertainty_quantification as rheia_uq

dict_uq = {'case': 'CASE_1',
           'pol order': 2,
           'objective names': ['output_1', 'output_2', 'output_3'],
           'objective of interest': 'output_2',
           'results dir': 'results_1'
           }

rheia_uq.run_uq(dict_uq)

Alternatively, an uncertainty quantification dictionary which considers random sampling and generates 100,000 PDF and CDF samples on the PCE surrogate:

import rheia.UQ.uncertainty_quantification as rheia_uq

dict_uq = {'case': 'CASE_1',
           'pol order': 2,
           'objective names': ['output_1', 'output_2', 'output_3'],
           'objective of interest': 'output_2',
           'results dir': 'results_1',
           'sampling method': 'RANDOM',
           'draw pdf cdf': [True, 1000000],
           }

rheia_uq.run_uq(dict_uq)

The post-processing of the results is described in lab:uqresults.

8.2. Create samples for an unconnected model

When it is burdensome to connect the system model to the framework, the framework provides the option to generate the random samples for uncertainty quantification without a model evaluation. As the generation of these random samples is based on the characterization of the uncertainties, a design_space.csv file and stochastic_space.csv file have to be defined. To generate the samples, use (or make a copy of) the NO_MODEL folder in CASES. In this folder, a case_description module is present, as well as design_space.csv and stochastic_space.csv. The case_description is empty, as no model evaluations are required. In design_space.csv and stochastic_space.csv, the stochastic design space of interest is defined. The samples can be generated as follows:

import rheia.UQ.uncertainty_quantification as rheia_uq

dict_uq = {'case':                  'NO_MODEL',
           'pol order':             2,
           'objective names':       ['output_1', 'output_2', 'output_3'],
           'objective of interest': 'output_2',
           'results dir':           'results_1',
           'create only samples':   True,
           }

rheia_uq.run_uq(dict_uq)

For this example, the samples are written in RESULTS\NO_MODEL\UQ\results_1\samples. Once these samples are evaluated in the model on an external location, the results can be added to the RESULTS\NO_MODEL\UQ\results_1\samples file. When the results are added for ‘output_1’, ‘output_2’, ‘output_3’, the PCE can be constructed for the three quantities of interest. In that case, the value for ‘create only samples’ is set back to False (i.e. the default value). To illustrate, for a PCE on ‘output_2’:

import rheia.UQ.uncertainty_quantification as rheia_uq

dict_uq = {'case':                  'NO_MODEL',
           'pol order':             2,
           'objective names':       ['output_1', 'output_2', 'output_3'],
           'objective of interest': 'output_2',
           'results dir':           'results_1',
           }

rheia_uq.run_uq(dict_uq)

Warning

Make sure that the result directory is equal to the result directory where the updated samples file is saved.

8.3. Post-processing of the results

The results path depends on the case name (e.g. CASE_1), the analysis type (UQ) and the results directory (e.g. results_1), i.e. \RESULTS\CASE_1\UQ\results_1. In this folder, at least 1 file is present: the samples file. This file includes the samples and the corresponding deterministic model response, when a system model is connected to the framework (i.e. ‘create only samples’ set to False). The second file and third file are named based on the selected maximum polynomial degree and the quantity of interest (e.g. full_pce_order_2_output_2 and full_pce_order_2_output_2_Sobol_indices for a polynomial order 2 PCE for the quantity of interest output_2). These files respectively include the PCE information (LOO error, mean and standard deviation) and the Sobol indices (first order and total order).

The Sobol’ indices can be represented in a bar chart:

import rheia.POST_PROCESS.post_process as rheia_pp
import matplotlib.pyplot as plt

case = 'case_name'

pol_order = 1

my_post_process_uq = rheia_pp.PostProcessUQ(case, pol_order)

result_dir = 'sample_0'

objective = 'output_2'

names, sobol = my_post_process_uq.get_sobol(result_dir, objective)

plt.barh(names, sobol)
plt.show()

The LOO-error can be extracted:

loo = my_post_process_uq.get_loo(result_dir, objective)

If the data for the Probability Density Function (PDF) and Cumulative Distribution Function (CDF) was generated, both functions can be plotted as follows:

x_pdf, y_pdf = my_post_process_uq.get_pdf(result_dir, objective)
x_cdf, y_cdf = my_post_process_uq.get_cdf(result_dir, objective)