Merge pull request #84 from NREL/bsd_cal_tut

Add BSD calibration tutorial and result to docs
NREL · Jan 15, 2025 · 2fe0740 · 2fe0740
commit 2fe0740
Show file tree

Hide file tree

Showing 131 changed files with 8,818 additions and 0 deletions.
diff --git a/.buildinfo b/.buildinfo
@@ -0,0 +1,4 @@
+# Sphinx build info version 1
+# This file records the configuration used when building these files. When it is not found, a full rebuild will be done.
+config: 5323d8be09131c0fa917404abdc54afd
+tags: 645f666f9bcd5a90fca523b33c5a78b7
diff --git a/.nojekyll b/.nojekyll
diff --git a/_images/3dsparger.png b/_images/3dsparger.png
diff --git a/_images/Loss_surr.png b/_images/Loss_surr.png
diff --git a/_images/Loss_surr1.png b/_images/Loss_surr1.png
diff --git a/_images/Surr_cal_2.0_5.0_corner.png b/_images/Surr_cal_2.0_5.0_corner.png
diff --git a/_images/Surr_cal_2.0_5.0_prop.png b/_images/Surr_cal_2.0_5.0_prop.png
diff --git a/_images/Surr_cal_5.0_5.0_corner.png b/_images/Surr_cal_5.0_5.0_corner.png
diff --git a/_images/Surr_cal_5.0_5.0_prop.png b/_images/Surr_cal_5.0_5.0_prop.png
diff --git a/_images/Surr_cal_binary_corner.png b/_images/Surr_cal_binary_corner.png
diff --git a/_images/Surr_cal_binary_prop.png b/_images/Surr_cal_binary_prop.png
diff --git a/_images/Surr_cal_ternary_corner.png b/_images/Surr_cal_ternary_corner.png
diff --git a/_images/Surr_cal_ternary_prop.png b/_images/Surr_cal_ternary_prop.png
diff --git a/_images/Surr_opt_2.0_5.0_corner.png b/_images/Surr_opt_2.0_5.0_corner.png
diff --git a/_images/Surr_opt_2.0_5.0_prop.png b/_images/Surr_opt_2.0_5.0_prop.png
diff --git a/_images/Surr_opt_5.0_5.0_corner.png b/_images/Surr_opt_5.0_5.0_corner.png
diff --git a/_images/Surr_opt_5.0_5.0_prop.png b/_images/Surr_opt_5.0_5.0_prop.png
diff --git a/_images/Surr_opt_binary_corner.png b/_images/Surr_opt_binary_corner.png
diff --git a/_images/Surr_opt_binary_prop.png b/_images/Surr_opt_binary_prop.png
diff --git a/_images/Surr_opt_ternary_corner.png b/_images/Surr_opt_ternary_corner.png
diff --git a/_images/Surr_opt_ternary_prop.png b/_images/Surr_opt_ternary_prop.png
diff --git a/_images/True_cal_2.0_5.0_corner.png b/_images/True_cal_2.0_5.0_corner.png
diff --git a/_images/True_cal_2.0_5.0_prop.png b/_images/True_cal_2.0_5.0_prop.png
diff --git a/_images/True_cal_5.0_5.0_corner.png b/_images/True_cal_5.0_5.0_corner.png
diff --git a/_images/True_cal_5.0_5.0_prop.png b/_images/True_cal_5.0_5.0_prop.png
diff --git a/_images/True_opt_2.0_5.0_corner.png b/_images/True_opt_2.0_5.0_corner.png
diff --git a/_images/True_opt_2.0_5.0_prop.png b/_images/True_opt_2.0_5.0_prop.png
diff --git a/_images/True_opt_5.0_5.0_corner.png b/_images/True_opt_5.0_5.0_corner.png
diff --git a/_images/True_opt_5.0_5.0_prop.png b/_images/True_opt_5.0_5.0_prop.png
diff --git a/_images/cmap_bf_binary.png b/_images/cmap_bf_binary.png
diff --git a/_images/cmap_bf_ternary.png b/_images/cmap_bf_ternary.png
diff --git a/_images/cmap_cr_binary.png b/_images/cmap_cr_binary.png
diff --git a/_images/cmap_cr_ternary.png b/_images/cmap_cr_ternary.png
diff --git a/_images/co2g_cond_mean.png b/_images/co2g_cond_mean.png
diff --git a/_images/early_det.png b/_images/early_det.png
diff --git a/_images/early_uq.png b/_images/early_uq.png
diff --git a/_images/gh_cond_mean.png b/_images/gh_cond_mean.png
diff --git a/_images/loop_react.png b/_images/loop_react.png
diff --git a/_images/schematic.png b/_images/schematic.png
diff --git a/_images/schematicSpargerInlet.png b/_images/schematicSpargerInlet.png
diff --git a/_images/schematicSpargerWalls.png b/_images/schematicSpargerWalls.png
diff --git a/_images/simpleOutput.png b/_images/simpleOutput.png
diff --git a/_images/stirred_tank.png b/_images/stirred_tank.png
diff --git a/_images/succ_fail.png b/_images/succ_fail.png
diff --git a/_images/test_surr.png b/_images/test_surr.png
diff --git a/_sources/acknowledgments.rst.txt b/_sources/acknowledgments.rst.txt
@@ -0,0 +1,6 @@
+Acknowledgments
+=====
+
+This work was authored by the National Renewable Energy Laboratory (NREL), operated by Alliance for Sustainable Energy, LLC, for the U.S. Department of Energy (DOE) under Contract No. DE-AC36-08GO28308. This work was supported by funding from DOE Bioenergy Technologies Office (BETO) `CO2RUe consortium <https://www.energy.gov/eere/co2rue>`_. The research was performed using computational resources sponsored by the Department of Energy's Office of Energy Efficiency and Renewable Energy and located at the National Renewable Energy Laboratory. The views expressed in the article do not necessarily represent the views of the DOE or the U.S. Government. The U.S. Government retains and the publisher, by accepting the article for publication, acknowledges that the U.S. Government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this work, or allow others to do so, for U.S. Government purposes.
+
+
diff --git a/_sources/bubbleColumn.rst.txt b/_sources/bubbleColumn.rst.txt
@@ -0,0 +1,14 @@
+Bubble column
+=====
+
+This tutorial demonstrates how to run a bubble column reactor case
+
+Running the tutorial
+------------
+
+To be completed ...
+
+
+
+
+
diff --git a/_sources/calibration.rst.txt b/_sources/calibration.rst.txt
@@ -0,0 +1,242 @@
+Parameter calibration for Normal and Beta distributions
+=====
+
+This tutorial demonstrates how to conduct the calibration with and without a surrogate forward model
+
+Running the tutorial
+------------
+We assume that bird is installed within an environment called ``bird`` (see either the :ref:`Installation section for developers<installation_dev>` or the :ref:`Installation section for users<installation_users>`)
+
+You will need to install additional dependencies
+
+.. code-block:: console
+
+   conda activate bird
+   BIRD_DIR = `python -c "import bird; print(bird.BIRD_DIR)"`
+   cd ${BIRD_DIR}/../
+   pip install .[calibration]
+
+The calibration tutorial is located at ``${BIRD_DIR}/../tutorial_cases/calibration/normal_beta``
+
+Calibration Objective
+------------
+
+The PDF of a Beta(alpha,beta) distribution is observed.
+Within the context of bioreactors, one can consider this PDF to be a measured bubble size distribution (BSD)
+
+A separate dummy numerical model is available and predicts BSD as normal distributions N(mu, sigma). The input parameter of the numerical models are mu and sigma. The objective of the modeler is to find mu and sigma such that the predicted BSD matches the observed BSD
+
+We assume that the numerical model is expensive and instead of using the expensive numerical model for the forward model used in the calibration, one would like to use a cheaper surrogate model. Here the surrogate model is a neural network.
+
+
+Building the surrogate
+------------
+
+The class ``Param_NN`` available in BiRD allows to create a parametric surrogate. Here, the input of the surrogate are the parameters ``mu`` and ``sigma``, and the variable ``x`` that parameterize the PDF.
+
+Constructing such a surrogate is not computationally useful here since the forward simulation is cheap. This is only for illustrative purposes!
+
+Generate dataset and train neural net surrogate with ``python tut_surrogate.py``.
+The following plots should be generated. The first one shows the training and testing loss convergence. The second one shows examples of the surrogate accuracy on unseen data.
+
+
+.. container:: figures-surr-train
+
+   .. figure:: ../assets/calibration/tutorial_normal_beta/Loss_surr.png
+      :width: 30%
+      :align: center
+      :alt: Loss history
+
+   .. figure:: ../assets/calibration/tutorial_normal_beta/test_surr.png
+      :width: 90%
+      :align: center
+      :alt: Test samples
+
+
+Calibration with optimized likelihood uncertainty
+------------
+
+For the calibration, we can use the true function or the surrogate model. The objective PDF is assumed to be noiseless. Even though the observation is noiseless, an uncertainty is computed to account for the missing physics.
+The likelihood uncertainty that represents the missing physics is optimized to ensure an uncertainty band overlap
+
+Calibrate without a surrogate for ``alpha=5, beta=5``
+
+
+.. code-block:: console
+
+   python tut_calibration.py --alpha 5 --beta 5
+
+
+.. container:: figures-cal-True-5-5
+
+   .. figure:: ../assets/calibration/tutorial_normal_beta/True_opt_5.0_5.0_prop.png
+      :width: 50%
+      :align: center
+      :alt: Calibrated prediction with the true forward model
+
+   .. figure:: ../assets/calibration/tutorial_normal_beta/True_opt_5.0_5.0_corner.png
+      :width: 50%
+      :align: center
+      :alt: Parameter PDF obtained with the true forward model
+
+
+Calibrate with a surrogate for ``alpha=5, beta=5``
+
+.. code-block:: console
+
+   python tut_calibration.py -useNN --alpha 5 --beta 5
+
+
+
+.. container:: figures-cal-Surr-5-5
+
+   .. figure:: ../assets/calibration/tutorial_normal_beta/Surr_opt_5.0_5.0_prop.png
+      :width: 50%
+      :align: center
+      :alt: Calibrated prediction with the surrogate forward model
+
+   .. figure:: ../assets/calibration/tutorial_normal_beta/Surr_opt_5.0_5.0_corner.png
+      :width: 50%
+      :align: center
+      :alt: Parameter PDF obtained with the surrogate forward model
+
+
+Calibrate without a surrogate for ``alpha=2, beta=5``
+
+
+.. code-block:: console
+
+   python tut_calibration.py --alpha 2 --beta 5
+
+
+.. container:: figures-cal-True-2-5
+
+   .. figure:: ../assets/calibration/tutorial_normal_beta/True_opt_2.0_5.0_prop.png
+      :width: 50%
+      :align: center
+      :alt: Calibrated prediction with the true forward model
+
+   .. figure:: ../assets/calibration/tutorial_normal_beta/True_opt_2.0_5.0_corner.png
+      :width: 50%
+      :align: center
+      :alt: Parameter PDF obtained with the true forward model
+
+
+Calibrate with a surrogate for ``alpha=2, beta=5``
+
+.. code-block:: console
+
+   python tut_calibration.py -useNN --alpha 2 --beta 5
+
+
+
+.. container:: figures-cal-True-2-5
+
+   .. figure:: ../assets/calibration/tutorial_normal_beta/Surr_opt_2.0_5.0_prop.png
+      :width: 50%
+      :align: center
+      :alt: Calibrated prediction with the surrogate forward model
+
+   .. figure:: ../assets/calibration/tutorial_normal_beta/Surr_opt_2.0_5.0_corner.png
+      :width: 50%
+      :align: center
+      :alt: Parameter PDF obtained with the surrogate forward model
+
+
+Clearly, the amount of missing physics vary depending on the observations.
+
+Using surrogate gives similar predictions as when not using a surrogate. But the surrogate was constructed with 200 forward simulations. In the case where forward simulations are expensive, the surrogate modeling approach is significantly faster.
+
+Calibration with calibrated likelihood uncertainty
+------------
+
+The same suite can be done by calibrating the likelihood uncertainty in lieu of optimizing it (with a bissection search). This has the advantage of rapid calibration since only one calibration is needed. Here the uncertainty minimizes the negative log likelihood.
+
+Calibrate without a surrogate for ``alpha=5, beta=5``
+
+
+.. code-block:: console
+
+   python tut_calibration.py --alpha 5 --beta 5 -cal_err
+
+
+.. container:: figures-cal-calerr-True-5-5
+
+   .. figure:: ../assets/calibration/tutorial_normal_beta/True_cal_5.0_5.0_prop.png
+      :width: 50%
+      :align: center
+      :alt: Calibrated prediction with the true forward model
+
+   .. figure:: ../assets/calibration/tutorial_normal_beta/True_cal_5.0_5.0_corner.png
+      :width: 50%
+      :align: center
+      :alt: Parameter PDF obtained with the true forward model
+
+
+Calibrate with a surrogate for ``alpha=5, beta=5``
+
+.. code-block:: console
+
+   python tut_calibration.py -useNN --alpha 5 --beta 5 -cal_err
+
+
+
+.. container:: figures-cal-calerr-Surr-5-5
+
+   .. figure:: ../assets/calibration/tutorial_normal_beta/Surr_cal_5.0_5.0_prop.png
+      :width: 50%
+      :align: center
+      :alt: Calibrated prediction with the surrogate forward model
+
+   .. figure:: ../assets/calibration/tutorial_normal_beta/Surr_cal_5.0_5.0_corner.png
+      :width: 50%
+      :align: center
+      :alt: Parameter PDF obtained with the surrogate forward model
+
+
+Calibrate without a surrogate for ``alpha=2, beta=5``
+
+
+.. code-block:: console
+
+   python tut_calibration.py --alpha 2 --beta 5 -cal_err
+
+
+.. container:: figures-cal-calerr-True-2-5
+
+   .. figure:: ../assets/calibration/tutorial_normal_beta/True_cal_2.0_5.0_prop.png
+      :width: 50%
+      :align: center
+      :alt: Calibrated prediction with the true forward model
+
+   .. figure:: ../assets/calibration/tutorial_normal_beta/True_cal_2.0_5.0_corner.png
+      :width: 50%
+      :align: center
+      :alt: Parameter PDF obtained with the true forward model
+
+
+Calibrate with a surrogate for ``alpha=2, beta=5``
+
+.. code-block:: console
+
+   python tut_calibration.py -useNN --alpha 2 --beta 5 -cal_err
+
+
+
+.. container:: figures-cal-calerr-True-2-5
+
+   .. figure:: ../assets/calibration/tutorial_normal_beta/Surr_cal_2.0_5.0_prop.png
+      :width: 50%
+      :align: center
+      :alt: Calibrated prediction with the surrogate forward model
+
+   .. figure:: ../assets/calibration/tutorial_normal_beta/Surr_cal_2.0_5.0_corner.png
+      :width: 50%
+      :align: center
+      :alt: Parameter PDF obtained with the surrogate forward model
+
+
+
+
+
+