Re-run notebooks that don't need human attention (pymc-devs#266)

* Re-run blackbox_external_likelihood_numpy.ipynb notebook * Re-run hierarchical_partial_pooling.ipynb notebook * Don't run pre-commit more often than needed * Re-run item_response_nba.ipynb notebook * Re-run GLM-binomial-regression.ipynb notebook * Re-run GLM-hierarchical.ipynb notebook * Re-run GLM-truncated-censored-regression.ipynb notebook * Re-run sampling_compound_step.ipynb notebook * Re-run MLDA_introduction.ipynb notebook * Re-run MLDA_simple_linear_regression.ipynb notebook
martiningram · Jan 10, 2022 · d3f8f0c · d3f8f0c
1 parent e927d9e
commit d3f8f0c
Show file tree

Hide file tree

Showing 10 changed files with 1,898 additions and 968 deletions.
diff --git a/examples/case_studies/blackbox_external_likelihood_numpy.ipynb b/examples/case_studies/blackbox_external_likelihood_numpy.ipynb
diff --git a/examples/case_studies/hierarchical_partial_pooling.ipynb b/examples/case_studies/hierarchical_partial_pooling.ipynb
diff --git a/examples/case_studies/item_response_nba.ipynb b/examples/case_studies/item_response_nba.ipynb
diff --git a/examples/generalized_linear_models/GLM-binomial-regression.ipynb b/examples/generalized_linear_models/GLM-binomial-regression.ipynb
diff --git a/examples/generalized_linear_models/GLM-hierarchical.ipynb b/examples/generalized_linear_models/GLM-hierarchical.ipynb
diff --git a/examples/generalized_linear_models/GLM-truncated-censored-regression.ipynb b/examples/generalized_linear_models/GLM-truncated-censored-regression.ipynb
diff --git a/examples/pymc3_howto/sampling_compound_step.ipynb b/examples/pymc3_howto/sampling_compound_step.ipynb
diff --git a/examples/samplers/MLDA_introduction.ipynb b/examples/samplers/MLDA_introduction.ipynb
@@ -6,7 +6,7 @@
    "source": [
     "# MLDA sampler: Introduction and resources\n",
     "\n",
-    "This notebook contains an introduction to the Multi-Level Delayed Acceptance MCMC algorithm (MLDA) proposed in [1]. It explains the main idea behind the method, gives an overview of the problems it is good for and points to specific notebooks with examples of how to use it within PyMC3. \n",
+    "This notebook contains an introduction to the Multi-Level Delayed Acceptance MCMC algorithm (MLDA) proposed in [1]. It explains the main idea behind the method, gives an overview of the problems it is good for and points to specific notebooks with examples of how to use it within PyMC. \n",
     "\n",
     "[1] Dodwell, Tim & Ketelsen, Chris & Scheichl, Robert & Teckentrup, Aretha. (2019). Multilevel Markov Chain Monte Carlo. SIAM Review. 61. 509-545. https://doi.org/10.1137/19M126966X"
    ]
@@ -41,15 +41,15 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### PyMC3 implementation\n",
+    "### PyMC implementation\n",
     "\n",
-    "MLDA is one of the MCMC inference methods available in PyMC3. You can instantiate an MLDA sampler using the `pm.MLDA(coarse_models=...)`, where you need to pass at least one coarse model within a list.\n",
+    "MLDA is one of the MCMC inference methods available in PyMC. You can instantiate an MLDA sampler using the `pm.MLDA(coarse_models=...)`, where you need to pass at least one coarse model within a list.\n",
     "\n",
-    "The PyMC3 implementation of MLDA supports any number of levels, tuning parameterization for the bottom-level sampler, separate subsampling rates for each level, choice between blocked and compound sampling for the bottom-level sampler, two types of bottom-level samplers (Metropolis, DEMetropolisZ), adaptive error correction and variance reduction.\n",
+    "The PyMC implementation of MLDA supports any number of levels, tuning parameterization for the bottom-level sampler, separate subsampling rates for each level, choice between blocked and compound sampling for the bottom-level sampler, two types of bottom-level samplers (Metropolis, DEMetropolisZ), adaptive error correction and variance reduction.\n",
     "\n",
     "For more details about the MLDA sampler and the way it should be used and parameterised, the user can refer to the docstrings in the code and to the other example notebooks (links below) which deal with more complex problem settings and more advanced MLDA features.\n",
     "\n",
-    "Please note that the MLDA sampler is new in PyMC3. The user should be extra critical about the results and report any problems as issues in the PyMC3's github repository."
+    "Please note that the MLDA sampler is new in PyMC. The user should be extra critical about the results and report any problems as issues in the PyMC's github repository."
    ]
   },
   {
@@ -59,31 +59,41 @@
     "### Notebooks with example code\n",
     "\n",
     "\n",
-    "[Simple linear regression](./MLDA_simple_linear_regression.ipynb): This notebook demonstrates the workflow for using MLDA within PyMC3. It employes a very simple toy model.\n",
+    "[Simple linear regression](./MLDA_simple_linear_regression.ipynb): This notebook demonstrates the workflow for using MLDA within PyMC. It employes a very simple toy model.\n",
     "\n",
-    "[Gravity surveying](./MLDA_gravity_surveying.ipynb): In this notebook, we use MLDA to solve a 2-dimensional gravity surveying inverse problem. Evaluating the likelihood requires solving a PDE, which we do using [scipy](https://www.scipy.org/). We also compare the performance of MLDA with other PyMC3 samplers (Metropolis, DEMetropolisZ).\n",
+    "[Gravity surveying](./MLDA_gravity_surveying.ipynb): In this notebook, we use MLDA to solve a 2-dimensional gravity surveying inverse problem. Evaluating the likelihood requires solving a PDE, which we do using [scipy](https://www.scipy.org/). We also compare the performance of MLDA with other PyMC samplers (Metropolis, DEMetropolisZ).\n",
     "\n",
-    "[Variance reduction 1](./MLDA_variance_reduction_linear_regression.ipynb) and [Variance reduction 2](https://github.com/alan-turing-institute/pymc3/blob/mlda_all_notebooks/docs/source/notebooks/MLDA_variance_reduction_groundwater.ipynb) (external link): Those two notebooks demonstrate the variance reduction feature in a linear regression model and a groundwater flow model. This feature allows the user to define a quantity of interest that they need to estimate using the MCMC samples. It then collects those quantities of interest, as well as differences of these quantities between levels, during MLDA sampling. The collected quentities can then be used to produce an estimate which has lower variance than a standard estimate that uses samples from the fine chain only. The first notebook does not have external dependencies, while the second one requires FEniCS. Note that the second notebook is outside the core PyMC3 repository because FEniCS is not a PyMC3 dependency.\n",
+    "[Variance reduction 1](./MLDA_variance_reduction_linear_regression.ipynb) and [Variance reduction 2](https://github.com/alan-turing-institute/pymc/blob/mlda_all_notebooks/docs/source/notebooks/MLDA_variance_reduction_groundwater.ipynb) (external link): Those two notebooks demonstrate the variance reduction feature in a linear regression model and a groundwater flow model. This feature allows the user to define a quantity of interest that they need to estimate using the MCMC samples. It then collects those quantities of interest, as well as differences of these quantities between levels, during MLDA sampling. The collected quentities can then be used to produce an estimate which has lower variance than a standard estimate that uses samples from the fine chain only. The first notebook does not have external dependencies, while the second one requires FEniCS. Note that the second notebook is outside the core PyMC repository because FEniCS is not a PyMC dependency.\n",
     "\n",
-    "[Adaptive error model](https://github.com/alan-turing-institute/pymc3/blob/mlda_all_notebooks/docs/source/notebooks/MLDA_adaptive_error_model.ipynb) (external link): In this notebook we use MLDA to tackle another inverse problem; groundwarer flow modeling. The aim is to infer the posterior distribution of model parameters (hydraulic conductivity) given data (measurements of hydraulic head). In this example we make use of Theano Ops in order to define a \"black box\" likelihood, i.e. a likelihood that uses external code. Specifically, our likelihood uses the [FEniCS](https://fenicsproject.org/) library to solve a PDE. This is a common scenario, as PDEs of this type are slow to solve with scipy or other standard libraries. Note that this notebook is outside the core PyMC3 repository because FEniCS is not a PyMC3 dependency. We employ the adaptive error model (AEM) feature and compare the performance of basic MLDA with AEM-enhanced MLDA. The idea of Adaptive Error Model (AEM) is to estimate the mean and variance of the forward-model error between adjacent levels, i.e. estimate the bias of the coarse forward model compared to the fine forward model, and use those estimates to correct the coarse model. Using the technique should improve ESS/sec on the fine level.\n",
+    "[Adaptive error model](https://github.com/alan-turing-institute/pymc/blob/mlda_all_notebooks/docs/source/notebooks/MLDA_adaptive_error_model.ipynb) (external link): In this notebook we use MLDA to tackle another inverse problem; groundwarer flow modeling. The aim is to infer the posterior distribution of model parameters (hydraulic conductivity) given data (measurements of hydraulic head). In this example we make use of Aesara Ops in order to define a \"black box\" likelihood, i.e. a likelihood that uses external code. Specifically, our likelihood uses the [FEniCS](https://fenicsproject.org/) library to solve a PDE. This is a common scenario, as PDEs of this type are slow to solve with scipy or other standard libraries. Note that this notebook is outside the core PyMC repository because FEniCS is not a PyMC dependency. We employ the adaptive error model (AEM) feature and compare the performance of basic MLDA with AEM-enhanced MLDA. The idea of Adaptive Error Model (AEM) is to estimate the mean and variance of the forward-model error between adjacent levels, i.e. estimate the bias of the coarse forward model compared to the fine forward model, and use those estimates to correct the coarse model. Using the technique should improve ESS/sec on the fine level.\n",
     "\n",
-    "[Benchmarks and tuning](https://github.com/alan-turing-institute/pymc3/blob/mlda_all_notebooks/docs/source/notebooks/MLDA_benchmarks_tuning.ipynb) (external link): In this notebook we benchmark MLDA against other samplers using different parameterizations of the groundwater flow model. We also give some advice on tuning MLDA. Note that this notebook is outside the core PyMC3 repository because FEniCS is not a PyMC3 dependency."
+    "[Benchmarks and tuning](https://github.com/alan-turing-institute/pymc/blob/mlda_all_notebooks/docs/source/notebooks/MLDA_benchmarks_tuning.ipynb) (external link): In this notebook we benchmark MLDA against other samplers using different parameterizations of the groundwater flow model. We also give some advice on tuning MLDA. Note that this notebook is outside the core PyMC repository because FEniCS is not a PyMC dependency."
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 1,
-   "metadata": {},
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2022-01-09T20:50:25.609487Z",
+     "iopub.status.busy": "2022-01-09T20:50:25.607489Z",
+     "iopub.status.idle": "2022-01-09T20:50:25.638235Z",
+     "shell.execute_reply": "2022-01-09T20:50:25.639236Z"
+    }
+   },
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "last updated: Sat Oct 10 2020 \n",
+      "Last updated: Sun Jan 09 2022\n",
+      "\n",
+      "Python implementation: CPython\n",
+      "Python version       : 3.8.10\n",
+      "IPython version      : 7.30.1\n",
       "\n",
-      "CPython 3.6.9\n",
-      "IPython 7.16.1\n",
-      "watermark 2.0.2\n"
+      "Watermark: 2.3.0\n",
+      "\n"
      ]
     }
    ],
@@ -109,7 +119,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.5"
+   "version": "3.8.10"
   }
  },
  "nbformat": 4,

diff --git a/examples/samplers/MLDA_simple_linear_regression.ipynb b/examples/samplers/MLDA_simple_linear_regression.ipynb
diff --git a/scripts/rerun.py b/scripts/rerun.py
@@ -69,6 +69,7 @@ def run_precommit(fp: pathlib.Path, attempts: int = 2):
             _log.info("⏳ Running pre-commit attempt %i on %s", a, fp)
             try:
                 subprocess.check_call(["pre-commit", "run", "--files", str(fp)])
+                break
             except subprocess.CalledProcessError:
                 if a == attempts - 1:
                     raise