-
Notifications
You must be signed in to change notification settings - Fork 51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introducing gulmc
: full monte carlo loss engine
#1137
Conversation
9d2dd87
to
d525acc
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
-
Nice to have: It would be useful if the gulmc testing could output the failed differences between expected and results. At the moment the output notes that files differ (true/false). However, the binary format needs to be converted to something human readable first.
-
Note: gulmc testing has failures when running on a non-ubuntu based distro (arch linux based). We suspect it might be linked to OS libraries, but haven't found the cause.
Marco Tazzari
if we can get tests running on other OSs then we can see what we can support, which would be even better, but being so focused on numerical performance, it's fair to pick a specific OS and rely on its reproducibility
rather than aiming at a Windows-style "we work on any computer", which it's going to be hard to maintain
Stephane Struzik
I don't agree on this one, I think the test should pass on all platform we advertise. If we just have rounding error I would prefer that we convert the result to csv and use dataframe to compare. It also has the added benefit of providing a better diff message
Thank you @sambles for the comments. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just few minor comments. The logic looks good.
Codecov Report
@@ Coverage Diff @@
## develop #1137 +/- ##
===========================================
+ Coverage 43.18% 45.63% +2.44%
===========================================
Files 90 93 +3
Lines 11175 11584 +409
===========================================
+ Hits 4826 5286 +460
+ Misses 6349 6298 -51
Continue to review full report at Codecov.
|
* [mcgul] first implementation of full MC gul * [modelpy] montecarlo implementation in modelpy * stop tracking mcgul * [modelpy] fixes * simplify algorithm * remove unused imports * use numba, process last areaperil id, cleanup * [modelpy] add docstrings * [modelpy] function namechange * [modelpy] add TODOs not to forget * [gulpy] bugfix in calling read_getmodel_data * [gulpy] drafting monte carlo implementation * [mcgul] Add major modelpy and gulpy rewrite as one tool * [mcgul] do not sample haz if no haz uncertainty * [mcgul] cleanup * [mcgul] good working implementation * [mcgul] perfectly reproduces effective damageability * [mcgul] further simplification * [mcgul] wip * [mcgul] compute haz cdf in map_areaperil_ids_in_footprint * [gulmc] update cli * [getmodel] reverting full mc modifications * [gul] reverting mc modifications * [getmodel] reverting mc modifications * [getmodel] Reverting unused mc modifications * [gul] updating docstring * [getmode;] update docstring * [gulmc] dynamic buff_size * [gulmc] imports cleanup * [gulmc] cleanup * [gulmc] dynamic buff size * [gulmc] compute effective damageability * [gulmc] effective damageability with numba * [gulpy] minor bugfix * [gulmc] bugfix: use 4 as item size in int32_mv * [gulmc] minor cleanup * [gulmc] fix conflicts with stashed edits * [gulmc] cleanup * [gulmc] remove unused imports * [modelpy] remove one blank line * [gulmc] add effective_damageability optional arg * [gulmc] bugfix effective damageability * [gulmc] add tests * [gulmc] add tests for effective damageability * [gulmc] move gulpy tests to separate module * [tests] add test_model_1 to the tests assets * [tests] use typing.Tuple for type hints * [gulmc] better tests, tiv set to float64 * [gulmc] log info about effective_damageabilty * [gulmc] cleaning up, adding docs (WIP). * [gulmc] adding documentation and docstrings * [gulmc] bugfix * [gulmc] adding docs * [gulmc] add docs * [gulmc] rewrite complex outputs as tuples * [gulmc] add final docs * [gulmc] remove unused import * [gulmc] Improve --debug flag * [gulmc] raise ValueError if alloc_rule>0 when debug is 1 or 2 * [gulmc] cleanup * [flake8] fix error code in ignore config * [requirements] testing unpinning virtualenv * [requirements] testing unpinning virtualenv * [requirements] fixing package clash * [gulmc] test ValueError if alloc_rule is invalid * [gulmc] improve tests * [gulmc] remove unnecessary binary files * [requirements] removing unnecessary virtualenv * [CI] specify pip-compile resolver for py 3.7 * [CI] fix bug in CI * [CI] bugfix * [gulmc] update following review comments * [gulmc] implement fixes following review * [gulmc] bugfix in logging
This PR introduces
gulmc
, a new tool that uses a "full Monte Carlo" approach for ground up losses calculation that, instead of drawing loss samples from the 'effective damageability' probability distribution (as done by callingeve | modelpy | gulpy
), it first draws a sample of the hazard intensity, and then draws a sample of the damage from the vulnerability function corresponding to the hazard intensity sample.Introducing
gulmc
, full Monte Carlo loss calculation engineThis PR introduces
gulmc
, a new tool that uses a "full Monte Carlo" approach for ground up losses calculation that, instead of drawing loss samples from the 'effective damageability' probability distribution (as done by callingeve | modelpy | gulpy
), it first draws a sample of the hazard intensity, and then draws a sample of the damage from the vulnerability function corresponding to the hazard intensity sample.Comparing
gulpy
andgulmc
outputgulmc
runs the same algorithm ofeve | modelpy | gulpy
, i.e., it runs the 'effective damageability' calculation mode, with the same command line arguments. For example, to run a model with 1000 samples, alloc rule 1, and streaming the binary output to theoutput.bin
file, can be done with:or
eve 1 1 | gulmc -S1000 -a1 -o output.bin
Hazard uncertainty treatment
If the hazard intensity in the fooprint has no uncertainty, i.e.:
then
gulpy
andgulmc
produce the same outputs. However, if the hazard intensity has a probability distribution, e.g.:then, by default,
gulmc
runs the full Monte Carlo sampling of the hazard intensity, and then of damage. In order to reproduce the same results thatgulpy
produces can be achieved by using the--effective-damageability
flag:eve 1 1 | gulmc -S1000 -a1 -o output.bin --effective-damageability
On the usage of
modelpy
andeve
withgulmc
Due to internal refactoring,
gulmc
now incorporates the functionality performed bymodelpy
, thereforemodelpy
should not be used in a pipe withgulmc
:Printing the random values used for sampling
Since we now sample in two dimensions (hazard intensity and damage), the
-d
flag is revamped to output both random values used for sampling. Whilegulpy -d
printed the random values used to sample the effective damageability distribution, ingulmc
:Testing suite
This PR introduces:
tests/assets/test_model_1/
that can be used to run unit tests on various functionality in the repository. A more detailed description of the content of the model can be found attests/assets/test_model_1/README.md
.gulmc
output for combinations of input parameters (alloc rule, correlation, etc.). Binary files with the expected outputs are stored attests/assets/test_model_1/expected/
.gulpy
output for combinations of input parameters.ValueErrors
ingulmc
are raised when expected.