README. Modeling spatiotemporal domestic wastewater variability: Implications to measure treatment efficiency
Néstor DelaPaz-Ruíz, Ellen-Wien Augustijn, Mahdi Farnaghi, Raul Zurita-Milla. February, 2023
Affiliation: Department of Geo-Information Processing (GIP), Faculty of Geo-Information Science and Earth Observation (ITC), University of Twente, Drienerlolaan 5, 7522 NB Enschede, The Netherlands
Journal article: https://doi.org/10.1016/j.jenvman.2023.119680
This repository provides access to the materials (code-data), software-environments (NetLogo, Rstudio, Docker containers), and the steps for reproducing the results from the publication: Modeling spatiotemporal domestic wastewater variability: Implications to measure sanitation efficiency.
The information below guides you to execute two NetLogo simulations. One represents a non-calibrated model, and the other is a calibrated and evaluated model. The main outputs are the figures and tables representing the results assessment of the mentioned publication.
Note: If you found this repository useful and would like to support reproducibility and open science, I would appreciate a star for this repository. You can also send an email if you could replicate the results. Your reaction is relevant for monitoring and maintaining this material.
You will need the following:
- Data:
- Download or copy this repository to the local folder where you plan to execute the code scripts.
- Open-source software:
- You have Docker Desktop installed and running in your computer (v.4.16.3 is recommended).See: https://www.docker.com/products/docker-desktop/
- A command-line shell: Git Bash is used for running the commands. See: https://gitforwindows.org/
-
Read the full section of
Usage
before executing code. -
NetLogo alternative:
- Before building and running NetLogo images and containers in Docker Desktop consider that you can run the NetLogo models once you install NetLogo 6.1.1. Running NetLogo in Docker Desktop can take a couple of hours. Link to NetLogo 6.1.1 Downloads. Running NetLogo with your own NetLogo 6.1.1 installation can take less than 1 hour if your computer has multiple processors and good RAM. For this option copy the provided
NetLogo.cfg
file and replace it at the folder:C:\Program Files\NetLogo 6.1.1\app
. In this way, you are free to use several processors. Please, refer to the following link: FAQ: How big can my model be? How many turtles, patches, procedures, buttons, and so on can my model contain?. After the NetLogo installation, make sure to execute the. ./code/newfiles.txt
in Git Bash and then you can jump to step 3) in the bellow Usage section and run the required 2 simulations.
-
Open Git Bash in the folder path of the repository.
-
Execute the following command in Git Bash to set up new files to run NetLogo:
. ./code/newfiles.txt
-
Open Docker Desktop and verify the engine is running (green icon at the left-bottom corner).
-
Execute the below commands in Git Bash to create NetLogo Docker images. Note that the two models must be created and run:
- Model not calibrated, and model calibrated:
docker build -f DockerfileNetlogoNocalibration --build-arg MODEL_NAME=dw.sms.abm.snt.2020.no.cal.val.1.nlogo --build-arg NETLOGO_VERSION=6.1.1 -t dwsmsabmsntnocalibration .
docker build -f DockerfileNetlogo --build-arg MODEL_NAME=dw.sms.abm.snt.2020.cal.val.1.nlogo --build-arg NETLOGO_VERSION=6.1.1 -t dwsmsabmsntresults .
- Verify that the following images were created. Go to Docker Desktop/images:
dwsmsabmsntnocalibration
&dwsmsabmsntresults
.
- Execute the below command in Git Bash to run the XPRA X11 tool.
docker run -d --name x11-bridge -e MODE="tcp" -e XPRA_HTML="yes" -e DISPLAY=:14 -e XPRA_PASSWORD=111 -p 10000:10000 jare/x11-bridge
-
Go to Docker Desktop/containers to verify that the container
x11-bridge
is running. -
Run the calibrated and no calibrated models with the following steps:
Note that in the two commands bellow (1,7) you have to replace the 'my/directory/path/to/results'
to your working directory. For example: 'my/directory/path/to/results'
= C:/Mydocker.dwvariability/results
.
- Execute the below command in Git Bash to run the container
dwsmsabmsntnocalibration
.
docker run -it -d -m 20024M -d --name dwnetlogonocalibration --volumes-from x11-bridge -v 'my/directory/path/to/results':/home/results dwsmsabmsntnocalibration
- Once the Docker container is running, open your web browser (e.g.B Chrome, Firefox) and paste the following URL in the search bar: http://localhost:10000/index.html?encoding=rgb32&password=111
You should see the NetLogo model.
-
Press
ctrl+shift+B
to open the Behavior space menu and you should see the experiment:cal.val.1 (50 runs)
. -
Press
Run
. InSimultaneous runs in parallel
type 2 or 3 processors and deselect allRun options
.
Note: If you get the message running out of memory, go to containers in Docker Desktop. Stop and start again the container named dwsmsabmsntnocalibration
to run again NetLogo and try 1 processor.
-
Make sure that the NetLogo window
Running Experiment:
with a timer appears. Move the bar ofnormal speed
to the right until showing fast. Processing time depends on the number of processors and computer features. For example, with 3 processors the simulation should take around 1:30 hrs. -
In Docker Desktop go to containers. Stop the
dwsmsabmsntnocalibration
container. -
To execute the following model container:
dwsmsabmsntresults
go to Git Bash and run the command:
docker run -it --cpus=12 -d -m 20024M -d --name dwnetlogoresults --volumes-from x11-bridge -v 'my/directory/path/to/results':/home/results dwsmsabmsntresults
- For the
dwsmsabmsntresults
container, repeat steps from 2 to 6.
You can refer to the following tutorial which shows a video to run NetLogo images in Docker:
https://forum.comses.net/t/containerizing-a-netlogo-model-gui-version/8236
- In Git Bash execute the following command to create the Rstudio image in Docker Desktop.
docker build -f DockerfileRstudioDWresults -t dwsmsabmsnt2020rproj .
- Verify that the following image was created. Go to Docker Desktop/images:
dwsmsabmsnt2020rproj
.
- Execute the below command in Git Bash to run RStudio. Do not forget to replace your own working directory. For example:
'my/directory/path/to/results'
=C:/Mydocker.dwvariability/results
docker run -d --name dwrstudio -p 8787:8787 -e PASSWORD=mypassword -v 'my/directory/path/to/results':/home/rstudio/results dwsmsabmsnt2020rproj
- Go to Docker Desktop/containers to verify that the container
dwrstudio
is running. Open the following link in a web browser:
-
Type the following credentials:
username:
rstudio
mypassword:
mypassword -
In the RStudio user interface, go to files and open the
DW_ABM_before_after_calibration_and_validation.Rmd
file. The file content will appear. Press theKnit
icon. -
The viewer panel shows the results.
Download the repository and open the file: DW_ABM_before_after_calibration_and_validation.html
to see expected results after pressing Knit
in RStudio.
This section indicates the steps for a quick reproduction without Docker and half of the simulation and processing time. Experience in NetLogo and R is expected.
- In a new folder, execute the following commands in git bash:
git clone https://github.com/delapazruiz/domestic.wastewater.variability.git
. ./code/newfiles.txt
- Open the NetLogo(6.1.1) files and run the experiments. In Tools/BehaviorSpace, change to 25 runs and select 1 ro 2 processors to run each experiment. (time: around 25 min each)
dw.sms.abm.snt.2020.cal.val.1.nlogo
dw.sms.abm.snt.2020.no.cal.val.1.nlogo
- Open Rstudio and knit the following file: (time: around 25 min)
DW_ABM_before_after_calibration_and_validation.Rmd
This repository is expected to be in continuous improvement. For major changes, please open an issue first to discuss what you would like to improve.
This project is licensed under the MIT license. Feel free to edit and distribute this template as you like. See LICENSE for more information.
The authors wish to express their gratitude for the valuable support in the local implementation of this study, without which this research cannot be consolidated: Carlos Pailles, Ana Velasco, LucC-a Guardamino, Rodrigo Tapia-McClung, Araceli ChC!vez, Diana Ramos, Daniela GomC)z, JosC) Luis PC)rez, Natalia Volkow, and the anonymous facilitators from Mexico City, citizens of Tepeji del RC-o, and the INEGI department of microdata. Scholarship funder: CONACYT-Alianza FiiDEM.