Skip to content

Commit

Permalink
Improve logging and carbon tracking consistency (#371)
Browse files Browse the repository at this point in the history 
* Raise error if tracking not set up

* Add troubleshooting for carbon tracking

* Ensure track_carbon flag set correctly

* Pass tracking flag to sub-calculations

* Allow extra kwargs to be passed to carbon tracker

* Set attach_logger correctly

* Refactor setting log and tracker defaults
  • Loading branch information
@ElliottKasoar
ElliottKasoar authored Jan 10, 2025
1 parent bf53f87 commit 1d79227
Show file tree
Hide file tree
Showing 14 changed files with 254 additions and 83 deletions.
1 change: 1 addition & 0 deletions docs/source/user_guide/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ User guide

command_line
python
troubleshooting
56 changes: 56 additions & 0 deletions docs/source/user_guide/troubleshooting.rst
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
===============
Troubleshooting
===============

Carbon tracking
---------------

Enabling tracking (Python)
++++++++++++++++++++++++++

Carbon tracking can be enabled through the ``track_carbon`` option.
By default, this is ``True`` if logging is enabled, but requires setting ``attach_logger``, as this defaults to ``False``.

For example, to track the carbon emissions during a single point calculation:

.. code-block:: python
from janus_core.calculations.single_point import SinglePoint
sp = SinglePoint(
struct_path="tests/data/NaCl.cif",
attach_logger=True,
track_carbon=True,
)
This generates a log file, ``NaCl-singlepoint-log.yml``, which stores the emissions for the calculation.


In the case of multiple calculations, such as geometry optimisation triggered during molecular dynamics,
the emissions for each component of the calculation will be separate items in the log.


Disabling tracking (CLI)
++++++++++++++++++++++++

Currently, carbon tracking is enabled by default when using the command line interface,
saving the total calculating emissions to the generated summary file, as well as additional details and
per-calculation emissions to the log file.

This can be disabled by passing the ``--no-tracker`` flag to any command. For example:

.. code-block:: bash
janus singlepoint --struct tests/data/NaCl.cif --no-tracker
Sudo access
+++++++++++

On some systems, such as MacOS, the carbon tracker may prompt for your password, if you have sudo access.
To avoid this, you can:

1. Disable carbon tracking, as described in `Disabling tracking (CLI)`_.
3. Modify your sudoers file, as described `here <https://mlco2.github.io/codecarbon/methodology.html#cpu>`_, to provide sudo rights for all future calculations.
2. Provide your password. This may be saved for a period of time, but will need to be entered again in future.
4. Fail authentication, for example by entering an invalid or no password three times, which triggers the tracking to default to a constant power.
31 changes: 19 additions & 12 deletions janus_core/calculations/base.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
)
from janus_core.helpers.log import config_logger, config_tracker
from janus_core.helpers.struct_io import input_structs
from janus_core.helpers.utils import FileNameMixin, none_to_dict
from janus_core.helpers.utils import FileNameMixin, none_to_dict, set_log_tracker


class BaseCalculation(FileNameMixin):
Expand Down Expand Up @@ -47,12 +47,14 @@ class BaseCalculation(FileNameMixin):
Keyword arguments to pass to the selected calculator. Default is {}.
set_calc : bool | None
Whether to set (new) calculators for structures. Default is None.
attach_logger : bool
Whether to attach a logger. Default is False.
attach_logger : bool | None
Whether to attach a logger. Default is True if "filename" is passed in
log_kwargs, else False.
log_kwargs : dict[str, Any] | None
Keyword arguments to pass to `config_logger`. Default is {}.
track_carbon : bool
Whether to track carbon emissions of calculation. Default is True.
track_carbon : bool | None
Whether to track carbon emissions of calculation. Requires attach_logger.
Default is True if attach_logger is True, else False.
tracker_kwargs : dict[str, Any] | None
Keyword arguments to pass to `config_tracker`. Default is {}.
file_prefix : PathLike | None
Expand Down Expand Up @@ -83,9 +85,9 @@ def __init__(
sequence_allowed: bool = True,
calc_kwargs: dict[str, Any] | None = None,
set_calc: bool | None = None,
attach_logger: bool = False,
attach_logger: bool | None = None,
log_kwargs: dict[str, Any] | None = None,
track_carbon: bool = True,
track_carbon: bool | None = None,
tracker_kwargs: dict[str, Any] | None = None,
file_prefix: PathLike | None = None,
additional_prefix: str | None = None,
Expand Down Expand Up @@ -118,12 +120,14 @@ def __init__(
Keyword arguments to pass to the selected calculator. Default is {}.
set_calc : bool | None
Whether to set (new) calculators for structures. Default is None.
attach_logger : bool
Whether to attach a logger. Default is False.
attach_logger : bool | None
Whether to attach a logger. Default is True if "filename" is passed in
log_kwargs, else False.
log_kwargs : dict[str, Any] | None
Keyword arguments to pass to `config_logger`. Default is {}.
track_carbon : bool
Whether to track carbon emissions of calculation. Default is True.
track_carbon : bool | None
Whether to track carbon emissions of calculation. Requires attach_logger.
Default is True if attach_logger is True, else False.
tracker_kwargs : dict[str, Any] | None
Keyword arguments to pass to `config_tracker`. Default is {}.
file_prefix : PathLike | None
Expand All @@ -146,12 +150,15 @@ def __init__(
self.read_kwargs = read_kwargs
self.calc_kwargs = calc_kwargs
self.log_kwargs = log_kwargs
self.track_carbon = track_carbon
self.tracker_kwargs = tracker_kwargs

if not self.model_path and "model_path" in self.calc_kwargs:
raise ValueError("`model_path` must be passed explicitly")

attach_logger, self.track_carbon = set_log_tracker(
attach_logger, log_kwargs, track_carbon
)

# Read structures and/or attach calculators
# Note: logger not set up so yet so not passed here
self.struct = input_structs(
Expand Down
24 changes: 14 additions & 10 deletions janus_core/calculations/descriptors.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -47,12 +47,14 @@ class Descriptors(BaseCalculation):
Keyword arguments to pass to the selected calculator. Default is {}.
set_calc : bool | None
Whether to set (new) calculators for structures. Default is None.
attach_logger : bool
Whether to attach a logger. Default is False.
attach_logger : bool | None
Whether to attach a logger. Default is True if "filename" is passed in
log_kwargs, else False.
log_kwargs : dict[str, Any] | None
Keyword arguments to pass to `config_logger`. Default is {}.
track_carbon : bool
Whether to track carbon emissions of calculation. Default is True.
track_carbon : bool | None
Whether to track carbon emissions of calculation. Default is True if
attach_logger is True, else False.
tracker_kwargs : dict[str, Any] | None
Keyword arguments to pass to `config_tracker`. Default is {}.
invariants_only : bool
Expand Down Expand Up @@ -83,9 +85,9 @@ def __init__(
read_kwargs: ASEReadArgs | None = None,
calc_kwargs: dict[str, Any] | None = None,
set_calc: bool | None = None,
attach_logger: bool = False,
attach_logger: bool | None = None,
log_kwargs: dict[str, Any] | None = None,
track_carbon: bool = True,
track_carbon: bool | None = None,
tracker_kwargs: dict[str, Any] | None = None,
invariants_only: bool = True,
calc_per_element: bool = False,
Expand Down Expand Up @@ -117,12 +119,14 @@ def __init__(
Keyword arguments to pass to the selected calculator. Default is {}.
set_calc : bool | None
Whether to set (new) calculators for structures. Default is None.
attach_logger : bool
Whether to attach a logger. Default is False.
attach_logger : bool | None
Whether to attach a logger. Default is True if "filename" is passed in
log_kwargs, else False.
log_kwargs : dict[str, Any] | None
Keyword arguments to pass to `config_logger`. Default is {}.
track_carbon : bool
Whether to track carbon emissions of calculation. Default is True.
track_carbon : bool | None
Whether to track carbon emissions of calculation. Requires attach_logger.
Default is True if attach_logger is True, else False.
tracker_kwargs : dict[str, Any] | None
Keyword arguments to pass to `config_tracker`. Default is {}.
invariants_only : bool
Expand Down
25 changes: 15 additions & 10 deletions janus_core/calculations/eos.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,12 +50,14 @@ class EoS(BaseCalculation):
Keyword arguments to pass to the selected calculator. Default is {}.
set_calc : bool | None
Whether to set (new) calculators for structures. Default is None.
attach_logger : bool
Whether to attach a logger. Default is False.
attach_logger : bool | None
Whether to attach a logger. Default is True if "filename" is passed in
log_kwargs, else False.
log_kwargs : dict[str, Any] | None
Keyword arguments to pass to `config_logger`. Default is {}.
track_carbon : bool
Whether to track carbon emissions of calculation. Default is True.
track_carbon : bool | None
Whether to track carbon emissions of calculation. Default is True if
attach_logger is True, else False.
tracker_kwargs : dict[str, Any] | None
Keyword arguments to pass to `config_tracker`. Default is {}.
min_volume : float
Expand Down Expand Up @@ -115,9 +117,9 @@ def __init__(
read_kwargs: ASEReadArgs | None = None,
calc_kwargs: dict[str, Any] | None = None,
set_calc: bool | None = None,
attach_logger: bool = False,
attach_logger: bool | None = None,
log_kwargs: dict[str, Any] | None = None,
track_carbon: bool = True,
track_carbon: bool | None = None,
tracker_kwargs: dict[str, Any] | None = None,
min_volume: float = 0.95,
max_volume: float = 1.05,
Expand Down Expand Up @@ -157,12 +159,14 @@ def __init__(
Keyword arguments to pass to the selected calculator. Default is {}.
set_calc : bool | None
Whether to set (new) calculators for structures. Default is None.
attach_logger : bool
Whether to attach a logger. Default is False.
attach_logger : bool | None
Whether to attach a logger. Default is True if "filename" is passed in
log_kwargs, else False.
log_kwargs : dict[str, Any] | None
Keyword arguments to pass to `config_logger`. Default is {}.
track_carbon : bool
Whether to track carbon emissions of calculation. Default is True.
track_carbon : bool | None
Whether to track carbon emissions of calculation. Requires attach_logger.
Default is True if attach_logger is True, else False.
tracker_kwargs : dict[str, Any] | None
Keyword arguments to pass to `config_tracker`. Default is {}.
min_volume : float
Expand Down Expand Up @@ -263,6 +267,7 @@ def __init__(
"name": self.logger.name,
"filemode": "a",
}
self.minimize_kwargs["track_carbon"] = self.track_carbon

# Set output files
self.write_kwargs.setdefault("filename", None)
Expand Down
20 changes: 12 additions & 8 deletions janus_core/calculations/geom_opt.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -51,11 +51,13 @@ class GeomOpt(BaseCalculation):
set_calc : bool | None
Whether to set (new) calculators for structures. Default is None.
attach_logger : bool
Whether to attach a logger. Default is False.
Whether to attach a logger. Default is True if "filename" is passed in
log_kwargs, else False.
log_kwargs : dict[str, Any] | None
Keyword arguments to pass to `config_logger`. Default is {}.
track_carbon : bool
Whether to track carbon emissions of calculation. Default is True.
track_carbon : bool | None
Whether to track carbon emissions of calculation. Requires attach_logger.
Default is True if attach_logger is True, else False.
tracker_kwargs : dict[str, Any] | None
Keyword arguments to pass to `config_tracker`. Default is {}.
fmax : float
Expand Down Expand Up @@ -107,9 +109,9 @@ def __init__(
read_kwargs: ASEReadArgs | None = None,
calc_kwargs: dict[str, Any] | None = None,
set_calc: bool | None = None,
attach_logger: bool = False,
attach_logger: bool | None = None,
log_kwargs: dict[str, Any] | None = None,
track_carbon: bool = True,
track_carbon: bool | None = None,
tracker_kwargs: dict[str, Any] | None = None,
fmax: float = 0.1,
steps: int = 1000,
Expand Down Expand Up @@ -149,11 +151,13 @@ def __init__(
set_calc : bool | None
Whether to set (new) calculators for structures. Default is None.
attach_logger : bool
Whether to attach a logger. Default is False.
Whether to attach a logger. Default is True if "filename" is passed in
log_kwargs, else False.
log_kwargs : dict[str, Any] | None
Keyword arguments to pass to `config_logger`. Default is {}.
track_carbon : bool
Whether to track carbon emissions of calculation. Default is True.
track_carbon : bool | None
Whether to track carbon emissions of calculation. Requires attach_logger.
Default is True if attach_logger is True, else False.
tracker_kwargs : dict[str, Any] | None
Keyword arguments to pass to `config_tracker`. Default is {}.
fmax : float
Expand Down
25 changes: 15 additions & 10 deletions janus_core/calculations/md.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -70,12 +70,14 @@ class MolecularDynamics(BaseCalculation):
Keyword arguments to pass to the selected calculator. Default is {}.
set_calc : bool | None
Whether to set (new) calculators for structures. Default is None.
attach_logger : bool
Whether to attach a logger. Default is False.
attach_logger : bool | None
Whether to attach a logger. Default is True if "filename" is passed in
log_kwargs, else False.
log_kwargs : dict[str, Any] | None
Keyword arguments to pass to `config_logger`. Default is {}.
track_carbon : bool
Whether to track carbon emissions of calculation. Default is True.
track_carbon : bool | None
Whether to track carbon emissions of calculation. Requires attach_logger.
Default is True if attach_logger is True, else False.
tracker_kwargs : dict[str, Any] | None
Keyword arguments to pass to `config_tracker`. Default is {}.
struct : Atoms
Expand Down Expand Up @@ -185,9 +187,9 @@ def __init__(
read_kwargs: ASEReadArgs | None = None,
calc_kwargs: dict[str, Any] | None = None,
set_calc: bool | None = None,
attach_logger: bool = False,
attach_logger: bool | None = None,
log_kwargs: dict[str, Any] | None = None,
track_carbon: bool = True,
track_carbon: bool | None = None,
tracker_kwargs: dict[str, Any] | None = None,
ensemble: Ensembles | None = None,
steps: int = 0,
Expand Down Expand Up @@ -247,12 +249,14 @@ def __init__(
Keyword arguments to pass to the selected calculator. Default is {}.
set_calc : bool | None
Whether to set (new) calculators for structures. Default is None.
attach_logger : bool
Whether to attach a logger. Default is False.
attach_logger : bool | None
Whether to attach a logger. Default is True if "filename" is passed in
log_kwargs, else False.
log_kwargs : dict[str, Any] | None
Keyword arguments to pass to `config_logger`. Default is {}.
track_carbon : bool
Whether to track carbon emissions of calculation. Default is True.
track_carbon : bool | None
Whether to track carbon emissions of calculation. Requires attach_logger.
Default is True if attach_logger is True, else False.
tracker_kwargs : dict[str, Any] | None
Keyword arguments to pass to `config_tracker`. Default is {}.
ensemble : Ensembles
Expand Down Expand Up @@ -497,6 +501,7 @@ def __init__(
"name": self.logger.name,
"filemode": "a",
}
self.minimize_kwargs["track_carbon"] = self.track_carbon

self.dyn: Langevin | VelocityVerlet | ASE_NPT
self.n_atoms = len(self.struct)
Expand Down
Loading

0 comments on commit 1d79227

Please sign in to comment.