What's Changed

• Fix small issues by @stefsmeets in #727
• Fix sidebar logo display by @stefsmeets in #729
• Update dashboard logo to use new api by @stefsmeets in #730
• explicitly dependencies in tests by @v1kko in #731

Full Changelog: 3.1.0...3.1.1

This release of duqtools overhauls the structure of the documentation, making it easier to find things, and now supports HDF5 via the IMAS access layer.

What's Changed

• Support hdf5 via acccess layer by @v1kko in #712
• Add quickstart documentation for the gateway by @stefsmeets in #705
• Update dissemination page by @stefsmeets in #710
• Revise documentation by @stefsmeets in #714

Under the hood

• Specify lookup.json via environment variable by @stefsmeets in #711
• Simplify systems by @v1kko in #703

Bug fixes

• Fix cli version crash for pypi installs by @stefsmeets in #704
• Fix error to be relative by @v1kko in #697
• Fix IMAS tests by @stefsmeets in #718

Maintanance

• Remove ETS6 and Jetto-v210921 systems by @stefsmeets in #717

Full Changelog: 3.0.3...3.1.0

What's Changed

• Fix kwmain lookup crash by @stefsmeets in #702

Full Changelog: 3.0.2...3.0.3

This is a release to fix the publishing pipeline

Full Changelog: 3.0.1...3.0.2

This PR fixes a few small issues with duqtools.

What's Changed

• added jetto vector support by @v1kko in #688
• General bugfixes from application testing by @aaronkho in #689
• Update notebooks by @stefsmeets in #691
• Fix crash in dashboard because logo was missing from package by @stefsmeets in #695

Full Changelog: 3.0.0...3.0.1

This release of duqtools splits off the python API into its own library. This is a major change to the internals of duqtools that should not affect normal use of duqtools through the CLI. imas2xarray now handles the variable mappings to the IMAS data structure, and includes an api to load these data into xarray from hdf5. You can check it out here.

What's Changed

• Add compute_from to variable operations, and other improvements by @v1kko in #675
• Add option to customize axes in dashboard by @stefsmeets in #677
• Split off imas2xarray by @stefsmeets in #682

Under the hood

• Prepare imas2xarray submodule by @stefsmeets in #680
• Single source vars by @stefsmeets in #684

Bug fixes

• Fix imas tests by @stefsmeets in #676
• Dimensions Fix: recognize that some numpy ufuncs do not have out as a keyword by @v1kko in #678

Maintanance

• Switch to bump-my-version by @stefsmeets in #686
• Directly generate coverage badge svg by @stefsmeets in #687

Full Changelog: 2.0.0...3.0.0

This version of duqtools is all about future proofing:

• It now depends on pydantic v2 and altair v5, both major dependencies requiring substantial changes to duqtools.
• The configuration and system modules were also refactored, making it easier to modify or add new systems.
• Test coverage improved to 72%, making the duqtools more robust

What's Changed

• Add option to create array submissions script and limit total number of jobs to CLI by @stefsmeets in #644
• Adding a sync-prominence command by @v1kko in #648
• Add option to run duqtools without a system by @stefsmeets in #659
• Update pattern to allow parsing localdb strings by @stefsmeets in #666

Under the hood

• Update code to support pydantic V2 by @stefsmeets in #653
• Update code to be compatible with Pydantic V2 by @stefsmeets in #654
• Update code for API change in altair v5 by @stefsmeets in #656
• Refactor systems by @stefsmeets in #660
• Refactor systems and schemas by @stefsmeets in #661

Bug fixes

• Fix bug and add tests for reading data.csv by @stefsmeets in #647
• Fix compatibility with pydantic_yaml >= 1.0.0 by @stefsmeets in #652
• Skip loading variable on EmptyVarError in dashboard by @stefsmeets in #658

Maintanance

• Add Pypi trusted publishing by @v1kko in #649
• Update workflow to generate requirements.txt by @stefsmeets in #657
• Improve test coverage by @stefsmeets in #664
• Improve test coverage by @stefsmeets in #662
• Add test for duqtools merge by @stefsmeets in #667
• Add tests for status, setup and submit by @stefsmeets in #668

Full Changelog: 1.9.0...1.10.0

The highlight for this release is preliminary support for the ETS6 system.
The operators are now more flexible, with support for upper and lower bounds, gradients, and custom functions if that is still not enough.

Note that if you are updating to this version, systems are now defined in their own block in the config:

system:
  name: jetto
  ...

See the documentation for more info.

What's Changed

• Adding ets6 System for use with duqtools by @v1kko in #557
• Add ceiling and floor option to operator by @stefsmeets in #630
• Add slope to operator by @stefsmeets in #632
• Apply custom operator to data by @stefsmeets in #638

Under the hood

• Refactoring the Systems, also fixing ETS6 by @v1kko in #623
• Add naive copy instead of complex copy of ids data by @v1kko in #613
• Revise testing strategy and update python api by @stefsmeets in #610
• Move jruns keyword to system config by @stefsmeets in #634

Documentation

• Add banner to readme by @stefsmeets in #615
• Update logo and add it to dashboard by @stefsmeets in #618
• Add base / uq run example to docs by @stefsmeets in #629

Bug fixes

• Update default for max_jobs in submit cli by @stefsmeets in #642
• Fix missing config error for submit by @stefsmeets in #641
• Get prominence working again by @v1kko in #635
• Fix crashes in submit / status by @stefsmeets in #627

Maintanance

• Move setup config to pyproject.toml by @stefsmeets in #608
• Avoid pip install in workflow on cache hit by @stefsmeets in #611
• Avoid installing typing-extensions==4.6.0 which crashes duqtools on Python 3.9 by @stefsmeets in #617
• Update IMAS docker workflow to only run once a week by @stefsmeets in #622

Full Changelog: 1.8.0...1.9.0

This release focuses on expanding the use-cases for duqtools. We have added a way to directly specify which operations to always apply to a dataset (operations keyword in duqtools.yaml). This will make it easier to separate 'fixes' to a dataset from sampling variables or for generating base runs.

This release also introduces a Python API, so that runs can be created and submitted from a Python script or Jupyter Notebook. For more information, see the documentation.

The duqduq CLI is also more flexible to specify which runs to create and submit, which will give more control over your data for large scale validation.

What's Changed

• Add path and imas filters for duqduq create and submit by @stefsmeets in #593
• Update duqtools init template by @stefsmeets in #594
• Add operations keyword to create config by @stefsmeets in #598
• Add workflow for generating base and uq runs from a single template config by @stefsmeets in #603
• Add python api for duqtools create and submit by @stefsmeets in #606

Under the hood

• Revise config loading by @stefsmeets in #604

Documentation

• Update contributing guidelines by @stefsmeets in #592
• Add documentation for setup by @stefsmeets in #595
• Add quickstart example to cli by @stefsmeets in #596
• Add option to create base run only by @stefsmeets in #600
• Add docs for top level keywords in config by @stefsmeets in #602

Bug fixes

• Remove print statement in operations.py by @v1kko in #589
• Do not copy restart files if restart toggle is off by @stefsmeets in #599
• Fixes missing keyword argument in duqduq submit -array by @stefsmeets in #601
• Make dashboard merge compatible with local imas database by @stefsmeets in #607

Maintanance

• Add linting and autoformatting for notebooks by @stefsmeets in #591

Full Changelog: 1.7.0...1.8.0

What's Changed

• Add duqmap function to map functions over runs/handles by @v1kko in #585
• Implement --max_array_size (default 100) option in duqtools submit by @v1kko in #584

Bug fixes

• Fixed publish pipeline by @v1kko in #582
• Fix bug in submit.py by @v1kko in #579

Maintanance

• Add action for building imas on the CI by @stefsmeets in #586

Full Changelog: 1.6.4...1.7.0