Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add asv benchmark jobs to CI #5796

Merged
merged 123 commits into from
Oct 24, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
123 commits
Select commit Hold shift + click to select a range
ff0b563
Create benchmark-cron.yml
Illviljan Sep 14, 2021
bfb317c
move config
Illviljan Sep 14, 2021
6a3a91e
Create benchmarks.yml
Illviljan Sep 14, 2021
718ab63
Update asv.conf.json
Illviljan Sep 14, 2021
02daf51
Update asv.conf.json
Illviljan Sep 14, 2021
b5aebef
Create benchmarks-label-triggered.yml
Illviljan Sep 14, 2021
156a6f8
newlines at end
Illviljan Sep 14, 2021
448d92c
Update benchmark_cron.yml
Illviljan Sep 14, 2021
03c5c63
Delete benchmark_cron.yml
Illviljan Sep 14, 2021
2465696
Update benchmark-cron.yml
Illviljan Sep 14, 2021
13a1cee
rename
Illviljan Sep 14, 2021
9f79f15
Create benchmarks-no-label-triggered.yml
Illviljan Sep 14, 2021
5630aa2
Create README_CI.md
Illviljan Sep 15, 2021
30c801e
Update benchmarks-no-label-triggered.yml
Illviljan Sep 15, 2021
069acaa
Remove some test workflows.
Illviljan Sep 15, 2021
3b15f98
Update benchmarks-no-label-triggered.yml
Illviljan Sep 15, 2021
19727fb
Update benchmarks-no-label-triggered.yml
Illviljan Sep 15, 2021
8e9db9e
Update asv.conf.json
Illviljan Sep 15, 2021
7fdb4bd
Update asv.conf.json
Illviljan Sep 15, 2021
07b3b39
Remove label triggered workflow
Illviljan Sep 15, 2021
eac0236
Update benchmarks.yml
Illviljan Sep 15, 2021
459dc04
Add _skip_slow
Illviljan Sep 15, 2021
fab96b5
Update __init__.py
Illviljan Sep 15, 2021
1cd72dd
Update __init__.py
Illviljan Sep 15, 2021
15e03d0
global asv directory
Illviljan Sep 16, 2021
7b85bea
Update benchmarks.yml
Illviljan Sep 16, 2021
eef9311
Update benchmarks.yml
Illviljan Sep 16, 2021
0c9ddda
Update benchmarks.yml
Illviljan Sep 16, 2021
b72c984
Update benchmarks.yml
Illviljan Sep 16, 2021
888e275
Update benchmarks.yml
Illviljan Sep 16, 2021
4abd9c4
Update benchmarks.yml
Illviljan Sep 16, 2021
5f2bfec
Update benchmarks.yml
Illviljan Sep 16, 2021
1a04140
Update benchmarks.yml
Illviljan Sep 16, 2021
69316c4
Update repr.py
Illviljan Sep 16, 2021
61287e2
Update benchmarks.yml
Illviljan Sep 16, 2021
e361c5c
Update benchmarks.yml
Illviljan Sep 16, 2021
49e33fc
Update benchmarks.yml
Illviljan Sep 16, 2021
4c1adea
Create pandas.py
Illviljan Sep 16, 2021
9826976
Update pandas.py
Illviljan Sep 16, 2021
f2242d3
Update pandas.py
Illviljan Sep 16, 2021
64dd4ea
Create interp.py
Illviljan Sep 16, 2021
607916b
Update interp.py
Illviljan Sep 16, 2021
e4a2011
Update interp.py
Illviljan Sep 16, 2021
115f02c
Update interp.py
Illviljan Sep 16, 2021
468489b
check combine
Illviljan Sep 16, 2021
72e195f
Update combine.py
Illviljan Sep 16, 2021
15438bd
test missing
Illviljan Sep 16, 2021
f71630b
Update dataarray_missing.py
Illviljan Sep 16, 2021
46caf1d
Update dataarray_missing.py
Illviljan Sep 16, 2021
17b28f7
Update dataarray_missing.py
Illviljan Sep 16, 2021
ecde362
Update dataarray_missing.py
Illviljan Sep 16, 2021
94bc5ed
Update dataarray_missing.py
Illviljan Sep 16, 2021
66cb6e8
Update dataarray_missing.py
Illviljan Sep 16, 2021
2c481c3
Update dataarray_missing.py
Illviljan Sep 16, 2021
4db5d7e
Update dataarray_missing.py
Illviljan Sep 16, 2021
b9a8462
Update dataarray_missing.py
Illviljan Sep 16, 2021
1efb890
test unstacking
Illviljan Sep 16, 2021
603bec5
Update unstacking.py
Illviljan Sep 16, 2021
e598c29
test reindexing
Illviljan Sep 16, 2021
8fb88dc
Update reindexing.py
Illviljan Sep 16, 2021
19041ec
Update reindexing.py
Illviljan Sep 16, 2021
2bf57a2
test rolling
Illviljan Sep 16, 2021
46c1c43
Update rolling.py
Illviljan Sep 16, 2021
2a694ca
Update rolling.py
Illviljan Sep 16, 2021
278c858
Update rolling.py
Illviljan Sep 16, 2021
f1d9bef
Update rolling.py
Illviljan Sep 16, 2021
7b80778
add the tests back
Illviljan Sep 17, 2021
5e66930
Update dataarray_missing.py
Illviljan Sep 17, 2021
fa5855e
Merge branch 'pydata:main' into asv-benchmark-cron
Illviljan Sep 20, 2021
6553d58
Update rolling.py
Illviljan Sep 21, 2021
c068f8c
Update rolling.py
Illviljan Sep 22, 2021
5a846cc
nanmean gets divide by zero errors
Illviljan Sep 22, 2021
2745cbf
skip dataset_io
Illviljan Sep 24, 2021
5e639aa
Update dataset_io.py
Illviljan Sep 24, 2021
cffc06a
Update dataset_io.py
Illviljan Sep 24, 2021
2a4d32f
Update indexing.py
Illviljan Sep 25, 2021
7776a82
Update indexing.py
Illviljan Sep 25, 2021
1aaaa71
use parametrized
Illviljan Sep 27, 2021
0a378b5
Update indexing.py
Illviljan Sep 27, 2021
2e076bb
Update indexing.py
Illviljan Sep 27, 2021
89e203e
Update indexing.py
Illviljan Sep 28, 2021
fc6a2b5
Update indexing.py
Illviljan Sep 28, 2021
c1d85f1
Update whats-new.rst
Illviljan Sep 28, 2021
0278acc
Merge branch 'pydata:main' into asv-benchmark-cron
Illviljan Sep 28, 2021
b9496ae
Update dataset_io.py
Illviljan Sep 29, 2021
21f7a91
Update dataset_io.py
Illviljan Sep 29, 2021
d1aeeda
Run benchmarks with labels only, comment out ccache
Illviljan Oct 1, 2021
aeab4ca
Merge branch 'main' into asv-benchmark-cron
Illviljan Oct 1, 2021
b7b3070
Update benchmarks.yml
Illviljan Oct 1, 2021
dafbea6
Update benchmarks.yml
Illviljan Oct 1, 2021
206ac72
Update benchmarks.yml
Illviljan Oct 1, 2021
6b5c688
Update benchmarks.yml
Illviljan Oct 1, 2021
aad08ac
Revert "Merge branch 'main' into asv-benchmark-cron"
Illviljan Oct 1, 2021
7815882
Revert "Update benchmarks.yml"
Illviljan Oct 1, 2021
087ff76
Revert "Revert "Update benchmarks.yml""
Illviljan Oct 1, 2021
2b7858c
Update benchmarks.yml
Illviljan Oct 1, 2021
53cd0a7
test triggering with a label
Illviljan Oct 1, 2021
376e9e5
Update benchmarks.yml
Illviljan Oct 1, 2021
de63ee0
Update benchmarks.yml
Illviljan Oct 1, 2021
771b9b5
Merge branch 'main' into asv-benchmark-cron
Illviljan Oct 5, 2021
eac2545
Revert "Revert "Merge branch 'main' into asv-benchmark-cron""
Illviljan Oct 5, 2021
a66eef5
Update benchmarks.yml
Illviljan Oct 5, 2021
884b24a
Update benchmarks.yml
Illviljan Oct 5, 2021
743ba62
Update benchmarks.yml
Illviljan Oct 5, 2021
4d7ae6d
Update benchmarks.yml
Illviljan Oct 5, 2021
e3de111
Update benchmarks.yml
Illviljan Oct 5, 2021
431cfe6
Update benchmarks.yml
Illviljan Oct 5, 2021
6e73f9b
Update benchmarks.yml
Illviljan Oct 5, 2021
24ca03e
Update benchmarks.yml
Illviljan Oct 5, 2021
5c36e26
remove ccache
Illviljan Oct 6, 2021
919d794
Update benchmarks.yml
Illviljan Oct 6, 2021
91edc08
Try something else than mamba
Illviljan Oct 6, 2021
318e99e
Update benchmarks.yml
Illviljan Oct 6, 2021
6a2b855
Update benchmarks.yml
Illviljan Oct 6, 2021
321a761
test missing again
Illviljan Oct 6, 2021
1eba65c
Update dataarray_missing.py
Illviljan Oct 6, 2021
56556f1
Update dataarray_missing.py
Illviljan Oct 6, 2021
8f08506
Update dataarray_missing.py
Illviljan Oct 6, 2021
d1b908a
Update dataarray_missing.py
Illviljan Oct 6, 2021
8f262f9
Update dataarray_missing.py
Illviljan Oct 7, 2021
0b7b1a0
Update dataarray_missing.py
Illviljan Oct 7, 2021
712a453
Update dataarray_missing.py
Illviljan Oct 7, 2021
70cd679
add back tests
Illviljan Oct 7, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Revert "Merge branch 'main' into asv-benchmark-cron"
This reverts commit aeab4ca, reversing
changes made to d1aeeda.
  • Loading branch information
Illviljan committed Oct 1, 2021
commit aad08ac8cc570eb3e4156d1d2525e2adb2699d8d
4 changes: 2 additions & 2 deletions doc/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -260,8 +260,8 @@
ogp_image = "https://xarray.pydata.org/en/stable/_static/dataset-diagram-logo.png"
ogp_custom_meta_tags = [
'<meta name="twitter:card" content="summary_large_image" />',
'<meta property="twitter:site" content="@xarray_dev" />',
'<meta name="image" property="og:image" content="https://xarray.pydata.org/en/stable/_static/dataset-diagram-logo.png" />',
'<meta property="twitter:site" content="@xarray_dev />',
'<meta name="image" property="og:image" content="https://xarray.pydata.org/en/stable/_static/dataset-diagram-logo.png">',
]

# Redirects for pages that were moved to new locations
Expand Down
12 changes: 2 additions & 10 deletions doc/whats-new.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,15 +26,14 @@ New Features
By `Pushkar Kopparla <https://github.com/pkopparla>`_.
- Xarray now does a better job rendering variable names that are long LaTeX sequences when plotting (:issue:`5681`, :pull:`5682`).
By `Tomas Chor <https://github.com/tomchor>`_.
- Add an option to disable the use of ``bottleneck`` (:pull:`5560`)
- Add a option to disable the use of ``bottleneck`` (:pull:`5560`)
By `Justus Magin <https://github.com/keewis>`_.
- Added ``**kwargs`` argument to :py:meth:`open_rasterio` to access overviews (:issue:`3269`).
By `Pushkar Kopparla <https://github.com/pkopparla>`_.
- Added ``storage_options`` argument to :py:meth:`to_zarr` (:issue:`5601`).
By `Ray Bell <https://github.com/raybellwaves>`_, `Zachary Blackwood <https://github.com/blackary>`_ and
`Nathan Lis <https://github.com/wxman22>`_.
- Histogram plots are set with a title displaying the scalar coords if any, similarly to the other plots (:issue:`5791`, :pull:`5792`).
By `Maxime Liquet <https://github.com/maximlt>`_.


Breaking changes
~~~~~~~~~~~~~~~~
Expand All @@ -59,9 +58,6 @@ Deprecations

Bug fixes
~~~~~~~~~
- Fixed performance bug where ``cftime`` import attempted within various core operations if ``cftime`` not
installed (:pull:`5640`).
By `Luke Sewell <https://github.com/lusewell>`_

- Numbers are properly formatted in a plot's title (:issue:`5788`, :pull:`5789`).
By `Maxime Liquet <https://github.com/maximlt>`_.
Expand All @@ -88,10 +84,6 @@ Internal Changes
By `Jimmy Westling <https://github.com/illviljan>`_.
- Use isort's `float_to_top` config. (:pull:`5695`).
By `Maximilian Roos <https://github.com/max-sixty>`_.
- Refactor `xarray.core.duck_array_ops` to no longer special-case dispatching to
dask versions of functions when acting on dask arrays, instead relying numpy
and dask's adherence to NEP-18 to dispatch automatically. (:pull:`5571`)
By `Tom Nicholas <https://github.com/TomNicholas>`_.
- Add an ASV benchmark CI and improve performance of the benchmarks (:pull:`5796`)
By `Jimmy Westling <https://github.com/illviljan>`_.

Expand Down
27 changes: 9 additions & 18 deletions xarray/coding/cftime_offsets.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,15 +52,12 @@
from .cftimeindex import CFTimeIndex, _parse_iso8601_with_reso
from .times import format_cftime_datetime

try:
import cftime
except ImportError:
cftime = None


def get_date_type(calendar):
"""Return the cftime date type for a given calendar name."""
if cftime is None:
try:
import cftime
except ImportError:
raise ImportError("cftime is required for dates with non-standard calendars")
else:
calendars = {
Expand Down Expand Up @@ -102,8 +99,7 @@ def __add__(self, other):
return self.__apply__(other)

def __sub__(self, other):
if cftime is None:
raise ModuleNotFoundError("No module named 'cftime'")
import cftime

if isinstance(other, cftime.datetime):
raise TypeError("Cannot subtract a cftime.datetime from a time offset.")
Expand Down Expand Up @@ -225,8 +221,7 @@ def _adjust_n_years(other, n, month, reference_day):

def _shift_month(date, months, day_option="start"):
"""Shift the date to a month start or end a given number of months away."""
if cftime is None:
raise ModuleNotFoundError("No module named 'cftime'")
import cftime

delta_year = (date.month + months) // 12
month = (date.month + months) % 12
Expand Down Expand Up @@ -383,8 +378,7 @@ def onOffset(self, date):
return mod_month == 0 and date.day == self._get_offset_day(date)

def __sub__(self, other):
if cftime is None:
raise ModuleNotFoundError("No module named 'cftime'")
import cftime

if isinstance(other, cftime.datetime):
raise TypeError("Cannot subtract cftime.datetime from offset.")
Expand Down Expand Up @@ -469,8 +463,7 @@ def __apply__(self, other):
return _shift_month(other, months, self._day_option)

def __sub__(self, other):
if cftime is None:
raise ModuleNotFoundError("No module named 'cftime'")
import cftime

if isinstance(other, cftime.datetime):
raise TypeError("Cannot subtract cftime.datetime from offset.")
Expand Down Expand Up @@ -695,8 +688,7 @@ def to_offset(freq):


def to_cftime_datetime(date_str_or_date, calendar=None):
if cftime is None:
raise ModuleNotFoundError("No module named 'cftime'")
import cftime

if isinstance(date_str_or_date, str):
if calendar is None:
Expand Down Expand Up @@ -732,8 +724,7 @@ def _maybe_normalize_date(date, normalize):
def _generate_linear_range(start, end, periods):
"""Generate an equally-spaced sequence of cftime.datetime objects between
and including two dates (whose length equals the number of periods)."""
if cftime is None:
raise ModuleNotFoundError("No module named 'cftime'")
import cftime

total_seconds = (end - start).total_seconds()
values = np.linspace(0.0, total_seconds, periods, endpoint=True)
Expand Down
15 changes: 3 additions & 12 deletions xarray/coding/cftimeindex.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,12 +54,6 @@
from ..core.options import OPTIONS
from .times import _STANDARD_CALENDARS, cftime_to_nptime, infer_calendar_name

try:
import cftime
except ImportError:
cftime = None


# constants for cftimeindex.repr
CFTIME_REPR_LENGTH = 19
ITEMS_IN_REPR_MAX_ELSE_ELLIPSIS = 100
Expand Down Expand Up @@ -120,8 +114,7 @@ def parse_iso8601_like(datetime_string):


def _parse_iso8601_with_reso(date_type, timestr):
if cftime is None:
raise ModuleNotFoundError("No module named 'cftime'")
import cftime

default = date_type(1, 1, 1)
result = parse_iso8601_like(timestr)
Expand Down Expand Up @@ -196,8 +189,7 @@ def _field_accessor(name, docstring=None, min_cftime_version="0.0"):
"""Adapted from pandas.tseries.index._field_accessor"""

def f(self, min_cftime_version=min_cftime_version):
if cftime is None:
raise ModuleNotFoundError("No module named 'cftime'")
import cftime

version = cftime.__version__

Expand All @@ -223,8 +215,7 @@ def get_date_type(self):


def assert_all_valid_date_type(data):
if cftime is None:
raise ModuleNotFoundError("No module named 'cftime'")
import cftime

if len(data) > 0:
sample = data[0]
Expand Down
12 changes: 3 additions & 9 deletions xarray/coding/times.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,6 @@
unpack_for_encoding,
)

try:
import cftime
except ImportError:
cftime = None

# standard calendars recognized by cftime
_STANDARD_CALENDARS = {"standard", "gregorian", "proleptic_gregorian"}

Expand Down Expand Up @@ -169,8 +164,8 @@ def _decode_cf_datetime_dtype(data, units, calendar, use_cftime):


def _decode_datetime_with_cftime(num_dates, units, calendar):
if cftime is None:
raise ModuleNotFoundError("No module named 'cftime'")
import cftime

return np.asarray(
cftime.num2date(num_dates, units, calendar, only_use_cftime_datetimes=True)
)
Expand Down Expand Up @@ -419,8 +414,7 @@ def _encode_datetime_with_cftime(dates, units, calendar):
This method is more flexible than xarray's parsing using datetime64[ns]
arrays but also slower because it loops over each element.
"""
if cftime is None:
raise ModuleNotFoundError("No module named 'cftime'")
import cftime

if np.issubdtype(dates.dtype, np.datetime64):
# numpy's broken datetime conversion only works for us precision
Expand Down
47 changes: 19 additions & 28 deletions xarray/core/combine.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
import itertools
import warnings
from collections import Counter
from typing import Iterable, Sequence, Union

import pandas as pd

Expand Down Expand Up @@ -370,23 +369,16 @@ def _nested_combine(
return combined


# Define type for arbitrarily-nested list of lists recursively
# Currently mypy cannot handle this but other linters can (https://stackoverflow.com/a/53845083/3154101)
DATASET_HYPERCUBE = Union[Dataset, Iterable["DATASET_HYPERCUBE"]] # type: ignore


def combine_nested(
datasets: DATASET_HYPERCUBE,
concat_dim: Union[
str, DataArray, None, Sequence[Union[str, "DataArray", pd.Index, None]]
],
compat: str = "no_conflicts",
data_vars: str = "all",
coords: str = "different",
fill_value: object = dtypes.NA,
join: str = "outer",
combine_attrs: str = "drop",
) -> Dataset:
datasets,
concat_dim,
compat="no_conflicts",
data_vars="all",
coords="different",
fill_value=dtypes.NA,
join="outer",
combine_attrs="drop",
):
"""
Explicitly combine an N-dimensional grid of datasets into one by using a
succession of concat and merge operations along each dimension of the grid.
Expand Down Expand Up @@ -659,17 +651,16 @@ def _combine_single_variable_hypercube(

# TODO remove empty list default param after version 0.21, see PR4696
def combine_by_coords(
data_objects: Sequence[Union[Dataset, DataArray]] = [],
compat: str = "no_conflicts",
data_vars: str = "all",
coords: str = "different",
fill_value: object = dtypes.NA,
join: str = "outer",
combine_attrs: str = "no_conflicts",
datasets: Sequence[Dataset] = None,
) -> Union[Dataset, DataArray]:
data_objects=[],
compat="no_conflicts",
data_vars="all",
coords="different",
fill_value=dtypes.NA,
join="outer",
combine_attrs="no_conflicts",
datasets=None,
):
"""

Attempt to auto-magically combine the given datasets (or data arrays)
into one by using dimension coordinates.

Expand Down Expand Up @@ -764,7 +755,7 @@ def combine_by_coords(

Returns
-------
combined : xarray.Dataset or xarray.DataArray
combined : xarray.Dataset

See also
--------
Expand Down
11 changes: 4 additions & 7 deletions xarray/core/common.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,6 @@
from .rolling_exp import RollingExp
from .utils import Frozen, either_dict_or_kwargs, is_scalar

try:
import cftime
except ImportError:
cftime = None

# Used as a sentinel value to indicate a all dimensions
ALL_DIMS = ...

Expand Down Expand Up @@ -1825,7 +1820,9 @@ def is_np_timedelta_like(dtype: DTypeLike) -> bool:

def _contains_cftime_datetimes(array) -> bool:
"""Check if an array contains cftime.datetime objects"""
if cftime is None:
try:
from cftime import datetime as cftime_datetime
except ImportError:
return False
else:
if array.dtype == np.dtype("O") and array.size > 0:
Expand All @@ -1834,7 +1831,7 @@ def _contains_cftime_datetimes(array) -> bool:
sample = sample.compute()
if isinstance(sample, np.ndarray):
sample = sample.item()
return isinstance(sample, cftime.datetime)
return isinstance(sample, cftime_datetime)
else:
return False

Expand Down
Loading