Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue 389: minimum supported R version 3.5 or 4.1 #402

Merged
merged 22 commits into from
Jan 11, 2024
Merged

Issue 389: minimum supported R version 3.5 or 4.1 #402

merged 22 commits into from
Jan 11, 2024

Conversation

medewitt
Copy link
Collaborator

@medewitt medewitt commented Nov 25, 2023

Description

This PR closes #389 (maybe...).

Replaces post R4.1.0 anonymous function syntax (\(x)) and native pipe (|>) from the source and test files.
Does not remove or notify the users that R >= 4.1.0 is necessary for running or rebuilding the vignettes.
This will at least guarantee that the package will build and test on platforms with R >= 3.5.

Checklist

  • My PR is based on a package issue and I have explicitly linked it.
  • I have included the target issue or issues in the PR title in the for Issue(s) issue-numbers: PR title
  • I have read the contribution guidelines.
  • I have tested my changes locally.
  • I have added or updated unit tests where necessary.
  • I have updated the documentation if required.
  • My code follows the established coding standards.
  • I have added a news item linked to this PR.
  • I have reviewed CI checks for this PR and addressed them as far as I am able.

Copy link

codecov bot commented Nov 25, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (fda8ddb) 96.26% compared to head (4ec0f5f) 96.27%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #402   +/-   ##
=======================================
  Coverage   96.26%   96.27%           
=======================================
  Files          15       15           
  Lines        1903     1905    +2     
=======================================
+ Hits         1832     1834    +2     
  Misses         71       71           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if f448548 is merged into main:

  •   :ballot_box_with_check:day_of_week_model: 19.9s -> 19.6s [-9.68%, +6.66%]
  •   :ballot_box_with_check:latent_renewal_model: 23.7s -> 24.5s [-10.41%, +16.74%]
  •   :ballot_box_with_check:missingness_model: 1.3m -> 1.31m [-2.93%, +3.16%]
  •   :ballot_box_with_check:multi_group_latent_renewal_model: 6.07s -> 6.38s [-7.29%, +17.43%]
  •   :ballot_box_with_check:preprocessing: 694ms -> 696ms [-2.47%, +3.09%]
  •   :ballot_box_with_check:simple_model: 6.34s -> 4.27s [-112.73%, +47.63%]
  •   :ballot_box_with_check:simple_negbin_model_with_pp: 4.31s -> 4.31s [-25.5%, +25.22%]
    These benchmarks are based on package examples which are available here. Further explanation regarding interpretation and methodology can be found in the documentation of touchstone.

Copy link
Collaborator

@pearsonca pearsonca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor spacing notes - bit surprised lintr not triggering on them? maybe i misunderstand its rules.

tests/testthat/test-epinowcast.R Outdated Show resolved Hide resolved
tests/testthat/test-epinowcast.R Outdated Show resolved Hide resolved
tests/testthat/test-epinowcast.R Outdated Show resolved Hide resolved
@Bisaloo
Copy link
Collaborator

Bisaloo commented Nov 26, 2023

Minor spacing notes - bit surprised lintr not triggering on them? maybe i misunderstand its rules.

lintr is disabled in tests/. I agree it's not ideal. Happy to submit a PR for this.

@pearsonca
Copy link
Collaborator

Minor spacing notes - bit surprised lintr not triggering on them? maybe i misunderstand its rules.

lintr is disabled in tests/. I agree it's not ideal. Happy to submit a PR for this.

works for me - can't imagine there's so much legitimate false positive in /tests that we'd prefer to have it off. @seabbs ?

Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if f448548 is merged into main:

  •   :rocket:day_of_week_model: 20.9s -> 19.7s [-8.4%, -2.78%]
  •   :ballot_box_with_check:latent_renewal_model: 23.7s -> 23.8s [-22.49%, +23.36%]
  •   :ballot_box_with_check:missingness_model: 1.3m -> 1.29m [-4.19%, +2.09%]
  •   :ballot_box_with_check:multi_group_latent_renewal_model: 6.27s -> 6.08s [-19.77%, +13.55%]
  •   :ballot_box_with_check:preprocessing: 685ms -> 685ms [-1.48%, +1.31%]
  •   :ballot_box_with_check:simple_model: 4.42s -> 4.19s [-23.64%, +13.46%]
  •   :rocket:simple_negbin_model_with_pp: 4.71s -> 3.8s [-36.19%, -2.31%]
    These benchmarks are based on package examples which are available here. Further explanation regarding interpretation and methodology can be found in the documentation of touchstone.

@pearsonca
Copy link
Collaborator

Noted in conversation w/ @seabbs: should this work also include a specific check that epinowcast works with a minimum version? That is, an update to the CI to confirm install / test / etc works for == 3.5

@seabbs
Copy link
Collaborator

seabbs commented Nov 28, 2023

If we wanted to do this it would involve adding a matrix component for R 3.5 here https://github.com/epinowcast/epinowcast/blob/main/.github/workflows/R-CMD-check.yaml

@medewitt
Copy link
Collaborator Author

Good point. Perhaps just on Ubuntu? Or do we want the CI to include OSX?

@seabbs
Copy link
Collaborator

seabbs commented Nov 28, 2023

Perhaps just on Ubuntu?

Yeah I think so.

Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if b80247f is merged into main:

  •   :ballot_box_with_check:day_of_week_model: 20s -> 19.9s [-2.65%, +1.34%]
  •   :ballot_box_with_check:latent_renewal_model: 24s -> 24.7s [-7.04%, +12.85%]
  •   :ballot_box_with_check:missingness_model: 1.32m -> 1.32m [-3.82%, +4%]
  •   :ballot_box_with_check:multi_group_latent_renewal_model: 6.3s -> 6.08s [-8.4%, +1.27%]
  •   :ballot_box_with_check:preprocessing: 701ms -> 701ms [-2.32%, +2.35%]
  •   :ballot_box_with_check:simple_model: 4.75s -> 4.92s [-24.2%, +31.63%]
  •   :ballot_box_with_check:simple_negbin_model_with_pp: 4.18s -> 4.09s [-9.88%, +5.55%]
    These benchmarks are based on package examples which are available here. Further explanation regarding interpretation and methodology can be found in the documentation of touchstone.

@medewitt
Copy link
Collaborator Author

@pearsonca @seabbs good call as always on the addition of R3.5 to the test suite (I was holding my breathe with the GHA flow control). Looks like lifecycle is the only hard dependency that requires R >=3.6 while usethis and testthat are suggests requiring R>=3.6.

I think R3.6 is what ships with most Linux distributions as default now, so the bump to 3.6 as a minimum might be acceptable to most users. What do y'all think?

@pearsonca
Copy link
Collaborator

3.6, aye. I supposed the lifecycle hard dependency is on changes to how packages were managed in 3.5 => 3.6? Dunno.

Copy link
Collaborator

@seabbs seabbs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me once we bump up to 3.6 and add a news item. Nice one @medewitt / @pearsonca

tests/testthat/test-epinowcast.R Outdated Show resolved Hide resolved
pearsonca
pearsonca previously approved these changes Nov 28, 2023
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if b80247f is merged into main:

  •   :ballot_box_with_check:day_of_week_model: 20.3s -> 20.5s [-4.72%, +6.69%]
  •   :ballot_box_with_check:latent_renewal_model: 23.9s -> 23.9s [-4.29%, +4.07%]
  •   :ballot_box_with_check:missingness_model: 1.3m -> 1.33m [-2.16%, +6.82%]
  •   :ballot_box_with_check:multi_group_latent_renewal_model: 5.83s -> 6.28s [-7.58%, +22.63%]
  •   :ballot_box_with_check:preprocessing: 702ms -> 700ms [-1.07%, +0.34%]
  •   :ballot_box_with_check:simple_model: 4.37s -> 4.39s [-17.55%, +18.63%]
  •   :ballot_box_with_check:simple_negbin_model_with_pp: 4.62s -> 7.17s [-123.41%, +233.85%]
    These benchmarks are based on package examples which are available here. Further explanation regarding interpretation and methodology can be found in the documentation of touchstone.

@medewitt
Copy link
Collaborator Author

Whelp... found a test with the new syntax I missed and have an esoteric error on date conversions that I will hunt down.

@medewitt
Copy link
Collaborator Author

medewitt commented Nov 29, 2023

Found the weird date issue.

In R 4.1 if you pass a NULL to coerce_date you'll get an IDate of Length 0 and everything is great.

epinowcast::coerce_date(NULL)                                                                                                          
#> IDate of length 0

In R 3.6.1 you get an error:

epinowcast::coerce_date(NULL)                                                                                                          
#>Error in as.Date.default(x, tz = tz, ...) : 
#>   do not know how to convert 'x' to class “Date”

Following the stacktrace, the failure was in the base as.Date when passed the NULL value for the holidays argument.
Digging more, it looks like some time between R 3.6 and R 4.1 (update, it was here in April 2020) there was an addition of some flow control to handle NULL date values in the default:

# R4.1 source

as.Date.default <-                                                                                                                         
function (x, ...) 
{
    if (inherits(x, "Date")) 
        x
    else if (is.null(x)) # <- this is the important bit...
        .Date(numeric())
    else if (is.logical(x) && all(is.na(x))) 
        .Date(as.numeric(x))
    else stop(gettextf("do not know how to convert '%s' to class %s", 
        deparse1(substitute(x)), dQuote("Date")), domain = NA)
}

# R 3.6.1
as.Date.default  <-                                                                                                                        
function (x, ...) 
{
    if (inherits(x, "Date")) 
        x
    else if (is.logical(x) && all(is.na(x))) 
        .Date(as.numeric(x))
    else stop(gettextf("do not know how to convert '%s' to class %s", 
        deparse(substitute(x)), dQuote("Date")), domain = NA)
}

However as.IDate(.Date(numeric())) fixes the issue when passed a NULL. This solution appears robust to R version as well.

@medewitt
Copy link
Collaborator Author

Well, globalCallingHandlers is an R 4.0 feature which is used in the testing helpers here for controlling state variables.

I don't have a good alternative at the moment.

Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if b80247f is merged into main:

  •   :ballot_box_with_check:day_of_week_model: 20s -> 20.4s [-2.28%, +5.78%]
  •   :ballot_box_with_check:latent_renewal_model: 23.2s -> 25.2s [-4.51%, +21.56%]
  •   :ballot_box_with_check:missingness_model: 1.29m -> 1.31m [-3.24%, +5.71%]
  •   :ballot_box_with_check:multi_group_latent_renewal_model: 6.37s -> 6.33s [-10.41%, +9.06%]
  •   :ballot_box_with_check:preprocessing: 703ms -> 704ms [-2.12%, +2.48%]
  •   :ballot_box_with_check:simple_model: 4.27s -> 4.89s [-6.48%, +35.57%]
  •   :ballot_box_with_check:simple_negbin_model_with_pp: 5.06s -> 4.22s [-51.4%, +18.24%]
    These benchmarks are based on package examples which are available here. Further explanation regarding interpretation and methodology can be found in the documentation of touchstone.

@pearsonca
Copy link
Collaborator

Well, globalCallingHandlers is an R 4.0 feature which is used in the testing helpers here for controlling state variables.

I don't have a good alternative at the moment.

Hmm.

I think we should prefer to have unit testing only require the same environment as the package requires - while possible to ensure that it's a real test for the minimal supported version of R, seems 1) time consuming to ensure and 2) still error prone. However, as a last resort, the tests are not what we provide the user.

@Bisaloo @seabbs Does it work for that script to have an if-else block to sniff version, and not add those handlers for R < 4? They don't seem to be critical to making tests pass, just causing them to fail if there's an inadvertant modification of global state. That seems like it can just be covered by testing for 4+, as I doubt there'd be a case where state would be modified in <4, but not in >= 4.

@pearsonca
Copy link
Collaborator

Found the weird date issue.

In R 4.1 if you pass a NULL to coerce_date you'll get an IDate of Length 0 and everything is great.

...also, is everything great? Do we want a NULL here to pass or to error? I'd have to look a bit closely at the context, but the rule of thumb is passing something a NULL indicates trying to do something that doesn't make sense (or at least warrants a warn/message).

@Bisaloo
Copy link
Collaborator

Bisaloo commented Nov 29, 2023

@Bisaloo @seabbs Does it work for that script to have an if-else block to sniff version, and not add those handlers for R < 4? They don't seem to be critical to making tests pass, just causing them to fail if there's an inadvertant modification of global state. That seems like it can just be covered by testing for 4+, as I doubt there'd be a case where state would be modified in <4, but not in >= 4.

Yes, this sounds good to me 👍

@medewitt
Copy link
Collaborator Author

Found the weird date issue.
In R 4.1 if you pass a NULL to coerce_date you'll get an IDate of Length 0 and everything is great.

...also, is everything great? Do we want a NULL here to pass or to error? I'd have to look a bit closely at the context, but the rule of thumb is passing something a NULL indicates trying to do something that doesn't make sense (or at least warrants a warn/message).

Right now we have holidays set to NULL in the enw_add_metaobs_features function, but describe it as an empty vector in the docs:

#' @param holidays a (potentially empty) vector of dates (or input

A guess a couple of actions we could do:

  1. Change the default in enw_add_metaobs_features to an empty vector i.e. : c() which default conververts to NULL
  2. Change the language in the docs to mention that if no holidays are passed no metadata are added regarding these
  3. Add a warning message to
    holidays <- coerce_date(holidays)
    indicating that no holiday metadata were added
  4. Add a warning message to coerce_date that a NULL was passed and treating as a zero-length date

I think (3) and (4) we should probably do, and maybe (2). What do you think?

@medewitt
Copy link
Collaborator Author

TIL stopifnot informative error messages are another R 4.0 innovation.

Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if b80247f is merged into main:

  •   :ballot_box_with_check:day_of_week_model: 19.4s -> 20.2s [-0.11%, +7.77%]
  •   :ballot_box_with_check:latent_renewal_model: 23.6s -> 24.2s [-9.27%, +14.36%]
  •   :ballot_box_with_check:missingness_model: 1.33m -> 1.31m [-6.14%, +2.75%]
  •   :ballot_box_with_check:multi_group_latent_renewal_model: 6.26s -> 6.13s [-18.74%, +14.53%]
  •   :ballot_box_with_check:preprocessing: 691ms -> 689ms [-1.94%, +1.39%]
  •   :ballot_box_with_check:simple_model: 4.25s -> 4.55s [-15.37%, +29.61%]
  •   :ballot_box_with_check:simple_negbin_model_with_pp: 4.31s -> 4.34s [-45.4%, +46.73%]
    These benchmarks are based on package examples which are available here. Further explanation regarding interpretation and methodology can be found in the documentation of touchstone.

@seabbs
Copy link
Collaborator

seabbs commented Nov 29, 2023

Wow this is turning into an iceberg...

informative error messages are another R 4.0 innovation.

It sounds like this might have to be the minimum version then unless we want to experience some real pain reverting these (and having a less clean code base).

It sounds like we probably want to do all of 1-4 as long as that doesn't result in all the default models throwing warnings (as not passing holidays is likely the default behaviour of most users).

@seabbs
Copy link
Collaborator

seabbs commented Nov 29, 2023

Given all this (sorry!) I am wondering if we just want to have 4.1 as our minimum version and make sure to test that going forward?

Copy link
Contributor

github-actions bot commented Jan 4, 2024

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 473ab89 is merged into main:

  •   :ballot_box_with_check:day_of_week_model: 20.1s -> 20.9s [-5.05%, +12.85%]
  •   :ballot_box_with_check:latent_renewal_model: 24.2s -> 23.3s [-15.31%, +7.9%]
  •   :ballot_box_with_check:missingness_model: 1.31m -> 1.31m [-6.68%, +6.74%]
  •   :ballot_box_with_check:multi_group_latent_renewal_model: 6.25s -> 6.7s [-1.93%, +16.07%]
  •   :ballot_box_with_check:preprocessing: 500ms -> 505ms [-1.1%, +2.9%]
  •   :ballot_box_with_check:simple_model: 4.31s -> 4.34s [-27.21%, +28.63%]
  •   :ballot_box_with_check:simple_negbin_model_with_pp: 4.21s -> 4.1s [-25.05%, +19.97%]
    These benchmarks are based on package examples which are available here. Further explanation regarding interpretation and methodology can be found in the documentation of touchstone.

Copy link
Collaborator

@pearsonca pearsonca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Documentation changes seem not quite right.

R/utils.R Outdated Show resolved Hide resolved
@seabbs seabbs requested a review from pearsonca January 4, 2024 15:56
pearsonca
pearsonca previously approved these changes Jan 4, 2024
Copy link
Contributor

github-actions bot commented Jan 4, 2024

This is how benchmark results would change (along with a 95% confidence interval in relative change) if fda8ddb is merged into main:

  •   :ballot_box_with_check:day_of_week_model: 20.2s -> 20.4s [-3.63%, +5.22%]
  •   :ballot_box_with_check:latent_renewal_model: 25.8s -> 23.6s [-22.73%, +5.65%]
  •   :ballot_box_with_check:missingness_model: 1.33m -> 1.33m [-3.09%, +2.68%]
  •   :ballot_box_with_check:multi_group_latent_renewal_model: 6.01s -> 6.31s [-13.73%, +23.54%]
  •   :ballot_box_with_check:preprocessing: 503ms -> 498ms [-3.94%, +1.81%]
  •   :ballot_box_with_check:simple_model: 5.78s -> 4.62s [-59.35%, +19.09%]
  •   :ballot_box_with_check:simple_negbin_model_with_pp: 4.54s -> 4.21s [-48%, +33.42%]
    These benchmarks are based on package examples which are available here. Further explanation regarding interpretation and methodology can be found in the documentation of touchstone.

@pearsonca
Copy link
Collaborator

Looking at the code coverage block, should be easy enough to address adding a copy-paste of the expect_identical block, but adding a integer() argument.

If you do so, eliminating the check null vs check empty branch distinction seems worthwhile to improve the static analysis. The tests set the contract, not the logic flow itself.

@seabbs seabbs requested a review from pearsonca January 4, 2024 23:04
Copy link
Contributor

github-actions bot commented Jan 4, 2024

This is how benchmark results would change (along with a 95% confidence interval in relative change) if fda8ddb is merged into main:

  •   :ballot_box_with_check:day_of_week_model: 19.9s -> 19.9s [-6.04%, +5.96%]
  •   :ballot_box_with_check:latent_renewal_model: 23.3s -> 23.5s [-7.9%, +9.25%]
  •   :ballot_box_with_check:missingness_model: 1.31m -> 1.34m [-1.57%, +6.24%]
  •   :ballot_box_with_check:multi_group_latent_renewal_model: 5.95s -> 6.18s [-8.25%, +15.85%]
  •   :ballot_box_with_check:preprocessing: 502ms -> 500ms [-1.2%, +0.51%]
  •   :ballot_box_with_check:simple_model: 4.41s -> 4.54s [-25.39%, +31.32%]
  •   :ballot_box_with_check:simple_negbin_model_with_pp: 4.17s -> 4.51s [-26.6%, +42.96%]
    These benchmarks are based on package examples which are available here. Further explanation regarding interpretation and methodology can be found in the documentation of touchstone.

@seabbs
Copy link
Collaborator

seabbs commented Jan 11, 2024

@pearsonca do you have time to give this a final review? I think its good t ogo.

@seabbs seabbs added this pull request to the merge queue Jan 11, 2024
Merged via the queue into main with commit 4c1b0fe Jan 11, 2024
13 checks passed
@seabbs seabbs deleted the issue389 branch January 11, 2024 15:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

minimum supported R version 3.5 or 4.1
5 participants