Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEAT] seed/num_samples usage possibility + MSSE evaluation example #147

Merged
merged 14 commits into from
Dec 31, 2022
Merged

[FEAT] seed/num_samples usage possibility + MSSE evaluation example #147

merged 14 commits into from
Dec 31, 2022

Conversation

kdgutier
Copy link
Collaborator

@kdgutier kdgutier commented Dec 30, 2022 •
edited
Loading

This PR enables the generation of MSSE/CRPS evaluation confidence intervals.

  • Added seed/num_samples usage possibility to core.reconcile, fit_predict, fit, and get_sampler.
  • Added core.bootstrap method that loops through random seeds.
  • Updated evaluation example notebook with CRPS/MSSE with 95% confidence intervals.
  • Fixed PERMBU bug that had its previous location as BU mean, changed it towards reconciler mean.
  • Made reconciliation multiplication computationally efficient S @ P @ y_hat vs S @ ( P @ y_hat).

@kdgutier kdgutier requested a review from cchallu December 30, 2022 17:42
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

@cchallu cchallu merged commit 52746fe into main Dec 31, 2022
@cchallu cchallu deleted the msse_evaluation branch December 31, 2022 21:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cchallu cchallu cchallu approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@kdgutier @cchallu