Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bugfix/global ens 2series #413

Merged
merged 10 commits into from
Jan 19, 2024
Merged

Bugfix/global ens 2series #413

merged 10 commits into from
Jan 19, 2024

Conversation

ShelleyMelchior-NOAA
Copy link
Contributor

@ShelleyMelchior-NOAA ShelleyMelchior-NOAA commented Jan 18, 2024 •
edited
Loading

Pull Request Testing

This PR addresses item 34 on Wei's document.
There are lingering errors related to ECME stats not being available to be plotted. There are a number of variables, levels, or forecast leads where ECME does not have stats to be plotted. Defining the model list based on these limitations eliminates the WARNINGs.

  • Describe testing already performed for this Pull Request:

Only local changes have been made on dogwood by Shelley during cactus outage.
Steve has visually reviewed the changes on github.
Shelley will run the plots as user emc.vpppg upon submission of the PR.

UPDATE

All changes have been tested and the offending WARNINGs for ECME have been removed.
Please proceed w/ testing this PR to confirm the WARNINGs are removed.
tar files are different size than NCO para tar files; however, the expected graphics are all produced. We can watch this.

  • Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:

The following plots drivers need to be run.
jevs_global_ens_atmos_gefs_grid2grid_past31days_plots.sh
jevs_global_ens_atmos_gefs_grid2grid_past90days_plots.sh
jevs_global_ens_atmos_gefs_profile1_past31days_plots.sh
jevs_global_ens_atmos_gefs_profile1_past90days_plots.sh
jevs_global_ens_atmos_gefs_profile2_past31days_plots.sh
jevs_global_ens_atmos_gefs_profile2_past90days_plots.sh
jevs_global_ens_atmos_gefs_profile3_past31days_plots.sh
jevs_global_ens_atmos_gefs_profile3_past90days_plots.sh
jevs_global_ens_atmos_gefs_profile4_past31days_plots.sh
jevs_global_ens_atmos_gefs_profile4_past90days_plots.sh
jevs_global_ens_atmos_gefs_snowfall_past31days_plots.sh
jevs_global_ens_atmos_gefs_snowfall_past90days_plots.sh

Check for successful completion of the tar file.
Check for no "WARNING: ECME data" messages in the log file.

  • Has the code been checked to ensure that no errors occur during the execution? [Yes or No]

  • Do these updates/additions include sufficient testing updates? [Yes or No]

  • Please complete this pull request review by [Fill in date].

Pull Request Checklist

  • Review the source issue metadata (required labels, projects, and milestone).

  • Complete the PR description above.

  • Ensure the PR title matches the feature branch name.

  • Check the following:

  • Instructions provided on how to run

  • Developer's name is replaced by ${user} where necessary throughout the code

  • Check that the ecf file has all the proper definitions of variables

  • Check that the jobs file has all the proper settings of COMIN and COMOUT and other input variables

  • Check to see that the output directory structure is followed

  • Be sure that you are not using MET utilities outside the METplus wrapper structure

  • After submitting the PR, select Development issue with the original issue number.

  • After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.

  • Close the linked issue.

@PerryShafran-NOAA
Copy link
Contributor

I am submitting a new tests after pulling in the change from your latest commit.

@PerryShafran-NOAA
Copy link
Contributor

OK, the plot jobs are completed.

I still see WARNINGs for ECME data not being found in some of the output.

Also - the tar files do not match what is found in the NCO para.

All the .o files are here: /lfs/h2/emc/vpppg/noscrub/emc.vpppg/pr413test/EVS/dev/drivers/scripts/plots/global_ens

The tarballs are here: /lfs/h2/emc/ptmp/emc.vpppg/evs/v1.0/plots/global_ens/atmos.20240116

The tarballs for the NCO para are here: /lfs/h1/ops/para/com/evs/v1.0/plots/global_ens/atmos.20240116

@PerryShafran-NOAA
Copy link
Contributor

It seems that Marcel and Steve are still discussing further solutions to this issue in an email chain, so I guess stay tuned?

@ShelleyMelchior-NOAA

@ShelleyMelchior-NOAA
Copy link
Contributor Author

ShelleyMelchior-NOAA commented Jan 18, 2024 •
edited
Loading

Correct -- PR testing is still premature.

@ShelleyMelchior-NOAA
Copy link
Contributor Author

@PerryShafran-NOAA , you now have the 🟢 light! The PR description has been updated to indicate the PR is now legitimately ready for testing.

@PerryShafran-NOAA
Copy link
Contributor

@ShelleyMelchior-NOAA I am running the jobs on Dogwood under emc.vpppg in the pridev queue, since I do not believe that the data we need exists yet on Dogwood. Stay tuned.

@PerryShafran-NOAA
Copy link
Contributor

@ShelleyMelchior-NOAA The jobs are complete on Dogwood.

Please look at the .o files here: /lfs/h2/emc/vpppg/noscrub/emc.vpppg/pr413test/EVS/dev/drivers/scripts/plots/global_ens

I am still seeing ECME data WARNINGs in some of the most recent runs. I did do a git pull before the run.

Tarballs are here: /lfs/h2/emc/ptmp/emc.vpppg/evs/v1.0/plots/global_ens/atmos.20240117
NCO para tarballs are here: /lfs/h1/ops/para/com/evs/v1.0/plots/global_ens/atmos.20240117

Please note that tarballs are not the same size but it's close.

@ShelleyMelchior-NOAA
Copy link
Contributor Author

I concur that Perry ran the latest code updates.
Perry's latest runs:

[dlogin06 /lfs/h2/emc/vpppg/noscrub/emc.vpppg/pr413test/EVS/dev/drivers/scripts/plots/global_ens]$ lltr jevs_global_ens_*.o1258*
-rw-r--r-- 1 emc.vpppg vpppg  5563865 Jan 19 14:42 jevs_global_ens_atmos_gefs_grid2grid_past31days_plots.o125893368
-rw-r--r-- 1 emc.vpppg vpppg  5930329 Jan 19 14:45 jevs_global_ens_atmos_gefs_profile2_past31days_plots.o125893783
-rw-r--r-- 1 emc.vpppg vpppg  7295713 Jan 19 14:45 jevs_global_ens_atmos_gefs_snowfall_past31days_plots.o125893267
-rw-r--r-- 1 emc.vpppg vpppg  5869556 Jan 19 14:46 jevs_global_ens_atmos_gefs_grid2grid_past90days_plots.o125893389
-rw-r--r-- 1 emc.vpppg vpppg 11009323 Jan 19 14:46 jevs_global_ens_atmos_gefs_profile1_past31days_plots.o125893397
-rw-r--r-- 1 emc.vpppg vpppg  4975785 Jan 19 14:46 jevs_global_ens_atmos_gefs_profile4_past31days_plots.o125893873
-rw-r--r-- 1 emc.vpppg vpppg  9064328 Jan 19 14:48 jevs_global_ens_atmos_gefs_profile3_past31days_plots.o125893786
-rw-r--r-- 1 emc.vpppg vpppg  6236157 Jan 19 14:54 jevs_global_ens_atmos_gefs_profile2_past90days_plots.o125893785
-rw-r--r-- 1 emc.vpppg vpppg 11313763 Jan 19 14:57 jevs_global_ens_atmos_gefs_profile1_past90days_plots.o125893732
-rw-r--r-- 1 emc.vpppg vpppg  7604924 Jan 19 14:57 jevs_global_ens_atmos_gefs_snowfall_past90days_plots.o125893270
-rw-r--r-- 1 emc.vpppg vpppg  5285442 Jan 19 14:59 jevs_global_ens_atmos_gefs_profile4_past90days_plots.o125893938
-rw-r--r-- 1 emc.vpppg vpppg  9358977 Jan 19 15:01 jevs_global_ens_atmos_gefs_profile3_past90days_plots.o125893790

grid2obs 31 ✔️ 90 ✔️
snowfall 31 ✔️ 90 ✔️
profile1 31 ✔️ 90 ✔️
profile2 31 ✔️ 90 ✔️
profile3 31 ❗ 90 ✔️ --> what is different about profile3 for 31 vs 90 past days? 🤔
profile4 31 ✔️ 90 ✔️

I have put the dogwood logfiles into /dfs/read/emc/vpppg/shelley.melchior/logs-global_ens_plots on cactus for review by Steve and Marcel.

@PerryShafran-NOAA
Copy link
Contributor

Also if this needs to be run again, I can probably now do so on Cactus, the dev machine. I believe I have all the stats data from Dogwood transferred over to Cactus.

@ShelleyMelchior-NOAA
Copy link
Contributor Author

Perry, I think you have old code for profile3 past 31. See the setting for HOMEevs for the profile 3 past31 vs past90.

$ grep HOMEevs= jevs_global_ens_atmos_gefs_profile3_past*
jevs_global_ens_atmos_gefs_profile3_past31days_plots.o125893786:++ export HOMEevs=/lfs/h2/emc/vpppg/noscrub/emc.vpppg/EVS
jevs_global_ens_atmos_gefs_profile3_past31days_plots.o125893786:++ HOMEevs=/lfs/h2/emc/vpppg/noscrub/emc.vpppg/EVS
jevs_global_ens_atmos_gefs_profile3_past31days_plots.o125893786:+ HOMEevs=/lfs/h2/emc/vpppg/noscrub/emc.vpppg/EVS
jevs_global_ens_atmos_gefs_profile3_past31days_plots.o125893786:export HOMEevs=/lfs/h2/emc/vpppg/noscrub/${USER}/EVS
jevs_global_ens_atmos_gefs_profile3_past90days_plots.o125893790:++ export HOMEevs=/lfs/h2/emc/vpppg/noscrub/emc.vpppg/pr413test/EVS
jevs_global_ens_atmos_gefs_profile3_past90days_plots.o125893790:++ HOMEevs=/lfs/h2/emc/vpppg/noscrub/emc.vpppg/pr413test/EVS
jevs_global_ens_atmos_gefs_profile3_past90days_plots.o125893790:+ HOMEevs=/lfs/h2/emc/vpppg/noscrub/emc.vpppg/pr413test/EVS
jevs_global_ens_atmos_gefs_profile3_past90days_plots.o125893790:export HOMEevs=/lfs/h2/emc/vpppg/noscrub/${USER}/pr413test/EVS

Please re-run profile3 past 31 with HOMEevs set to /lfs/h2/emc/vpppg/noscrub/emc.vpppg/pr413test/EVS

@PerryShafran-NOAA
Copy link
Contributor

@ShelleyMelchior-NOAA I just confirmed that. I am re-running.

@PerryShafran-NOAA
Copy link
Contributor

@ShelleyMelchior-NOAA The job is complete:

/lfs/h2/emc/vpppg/noscrub/emc.vpppg/pr413test/EVS/dev/drivers/scripts/plots/global_ens/jevs_global_ens_atmos_gefs_profile3_past31days_plots.o125919281

There are no WARNINGs in this .o file.

@ShelleyMelchior-NOAA
Copy link
Contributor Author

I think this PR is ready for merge. 🚀

@PerryShafran-NOAA
Copy link
Contributor

@ShelleyMelchior-NOAA Did you do a sync fork?

@AliciaBentley-NOAA
Copy link
Contributor

The changed files look good and the output looks good! This PR can be merged. Thanks, all!

@ShelleyMelchior-NOAA
Copy link
Contributor Author

Thank you for the reminder ... sync'd!

PerryShafran-NOAA
PerryShafran-NOAA approved these changes Jan 19, 2024
View reviewed changes
Copy link
Contributor

@PerryShafran-NOAA PerryShafran-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes are as expected. Ready to merge.

@PerryShafran-NOAA PerryShafran-NOAA merged commit 2159356 into NOAA-EMC:develop Jan 19, 2024
@ShelleyMelchior-NOAA ShelleyMelchior-NOAA deleted the bugfix/global_ens_2series branch January 19, 2024 17:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@PerryShafran-NOAA PerryShafran-NOAA PerryShafran-NOAA approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ShelleyMelchior-NOAA @PerryShafran-NOAA @AliciaBentley-NOAA