Bugfix/cam rev7 #405

MarcelCaron-NOAA · 2024-01-03T18:12:25Z

Pull Request Testing

Describe testing already performed for this Pull Request:

This PR addresses issues 34, 36, 39-43, 49, 55, and 57 noted in the Google Doc titled "EVS v1.0 issues". Information about the cause and fix of each:

34 - (1) A block of code writes a variable that isn't used. When another plots configuration variable, "event_equalization", is set to False, the unused variable is set to Nonetype, and python outputs a warning. Fix: I opted to entirely remove the block of code with the unused variable. (2) RH2m verification is not available for ECME, so a WARNING exists signaling that ECME data are not found when RH2m plots are made. Fix: Moved models list definition under an if blocks that check whether or not RH2m, DPT2m, or CAPEsfc is being plotted.
36 - OMP_NUM_THREADS was not set to "1" in the driver script. Fix: define OMP_NUM_THREADS=1 in the driver.
39 - stats from href products cannot be plotted for two reasons. (1) the model name in href_mean stat file names and contents don't match, and (2) href grid2obs stats job is not configured to produce href_avrg, href_lpmm, and href_pmmn stats (only href_mean and href_prob). Temporary fix: Remove href products from list of models to plot in headline graphics.
40 - METplus uses the time-minus setting in the prepbufr filename to verify forecasts valid at 03, 09, 15, and 21Z with 06, 12, 18, and 00Z prepbufr files, respectively. Time-minus offsets between 1 to 6 hours were attempted until a matching file was found, hence the warnings, all which were ultimately followed by a "found file" log message. Fix: To reduce warnings, I've changed METplus offsets list at line 50 in parm/metplus_config/stats/cam/grid2obs/PB2NC_obsMETAR.conf from "PB2NC_OFFSETS = 6, 5, 4, 3, 2, 1" to "PB2NC_OFFSETS = 6, 3".
41 - The original developer used the ensemble threshold setting to allow GenEnsProd jobs to succeed at F048 when some time-lagged members were not available, but, with this solution, METplus outputs warnings to the log files. Fix: Fixed based on the preference that no warning message occurs at all: Defined three new environment variables to be used by METplus; the first sets the number of model members, the second sets an ensemble threshold, the third sets the list of filenames to use, based on whether 7 or all 10 HREF members are used.
42 - Graphics were produced using all five cam models (hiresws, hrrr, and nam nest) but for some graphics, stats only exist for a subset of those models, hence the warnings. Fix: SNOD and 06Z/18Z graphics are only produced for the hrrr and namnest; graphics for longer forecasts are only produced for the namnest and hireswfv3.
43 - Runtime increased. Fix: Use the suggested resource allocation
49 - Several warnings in the href spcoutlook stats job caused by three separate issues. (1) Bilinear interpolation is selected with 4 interpolation points (incompatible), (2) MLCAPE levels are P0-90 and L100000, and (3) files that are never available are checked for. Fix: (1) Use 2 interp points, (2) Use P90-0 and L90000, and (3) do not check for files that are always unavailable.
55 - Default COMIN paths were incorrect. Fix: Added prefixes to correct.
57 - Default COMOUT paths were incorrect. Fix: Removed prefixes to correct.

This PR also addresses the following issues:

cam jobs fail when MAILTO is not set and SENDMAIL=NO. Fix: Access MAILTO environment variable from python only if SENDMAIL=YES, raise an error only if MAILTO is not set and SENDMAIL=YES.
When nam firewxnest files (used to determine the firewx domain) are missing, email warning messages refer to the files as "Unknown Data Type" because both NAM FireWx Nest and NAM Nest match the file name. Fix: Check that "firewx" is not included in the filename when attempting to match with NAM Nest forecasts.
"test" comment logged in threshold average processes. Fix: Removed.

For the most part, the affected jobs were not tested.

Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:

(1) Set up jobs

• symlink the EVS_fix directory locally as "fix"
• In all driver scripts, point all exports of "HOMEevs" to your EVS test directory
• In all stats driver scripts, change COMIN to the parallel directory (/lfs/h2/emc/vpppg/noscrub/emc.vpppg/$NET/$evs_ver_2d)
• Optional: In the stats driver scripts, change "MAILTO" to point to your email address

(2) Test Jobs

Use the following to submit ...

... jevs_cam_hireswarw_grid2obs_stats: qsub –v vhr=03
... all other jobs: qsub –v vhr=00

From your EVS testing directory, the following scripts should be run using the setup in step (1):

dev/drivers/scripts/prep/cam/jevs_cam_href_severe_prep.sh

dev/drivers/scripts/stats/cam/jevs_cam_hireswarw_grid2obs_stats.sh
dev/drivers/scripts/stats/cam/jevs_cam_href_spcoutlook_stats.sh
dev/drivers/scripts/stats/global_ens/jevs_global_ens_wave_grid2obs_stats.sh

dev/drivers/scripts/plots/cam/jevs_cam_grid2obs_plots.sh
dev/drivers/scripts/plots/cam/jevs_cam_headline_plots.sh
dev/drivers/scripts/plots/cam/jevs_cam_snowfall_plots.sh
dev/drivers/scripts/plots/global_ens/jevs_global_ens_atmos_gefs_grid2grid_past31days_plots
dev/drivers/scripts/plots/global_ens/jevs_global_ens_atmos_gefs_grid2obs_past31days_plots

[Total: 1 prep jobs, 2 stats jobs, 4 plots jobs]

(3) Review

I'll check for red flags:

check="FATAL\|error\|Killed\|Cgroup\|argument expected\|No such file\|cannot\|empty\|failed\|unexpected\|exceeded"
grep "$check" $outfile

Has the code been checked to ensure that no errors occur during the execution? NO

Do these updates/additions include sufficient testing updates? YES

Please complete this pull request review by ASAP.

Pull Request Checklist

… F048)

ShelleyMelchior-NOAA · 2024-01-05T14:42:33Z

I have cloned your fork and checked out branch bugfix/cam_rev7.

ShelleyMelchior-NOAA · 2024-01-05T14:49:16Z

Beginning w/ item 34 in Wei's document ...

I ran the global_ens plotting scripts for VDATE=20231230:
jevs_global_ens_atmos_gefs_grid2obs_past31days_separate_plots.sh
jevs_global_ens_atmos_gefs_grid2obs_past31days_plots.sh
jevs_global_ens_atmos_gefs_grid2grid_past31days_plots.sh
Output log files: /lfs/h2/emc/ptmp/shelley.melchior/EVS_out/jevs_global_ens_atmos_gefs_grid2*_plots.o1229792*
I no longer see any WARNINGs for event_equalization.

[dlogin06 /lfs/h2/emc/ptmp/shelley.melchior/EVS_out]$ grep event_equalization jevs_global_ens_atmos_gefs_grid2*plots.o12297*
[dlogin06 /lfs/h2/emc/ptmp/shelley.melchior/EVS_out]$

The only WARNINGs that persist are for ECME data not being available.

COMOUT=/lfs/h2/emc/ptmp/shelley.melchior/evs/v1.0/plots/global_ens/atmos.20231230
@StevenSimon-NOAA , can you check the plots to make sure all appears OK?

StevenSimon-NOAA · 2024-01-05T15:32:58Z

Beginning w/ item 34 in Wei's document ...

I ran the global_ens plotting scripts for VDATE=20231230: jevs_global_ens_atmos_gefs_grid2obs_past31days_separate_plots.sh jevs_global_ens_atmos_gefs_grid2obs_past31days_plots.sh jevs_global_ens_atmos_gefs_grid2grid_past31days_plots.sh Output log files: /lfs/h2/emc/ptmp/shelley.melchior/EVS_out/jevs_global_ens_atmos_gefs_grid2*_plots.o1229792* I no longer see any WARNINGs for event_equalization.
[dlogin06 /lfs/h2/emc/ptmp/shelley.melchior/EVS_out]$ grep event_equalization jevs_global_ens_atmos_gefs_grid2*plots.o12297*
[dlogin06 /lfs/h2/emc/ptmp/shelley.melchior/EVS_out]$
The only WARNINGs that persist are for ECME data not being available.

COMOUT=/lfs/h2/emc/ptmp/shelley.melchior/evs/v1.0/plots/global_ens/atmos.20231230 @StevenSimon-NOAA , can you check the plots to make sure all appears OK?

The tar files for these three plot jobs were successfully produced, and the expected plot png quantities for each tar file were confirmed.

ShelleyMelchior-NOAA · 2024-01-05T15:38:09Z

Item 36 in Wei's document ...

I ran jevs_global_ens_wave_grid2obs_stats.sh w/ VDATE=20231230.
logfile: /lfs/h2/emc/ptmp/shelley.melchior/EVS_out/jevs_global_ens_wave_grid2obs_stats.o122990555

The WARNING for OMP_NUM_THREADS is gone.

[dlogin06 /lfs/h2/emc/ptmp/shelley.melchior/EVS_out]$ grep WARNING: jevs_global_ens_wave_grid2obs_stats.o122990555
[dlogin06 /lfs/h2/emc/ptmp/shelley.melchior/EVS_out]$

I confirmed there are many of these warning statements in the output for the emc.vpppg parallel runs.

The PR corrects item 36. ✔️

ShelleyMelchior-NOAA · 2024-01-05T16:30:23Z

Item 40 in Wei's document ...

I ran jevs_cam_hireswarw_grid2obs_stats.sh w/ VDATE=20240101 and vhr=03
logfile: /lfs/h2/emc/ptmp/shelley.melchior/EVS_out/jevs_cam_hireswarw_grid2obs_stats.o122995121

The WARNING persists, but is reduced, as per the proposed improvement described by Marcel. Now the warning messages are for 3-hr intervals rather than an hourly cycle frequency. The WARNING messages are only cycles 09, 15.

[dlogin06 /lfs/h2/emc/ptmp/shelley.melchior/EVS_out]$ grep WARNING: jevs_cam_hireswarw_grid2obs_stats.o122995121 | grep Could | wc -l                      
56

The logfile for emc.vpppg (same VDATE and vhr: /lfs/h2/emc/ptmp/emc.vpppg/output/jevs_cam_hireswarw_grid2obs_stats.o122891654) shows more WARNING (~168) messages and they are for hourly cycles.

The PR addresses item 40 appropriately. ✔️

ShelleyMelchior-NOAA · 2024-01-05T18:36:59Z

Item 41 in Wei's document ...

I ran jevs_cam_href_severe_prep.sh for VDATE=20231229 and vhr=12.
logfile: /lfs/h2/emc/ptmp/shelley.melchior/EVS_out/jevs_cam_href_severe_prep.o123021975
No more WARNING messages are present.

[dlogin06 /lfs/h2/emc/ptmp/shelley.melchior/EVS_out]$ grep WARNING: jevs_cam_href_severe_prep.o123021975 | grep FCST_INPUT | wc -l
0

I confirmed they exist in Wei's parallel for the same VDATE and vhr.

[clogin08 /lfs/h1/ops/para/output/20240105]$ grep WARNING evs_cam_href_severe_prep_vhr_12.o108676344 | grep FCST_INPUT | wc -l
6

This PR corrects item 41. ✔️

ShelleyMelchior-NOAA · 2024-01-05T18:59:33Z

Items 42 and 43 in Wei's document ...

I am running jevs_cam_snowfall_plots.sh and jevs_cam_grid2obs_plots.sh, but set for VDATE=20240104. There are some missing stats so I expect to see some incomplete plots. These processes both take a little while to complete.

    Req'd  Req'd   Elap
Job ID                         Username        Queue           Jobname         SessID   NDS  TSK   Memory Time  S Time
------------------------------ --------------- --------------- --------------- -------- ---- ----- ------ ----- - -----
123024666.dbqs01               shelley.melchi* dev             jevs_cam_snowf*    66438    4   512    --  05:30 R 00:12
123025141.dbqs01               shelley.melchi* dev             jevs_cam_grid2*   144658   10  1280    --  12:00 R 00:10

For item 42 we are looking for no more WARNING messages related to Empty dataframes, as well as SNOD graphics only generated for hrrr and namnest. So far there are no occurrences of the WARNING message. The same run in Wei's parallel produced the warning message 75,420 times! 🤞

[clogin08 /lfs/h1/ops/para/output/20240104]$ grep WARNING evs_cam_snowfallots.o108226553 | grep Empty | wc -l
75420

For item 43 we are looking for reduced runtime of jevs_cam_grid2obs_plots.sh.

UPDATE: the snowfall plots job completed. I still see quite a few WARNING messages.

[dlogin06 /lfs/h2/emc/ptmp/shelley.melchior/EVS_out]$ grep WARNING jevs_cam_snowfall_plots.o123024666 | grep Empty | wc -l
6300

COMOUT=/lfs/h2/emc/ptmp/shelley.melchior/evs/v1.0/plots/cam/atmos.20240104

ShelleyMelchior-NOAA · 2024-01-08T14:39:53Z

It looks like COMOUTplots aged off already. I will re-run the cam plots processing once the dev queues are re-established on dogwood later today.

MarcelCaron-NOAA · 2024-01-08T16:51:55Z

Thanks very much Shelley and Steve for reviewing this PR so far.

Beginning w/ item 34 in Wei's document ...
The only WARNINGs that persist are for ECME data not being available.

I missed this second concern about the missing ECME data, looks to be a separate issue. I'll check that and update.

That leaves 34, 42, and 43. An update for 34 and 42 will be added tomorrow, and new items 39, 49, and 50 will be added shortly after. None of the new items should require retesting the above already-tested jobs.

MarcelCaron-NOAA · 2024-01-08T16:54:25Z

I forgot to add: I'm also adding item 55 today. For simplicity, this may be tested using one of the jobs we'll test for the other new items.

ShelleyMelchior-NOAA · 2024-01-08T17:11:01Z

Thanks Marcel! Re: item 43, I checked the log file (/lfs/h2/emc/ptmp/shelley.melchior/EVS_out/jevs_cam_grid2obs_plots.o123025141) since it seems the only thing we were really looking for was if the increased nproc could reduce the runtime. The job took from 18:44 to 23:27, for a total of 4:43.

But I also looked at the log file outputs from the NCO parallel, which appears to have nproc=1280 in the ecf script.
/lfs/h1/ops/para/packages/evs.v1.0.1/ecf/scripts/plots/cam/jevs_cam_grid2obs_plots.ecf
This run took from 00:01 to 13:41. /lfs/h1/ops/para/output/20240107/evs_cam_grid2obs_plots.o110528970
This run took from 00:00 to 13:45. /lfs/h1/ops/para/output/20240106/evs_cam_grid2obs_plots.o109748875

MarcelCaron-NOAA · 2024-01-08T17:26:24Z

Yes, the only change is increased processors, and 4:43 is an excellent result. I wonder though why the large difference compared to the parallel jobs. Is it possible to copy those logs to Hera today? I can also check them tomorrow.

ShelleyMelchior-NOAA · 2024-01-08T20:32:55Z

Marcel, I put the cactus log files in /dfs. Once dogwood returns, you can find them in /dfs/read/emc/vpppg/shelley.melchior/forMC

ShelleyMelchior-NOAA · 2024-01-10T19:08:26Z

I re-ran the global_ens grid2obs plots processes for 31 days following the commit #79566c4.
Furthermore, based on the changes Marcel provided, Steve identified another needed adjustment to address DPT2m.
The code I changed:

/lfs/h2/emc/vpppg/noscrub/shelley.melchior/githubwkspc/forks/MC/EVS/scripts/plots/global_ens/exevs_global_ens_atmos_gefs_grid2obs_plots.sh
/lfs/h2/emc/vpppg/noscrub/shelley.melchior/githubwkspc/forks/MC/EVS/scripts/plots/global_ens/exevs_global_ens_atmos_gefs_grid2obs_separate_plots.sh

The WARNING messages for ECME and CMCE are gone for the non-separate plots.
The WARNINGs persist (for ECME and CMCE) for the separate plots.
logfiles:
/lfs/h2/emc/ptmp/shelley.melchior/EVS_out/jevs_global_ens_atmos_gefs_grid2obs_past31days_plots.o112709582
/lfs/h2/emc/ptmp/shelley.melchior/EVS_out/jevs_global_ens_atmos_gefs_grid2obs_past31days_separate_plots.o112709771

MarcelCaron-NOAA · 2024-01-10T19:28:52Z

Thanks @ShelleyMelchior-NOAA and @StevenSimon-NOAA! I'll update soon, once I have a fix for the lingering issue in the "separate" job

A snowfall fix is committed here, but I am re-testing because I expect a few more WARNINGS will exist. I should also have an update about that soon.

AliciaBentley-NOAA · 2024-01-10T19:47:27Z

@MarcelCaron-NOAA Thank you for working on this. The "separate" scripts from Binbin separate different initialization times so that we have seperate 00Z and 12Z plots for near surface variables like 2-m temperature. Here is an example webpage:
https://www.emc.ncep.noaa.gov/users/verification/global/gefs/para/atmos/grid2obs/tmp_z2_small/

It looks like these plots broke several days ago (other plots have lines that extend out past this). I'm unsure if the timing of the break is associated with a PR that was merged or what, but it would be good to get this working again.

CCing @StevenSimon-NOAA and @ShelleyMelchior-NOAA

… "group" or "set" in model name

StevenSimon-NOAA · 2024-01-10T21:14:06Z

@StevenSimon-NOAA, @AliciaBentley-NOAA Yes that makes sense. The linked graphic represents the 31 days leading up to VDATE=20240107 (IDATE=20240102 for F120), and global_ens plots jobs use VDATE=PDYm2, so this was made yesterday. I think we are good!

It looks like the "missing ECME data will not be plotted" WARNINGs have been successfully addressed for the g2o separate init plots.

Please see the following logfile to double check if necessary: /lfs/h2/emc/ptmp/shelley.melchior/EVS_out/jevs_global_ens_atmos_gefs_grid2obs_past31days_separate_plots.o112718556

MarcelCaron-NOAA · 2024-01-10T21:23:19Z

Updates

Re: global-ens (34) - Thanks @StevenSimon-NOAA, I agree. Removing ECME from CAPEsfc plots I think fixed the ECME-only warnings. The CMCE warnings are also for CAPEsfc and trigger when the CAPE metrics are NaN. Part of those warning messages: "This may occur if no forecast or observed events were counted at any threshold for any model, so that all performance statistics are undefined." The lowest threshold is 250 J/kg. I think such warnings are expected during the cool season.

Re: snowfall (42) - Removed models from plots when they have no data (06Z, 18Z inits for hiresws; or longer leads for hrrr and some hiresws) and muted some unhelpful warnings. I think I got everything but I'm running one more test to check.

CC: @ShelleyMelchior-NOAA

ShelleyMelchior-NOAA · 2024-01-10T21:40:03Z

I concur that global_ens g2o plotting, Wei's item 34, is addressed. ✔️

MarcelCaron-NOAA · 2024-01-10T22:01:58Z

I tested the changes to jevs_cam_snowfall_plots and few warnings remain, all seemingly for graphics in CONUS_South, when the lowest snowfall thresholds are too high to find any events (similar to the CAPEsfc warnings described above). I think this job is ready to be reviewed here again.

ShelleyMelchior-NOAA · 2024-01-11T15:24:31Z

Marcel -- I am getting started on running tests. Can you do a sync fork on your branch. 2 PRs were merged yesterday. Thanks!

ShelleyMelchior-NOAA · 2024-01-11T15:36:15Z

Wei's item 43 -- I am re-running jevs_cam_snowfall_plots.sh. It will take a little bit to complete.

112803617.cbqs01               shelley.melchi* dev             jevs_cam_snowf*   207248    4   512    --  05:30 R 00:02

logfile: /lfs/h2/emc/ptmp/shelley.melchior/EVS_out/jevs_cam_snowfall_plots.o112803617

ShelleyMelchior-NOAA · 2024-01-11T15:43:35Z

Wei's item 55 -- Changes to JEVS_CAM_PREP and JEVS_MESOSCALE_STATS are correct and follow the suggestion. ✔️

ShelleyMelchior-NOAA · 2024-01-11T15:58:44Z

Wei's item 57 -- Changes to JEVS_CAM_PLOTS and JEVS_NARRE_PLOTS are correct and follow the suggestion. ✔️

ShelleyMelchior-NOAA · 2024-01-11T16:43:16Z

I tested the changes to jevs_cam_snowfall_plots and few warnings remain, all seemingly for graphics in CONUS_South, when the lowest snowfall thresholds are too high to find any events (similar to the CAPEsfc warnings described above). I think this job is ready to be reviewed here again.

The re-run for jevs_cam_snowfall_plots.sh has completed. As Marcel noted, there are still some WARNING messages, but the ones noted by Wei are no longer present.
logfile: /lfs/h2/emc/ptmp/shelley.melchior/EVS_out/jevs_cam_snowfall_plots.o112803617
COMOUT=/lfs/h2/emc/ptmp/shelley.melchior/evs/v1.0/plots/cam/atmos.20240110

MarcelCaron-NOAA · 2024-01-11T17:40:43Z

@ShelleyMelchior-NOAA Yes, these warning messages should reduce or disappear when we start to see larger snowfall events in the CONUS_South domain. I think the remaining messages are appropriate.

✔️ Otherwise the output file looks good, and I found no other concerning messages in the log file.

MarcelCaron-NOAA · 2024-01-11T20:57:52Z

Hi @ShelleyMelchior-NOAA:

Items 39 and 49 are addressed and the fixes are ready for review. Fixes for items 31 and 50 will be deferred to another PR.

For item 39, please test jevs_cam_headline_plots
For item 49, please test jevs_cam_href_spcoutlook_stats

Both jobs have relatively short runtimes (<30m). Changes made and instructions for running the above jobs have been added to the PR description.

CC: @AliciaBentley-NOAA

ShelleyMelchior-NOAA · 2024-01-12T01:30:30Z

Hi Marcel! I just submitted jevs_cam_href_spcoutlook_stats.sh and jevs_cam_headline_plots.sh.

Both processes are running at the moment.

112844085.cbqs01               shelley.melchi* dev             jevs_cam_href_*    37177    1     4  100gb 01:00 R 00:02
112844227.cbqs01               shelley.melchi* dev             jevs_cam_headl*    45515    1   128    --  00:30 R 00:01

Logfiles:
/lfs/h2/emc/ptmp/shelley.melchior/EVS_out/jevs_cam_href_spcoutlook_stats.o112844085
/lfs/h2/emc/ptmp/shelley.melchior/EVS_out/jevs_cam_headline_plots.o112844227

I'll leave these tests here for our review when we both return to work Friday.

AliciaBentley-NOAA · 2024-01-12T14:10:02Z

@ShelleyMelchior-NOAA @MarcelCaron-NOAA Thanks for your continued work on this PR. Wei would like EVS v1.0.2 by COB today, so I hope that this PR is nearly there and that it can be included before next week's extended dev outage! Please keep up the good work. Fingers crossed.

MarcelCaron-NOAA · 2024-01-12T16:06:18Z

✔️ jevs_cam_headline_plots outputted no warnings and output file looks good
🚩 jevs_cam_href_spcoutlook still outputted one of the warnings listed at item 49.

I need to fix the remaining warning message mentioned in item 49 and will update soon. If the fix tests successfully I think this PR will be ready for final review and merge.

AliciaBentley-NOAA · 2024-01-12T16:26:04Z

Thanks, @MarcelCaron-NOAA! We'll look for your fixes!

MarcelCaron-NOAA · 2024-01-12T19:14:28Z

Update re. jevs_cam_href_spcoutlook_stats:

Deeper dive into the warning message (part of item 49):

WARNING: PointStatVxOpt::process_config() -> The range of requested observation pressure levels is not contained within the range of requested forecast pressure levels.  No vertical interpolation will be performed for observations falling outside the range of forecast levels.  Instead, they will be matched to the single nearest forecast level.

This warning occurs consistently in every METplus process verifying MLCAPE, and impacts cam, mesoscale, and global-det jobs. The relevant met code producing this warning is /apps/ops/prod/libs/intel/19.1.3.304/met/11.0.2/MET-11.0.2/src/tools/core/point_stat/point_stat_conf_info.cc, lines 810-827:

  810    // Check the levels for the forecast and observation fields.  If the
  811    // forecast field is a range of pressure levels, check to see if the
  812    // range of observation field pressure levels is wholly contained in the
  813    // fcst levels.  If not, print a warning message.
  814    if(vx_pd.fcst_info->level().type() == LevelType_Pres &&
  815       !is_eq(vx_pd.fcst_info->level().lower(), vx_pd.fcst_info->level().upper()) &&
  816       (vx_pd.obs_info->level().lower() < vx_pd.fcst_info->level().lower() ||
  817        vx_pd.obs_info->level().upper() > vx_pd.fcst_info->level().upper())) {
  818 
  819       mlog << Warning
  820            << "\nPointStatVxOpt::process_config() -> "
  821            << "The range of requested observation pressure levels "
  822            << "is not contained within the range of requested "
  823            << "forecast pressure levels.  No vertical interpolation "
  824            << "will be performed for observations falling outside "
  825            << "the range of forecast levels.  Instead, they will be "
  826            << "matched to the single nearest forecast level.\n\n";
  827    }

The boolean statements at lines 816 and 817 check if the lowest fcst level is lower than (rather than $\leq$) the lowest obs level, and same for highest levels. Our levels configurations for MLCAPE are:

FCST_VAR2_LEVELS = P90-0
...
OBS_VAR2_LEVELS = L0-90000

Therefore, I think the warning triggers because the obs levels range is equal to the forecast levels range, not contained within it.

I don't know if the levels configuration can be changed to address this, or if it's something to resolve on the METplus side.

However, to progress this PR, since this warning affects several EVS components, and since changing the MLCAPE configuration to resolve a warning message may not be ideal, I suggest leaving the issue in this PR and addressing it separately if necessary.

AliciaBentley-NOAA · 2024-01-12T19:36:08Z

@MarcelCaron-NOAA Thank you for digging into this. We will leave this out of the current PR and either tell Wei we can't remove it because it's METplus or try to resolve it in a future PR. With so many dev outages, it's likely option 1. @ShelleyMelchior-NOAA will work to merge this in it's current form. Thanks!

ShelleyMelchior-NOAA · 2024-01-12T19:41:29Z

Thanks for this thorough digging, Marcel. This makes sense. We can delay addressing what you found and put it into a future PR.
34 ✔️
36 ✔️
39 ✔️
40 ✔️
41 ✔️
42 ✔️
43 ✔️
49 ❗ -- partially addressed; will require a future PR
55 ✔️
57 ✔️

I will conclude the code review and merge this PR after one final sync fork.

ShelleyMelchior-NOAA

All tests yield expected results, except for completing item 49. The remaining WARNINGs for item 49 will be addressed in a future PR.

marcel caron and others added 9 commits December 28, 2023 19:24

Fix allocations to reduce runtime

95be2cb

Rearrange graphics so that SNOD is only computed for hrrr and namnest

ffcc6b1

Reduce number of offsets used to search for prepbufr files

57e53d0

Set OMP_NUM_THREADS=1

472edfe

Remove needless warning message from log

80471a8

Remove needless variable computation from calculate_stat()

4d4c5fd

Allow case where three time-lagged href members are unavailable (e.g.…

952be6d

… F048)

Merge branch 'develop' into bugfix/cam_rev7

f1e5158

Access "MAILTO" only if "SENDMAIL" is set to "YES"

4d2e073

Correct default COMIN path definitions

81a8f6e

marcel caron and others added 3 commits January 9, 2024 20:53

Remove ECME from models list when plotting RH2m

79566c4

Remove "test" log

df9c1f9

Remove snowfall graphics requests for stats that don't exist

7351c0a

Remove hiresws from 06Z/18Z graphics + mute missing data warning when…

8b0aea9

… "group" or "set" in model name

marcel caron added 2 commits January 10, 2024 20:59

mute another missing data warning when "group" or "set" in model name

2cac390

Remove hrrr, hireswarw, and hireswarwmem2 from >F048 graphics

4daae59

Merge branch 'develop' into bugfix/cam_rev7

9c7002a

Corrections to COMOUT definitions.

c716e55

marcel caron added 4 commits January 11, 2024 17:59

Fix missing data file id logic for NAM firewx

6ed0b8c

Fine-tune check files logic

ebcf8b5

Correct MLCAPE levels config + change bilin interp width to 2

36b8107

Remove href products from headline graphics

c28a25b

Merge branch 'develop' into bugfix/cam_rev7

707adbe

ShelleyMelchior-NOAA approved these changes Jan 12, 2024

View reviewed changes

ShelleyMelchior-NOAA merged commit 6d044fc into NOAA-EMC:develop Jan 12, 2024

MarcelCaron-NOAA deleted the bugfix/cam_rev7 branch December 2, 2024 21:37

Bugfix/cam rev7 #405

Bugfix/cam rev7 #405

Conversation

MarcelCaron-NOAA commented Jan 3, 2024 • edited Loading

Pull Request Testing

(1) Set up jobs

(2) Test Jobs

(3) Review

Pull Request Checklist

ShelleyMelchior-NOAA commented Jan 5, 2024

ShelleyMelchior-NOAA commented Jan 5, 2024 • edited Loading

StevenSimon-NOAA commented Jan 5, 2024

ShelleyMelchior-NOAA commented Jan 5, 2024

ShelleyMelchior-NOAA commented Jan 5, 2024

ShelleyMelchior-NOAA commented Jan 5, 2024

ShelleyMelchior-NOAA commented Jan 5, 2024 • edited Loading

ShelleyMelchior-NOAA commented Jan 8, 2024

MarcelCaron-NOAA commented Jan 8, 2024

MarcelCaron-NOAA commented Jan 8, 2024

ShelleyMelchior-NOAA commented Jan 8, 2024 • edited Loading

MarcelCaron-NOAA commented Jan 8, 2024 • edited Loading

ShelleyMelchior-NOAA commented Jan 8, 2024

ShelleyMelchior-NOAA commented Jan 10, 2024

MarcelCaron-NOAA commented Jan 10, 2024

AliciaBentley-NOAA commented Jan 10, 2024

StevenSimon-NOAA commented Jan 10, 2024

MarcelCaron-NOAA commented Jan 10, 2024 • edited Loading

Updates

ShelleyMelchior-NOAA commented Jan 10, 2024

MarcelCaron-NOAA commented Jan 10, 2024 • edited Loading

ShelleyMelchior-NOAA commented Jan 11, 2024

ShelleyMelchior-NOAA commented Jan 11, 2024

ShelleyMelchior-NOAA commented Jan 11, 2024

ShelleyMelchior-NOAA commented Jan 11, 2024

ShelleyMelchior-NOAA commented Jan 11, 2024

MarcelCaron-NOAA commented Jan 11, 2024 • edited Loading

MarcelCaron-NOAA commented Jan 11, 2024 • edited Loading

ShelleyMelchior-NOAA commented Jan 12, 2024

AliciaBentley-NOAA commented Jan 12, 2024

MarcelCaron-NOAA commented Jan 12, 2024

AliciaBentley-NOAA commented Jan 12, 2024

MarcelCaron-NOAA commented Jan 12, 2024

Update re. jevs_cam_href_spcoutlook_stats:

AliciaBentley-NOAA commented Jan 12, 2024

ShelleyMelchior-NOAA commented Jan 12, 2024 • edited Loading

ShelleyMelchior-NOAA left a comment

Choose a reason for hiding this comment

MarcelCaron-NOAA commented Jan 3, 2024 •

edited

Loading

ShelleyMelchior-NOAA commented Jan 5, 2024 •

edited

Loading

ShelleyMelchior-NOAA commented Jan 5, 2024 •

edited

Loading

ShelleyMelchior-NOAA commented Jan 8, 2024 •

edited

Loading

MarcelCaron-NOAA commented Jan 8, 2024 •

edited

Loading

MarcelCaron-NOAA commented Jan 10, 2024 •

edited

Loading

MarcelCaron-NOAA commented Jan 10, 2024 •

edited

Loading

MarcelCaron-NOAA commented Jan 11, 2024 •

edited

Loading

MarcelCaron-NOAA commented Jan 11, 2024 •

edited

Loading

ShelleyMelchior-NOAA commented Jan 12, 2024 •

edited

Loading