Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/ETC_composites #153

Open
wants to merge 135 commits into
base: main
Choose a base branch
from
Open

Feature/ETC_composites #153

wants to merge 135 commits into from

Conversation

jeyavinoth
Copy link
Contributor

This is a preliminary version of the POD that tracks extra tropical cyclones and computes cyclone centered composites.

Summary:
The Extratropical cyclones are identified with the Modeling, Analysis and Prediction (MAP) Climatology of Midlatitude Storminess (MCMS) algorithm (Bauer et al. 2016) which uses 6-hourly gridded sea level pressure fields to locate storm centers and then track them through the cyclone lifetime. This algorithm has historically been used to create a database of cyclone locations using the ERA-Interim reanalysis (Dee et al. 2011). The MCMS algorithm is also applied to the model and re-analysis sea level pressure fields. We generate cyclone centered composites of cloud cover, as well as other atmospheric variables, over multiple instances of the same type of atmospheric phenomenon to compare the models with observations or reanalysis. The is a plan view as a passive instrument would observe. This method presents a great advantage for model evaluation as they allow multiple cases to be included and do not necessitate a match in time and space between the free running models and observations.

@jeyavinoth
added feature/eulerian-storm-track by Jeyavinoth Jeyaratnam
dc247e4
@jeyavinoth
merged MDTF develop branch for my eulerian storm tracker
a79ad0a
@jeyavinoth
ready to work on eulerian-storm-tracker
cd20158
@jeyavinoth
changing settings.json for my eulerian-strom-track
d4c8403
@jeyavinoth
copying over new code
7a1d6d7
@jeyavinoth
working version of eulerian_storm_tracker; needs more finer changes
1a73831
@jeyavinoth
adding wheeler kiladis to my test run; index.html has a bug when only…
dfa152c 
… a single POD is run
@jeyavinoth
Merge branch 'feature/eulerian-storm-track' into develop
8edcffb
@jeyavinoth
upstream update of develop
4236606
@jeyavinoth
Merge remote-tracking branch 'upstream/develop' into develop
b53a427
@jeyavinoth
trying out eulerian_storm_track
7e54769
@jeyavinoth
eulerian storm track works, but now have to filter out the topography…
ac27821 
… and create zonal means
@jeyavinoth
create topo file for obs data
3c47b35
@jeyavinoth
added zonal means
5a3281f
@jeyavinoth
added labels
c6acb99
@jeyavinoth
cleaned the eulerian_storm_track.yml file
fe98178
@jeyavinoth
copied over the tracker info the framework
4c6c392
@jeyavinoth
started working on the etc-composites
0d463d5
@jeyavinoth
updated the pdf
c435373
@jeyavinoth
removed old front_detection codde
5b6e81e
@jeyavinoth
removed old front_detection codde
e5ff616
@jeyavinoth
manually editted the yml env file
225386b
@jeyavinoth
updated the documentation
ef59c09
@jeyavinoth
created yml file for etc_composites
ac20eeb
@jeyavinoth
added back the etc_composites stuff
ddd5761
@jeyavinoth
the center finder runs fine, but crashes on track finder cuz of year …
5de80bf 
…in directory name
@jeyavinoth
working version of the tracker part; it creates the .mat file
3bd376d
@jeyavinoth
etc_composites done up to track_stats
3f313f5
@jeyavinoth
created the composites images; have to add it to the html
0b70d37
@jeyavinoth
removed eulerian_storm_track from git repo
083d6f0
wrongkindofdoctor and others added 5 commits December 16, 2021 13:33
@wrongkindofdoctor
Add pods to ci (NOAA-GFDL#301)
0d24321 
* added checks that plotted values fall within contour levels to prevent synthetic data nans from causing failure
and fixed the figure output path definition in ocn_surf_flux_diag.py

* added mixed layer depth and ocn surf flux matrix pods to test set 3 yamls

* removed the mdtf_test_data version spec from mdtf_tests.yml

* added temporary test workflow file for new pods

* fix name of ocn_surf_flux_diag tarball in test yaml

* convert start and end dates to iso format in mixed_layer_depth.py so that pod works with full range of year integer values

* added test lines to dump detailed output for mld pod to screen for debugging

* turned on experimental setting

* replace cat with printf

* add updated test files

* add continue on error to step, only build required environments

* move log dump to separate step

* fix test workflow file

* add esmpy to python3_base.yml and update xesmf version

* specify pip to install xesmf

* change package order

* update netcdf4, esmpy versions in python3_env yaml
add esmpy 8.2 to python3_env yaml to resolve package dependency issues

* update env_synthetic_data yaml package versions to match python3_base
add ocn_surf_flux_diag test back to test_tpsl to double check env changes
comment out mld log dump step

* add ocn_surf_flux_diag and mld PODs to mdtf_tests.yml
remove test_tpsl from remote repo
uncomment other set3 PODs in test jsonc files
@wrongkindofdoctor
Merge branch 'main' into develop
858303d
@jeyavinoth
Merge remote-tracking branch 'upstream/develop' into feature/ETC-comp…
ce68aef 
…osites
@jeyavinoth
Merge remote-tracking branch 'upstream/main' into feature/ETC-composites
4e36b5b
@jeyavinoth
Merge branch 'NOAA-GFDL:main' into feature/ETC-composites
b41168c
@lgtm-com
Copy link
Contributor

lgtm-com bot commented Dec 18, 2021

This pull request introduces 628 alerts when merging b41168c into 858303d - view on LGTM.com

new alerts:

  • 229 for Unused local variable
  • 121 for Unused import
  • 84 for Variable defined multiple times
  • 66 for Except block handles 'BaseException'
  • 54 for Unreachable code
  • 40 for Testing equality to None
  • 5 for Module is imported more than once
  • 5 for Suspicious unused loop iteration variable
  • 4 for Non-iterable used in for loop
  • 3 for Non-callable called
  • 3 for Syntax error
  • 3 for Nested loops with same variable
  • 2 for Redundant comparison
  • 2 for Wrong number of arguments for format
  • 2 for Wrong number of arguments in a call
  • 1 for Signature mismatch in overriding method
  • 1 for Unnecessary pass
  • 1 for Missing call to `__init__` during object initialization
  • 1 for Import of deprecated module
  • 1 for Modification of parameter with default

@wrongkindofdoctor wrongkindofdoctor linked an issue Dec 20, 2021 that may be closed by this pull request
Add variables to fieldlist_GFDL.jsonc to support ETC_composites POD #305
Closed
jeyavinoth and others added 8 commits March 1, 2022 05:46
@jeyavinoth
Merge remote-tracking branch 'upstream/main' into feature/ETC-composites
1e7620c 
Merged with main
@wrongkindofdoctor
Fix Explicit_file CLI definition
0443db3
@wrongkindofdoctor
Update the CM4 diag table
7eb04ad 
Add atmos variables to the diag table
Note that this currently generates variables required for the MJO_suite POD using GFDL standard output
more variables may be required for the other PODs. It is also preferable to output in CMIP6 format
@wrongkindofdoctor
remove --disable-epsv option from curl calls
9da4905
@wrongkindofdoctor
Merge branch 'main' into develop
748d6da
@jeyavinoth
updated POD to get it working with the new varlist
87316ea
@jeyavinoth
Merge branch 'feature/ETC-composites' of github.com:jeyavinoth/MDTF-d…
d0f8831 
…iagnostics into feature/ETC-composites
@jeyavinoth
Merge remote-tracking branch 'upstream/develop' into feature/ETC-comp…
5cbaaf3 
…osites

merged with develop branch
@lgtm-com
Copy link
Contributor

lgtm-com bot commented Mar 16, 2022

This pull request introduces 628 alerts when merging 5cbaaf3 into 748d6da - view on LGTM.com

new alerts:

  • 229 for Unused local variable
  • 121 for Unused import
  • 84 for Variable defined multiple times
  • 66 for Except block handles 'BaseException'
  • 54 for Unreachable code
  • 40 for Testing equality to None
  • 5 for Module is imported more than once
  • 5 for Suspicious unused loop iteration variable
  • 4 for Non-iterable used in for loop
  • 3 for Non-callable called
  • 3 for Syntax error
  • 3 for Nested loops with same variable
  • 2 for Redundant comparison
  • 2 for Wrong number of arguments for format
  • 2 for Wrong number of arguments in a call
  • 1 for Signature mismatch in overriding method
  • 1 for Unnecessary pass
  • 1 for Missing call to `__init__` during object initialization
  • 1 for Import of deprecated module
  • 1 for Modification of parameter with default

@lgtm-com
Copy link
Contributor

lgtm-com bot commented Mar 16, 2022

This pull request introduces 132 alerts when merging 5718729 into 748d6da - view on LGTM.com

new alerts:

  • 57 for Unused local variable
  • 23 for Unreachable code
  • 18 for Variable defined multiple times
  • 14 for Unused import
  • 11 for Except block handles 'BaseException'
  • 3 for Module is imported more than once
  • 2 for Nested loops with same variable
  • 1 for Signature mismatch in overriding method
  • 1 for Redundant comparison
  • 1 for Missing call to `__init__` during object initialization
  • 1 for Wrong number of arguments for format

@lgtm-com
Copy link
Contributor

lgtm-com bot commented Mar 16, 2022

This pull request introduces 107 alerts when merging 8697263 into 748d6da - view on LGTM.com

new alerts:

  • 48 for Unused local variable
  • 21 for Unreachable code
  • 11 for Except block handles 'BaseException'
  • 11 for Variable defined multiple times
  • 7 for Unused import
  • 3 for Module is imported more than once
  • 2 for Nested loops with same variable
  • 1 for Signature mismatch in overriding method
  • 1 for Redundant comparison
  • 1 for Missing call to `__init__` during object initialization
  • 1 for Wrong number of arguments for format

@lgtm-com
Copy link
Contributor

lgtm-com bot commented Mar 20, 2022

This pull request introduces 85 alerts when merging 6d5460c into 748d6da - view on LGTM.com

new alerts:

  • 39 for Unused local variable
  • 20 for Unreachable code
  • 11 for Except block handles 'BaseException'
  • 3 for Module is imported more than once
  • 3 for Unused import
  • 3 for Variable defined multiple times
  • 2 for Nested loops with same variable
  • 1 for Signature mismatch in overriding method
  • 1 for Redundant comparison
  • 1 for Missing call to `__init__` during object initialization
  • 1 for Wrong number of arguments for format

@lgtm-com
Copy link
Contributor

lgtm-com bot commented Mar 30, 2022

This pull request introduces 74 alerts when merging 7f9732c into 748d6da - view on LGTM.com

new alerts:

  • 33 for Unused local variable
  • 20 for Unreachable code
  • 6 for Variable defined multiple times
  • 5 for Except block handles 'BaseException'
  • 3 for Unused import
  • 2 for Nested loops with same variable
  • 1 for Signature mismatch in overriding method
  • 1 for Redundant comparison
  • 1 for Module is imported more than once
  • 1 for Missing call to `__init__` during object initialization
  • 1 for Wrong number of arguments for format

@wrongkindofdoctor wrongkindofdoctor changed the base branch from develop to main June 17, 2022 14:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jkrasting jkrasting jkrasting left review comments

@aradhakrishnanGFDL aradhakrishnanGFDL Awaiting requested review from aradhakrishnanGFDL aradhakrishnanGFDL is a code owner

@wrongkindofdoctor wrongkindofdoctor Awaiting requested review from wrongkindofdoctor wrongkindofdoctor is a code owner

Assignees

@wrongkindofdoctor wrongkindofdoctor

Labels
diagnostic Issue pertains to a contributed diagnostic
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add variables to fieldlist_GFDL.jsonc to support ETC_composites POD
6 participants
@jeyavinoth @wrongkindofdoctor @jkrasting @tsjackson-noaa @jhafner2 @chiaweh2