Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE REQUEST] Output satellite diagnostic as NetCDF #770

Closed
eamarais opened this issue Jul 1, 2021 · 21 comments
Closed

[FEATURE REQUEST] Output satellite diagnostic as NetCDF #770

eamarais opened this issue Jul 1, 2021 · 21 comments
Assignees
Labels
category: Feature Request New feature or request topic: Diagnostics Related to output diagnostic data
Milestone

Comments

@eamarais
Copy link

eamarais commented Jul 1, 2021

I've updated GEOS-Chem version 13.0.2 to output the satellite diagnostics (ND51) in NetCDF format by piggybacking on existing code infrastructure in diagnostics_mod.F90, history_mod.F90, history_netcdf_mod.F90, and state_diag_mod.F90. Im finding though that there are small differences between output from my updated code and output from ND51.

I've attached images I've generated that compare the old (ND51) and new (NetCDF) output on 20180102 during 12h00-15h00 local time in the lowest model layer for CH2O, relatively humidity (RH), and the number of data points. The latter was a sanity check to ensure that accumulation of model values was the same in the two diagnostics. It is. Is it possible the difference is due to differences in the timing of accumulating/updating the ND51 versus the History diagnostic routines?

GC-satellite-diagnostics-comparison

@eamarais eamarais added the category: Feature Request New feature or request label Jul 1, 2021
@yantosca
Copy link
Contributor

yantosca commented Jul 1, 2021

Thanks @eamarais for doing this. How did you get the local time diagnostics? In the past I would try to keep a counter field and then divide by that in post-processing. I tried doing that for the JValueLocalNoon collection a couple of years back but there were always artifacts that I couldn't remove. If I can find them I'll post a couple of PPTs (but they may be on my work PC back in the office so it may be a while before I get them.)

@yantosca yantosca self-assigned this Jul 1, 2021
@eamarais
Copy link
Author

eamarais commented Jul 1, 2021

@yantosca I can share the code updates as a patch if you'd like. I essentially created a new hour range (hrrange) entry in HISTORY.rc, wrote code in diagnostics_mod.F90 to identify when the relevant fields coincided with the local time range of interest using GET_LOCALTIME and included an IF statement in history_netcdf_mod.F90 to divide the satellite diagnostic arrays by a 3D or 2D array depending on the size of the item. It's probably not eloquent coding, but I think it works (though I'd feel more confident about the code if I could resolve the difference between my updated diagnostic and ND51).

@yantosca
Copy link
Contributor

yantosca commented Jul 1, 2021

Thanks @eamarais. The only thing is that we in the past have avoided bringing in any of the GEOS-Chem Classic specific time routines in order to make the HISTORY diagnostics usable by GCHP, GEOS, WRF-GC, etc.

@yantosca
Copy link
Contributor

yantosca commented Jul 1, 2021

Here are a couple of presentations I made to our Model Development Group from 2018 about trying to implement the local noon J-value diagnostics in HISTORY (which is similar to the satellite diagnostic problem):

  1. MDS 2018-01-05 Yantosca JValueDiags.pdf

  2. MDS_23Jan2018_BMY.pdf

@eamarais
Copy link
Author

eamarais commented Jul 1, 2021

Thanks @yantosca for the slides. When you say "GEOS-Chem Classic specific time routines", do you mean anything in time_mod.F90? I ask because there are routines from time_mod.F90 that are already used in a few HISTORY routines. Or is there something specific about GET_LOCALTIME that would make it incompatible with GCHP etc.?

@yantosca
Copy link
Contributor

yantosca commented Jul 1, 2021

@eamarais, I think I mean the local time. Just that GET_LOCALTIME isn't available in e.g. GCHP or GEOS since that uses the MAPL time routines.

@yantosca
Copy link
Contributor

yantosca commented Jul 1, 2021

@eamarais, I just took a quick look at the History code. The only time routines used are the ones in julday_mod.F90 and the routine YMD_EXTRACT, which just peels out Y, M, D from YYYYMMDD. History doesn't use any of the time computing functions.

@eamarais
Copy link
Author

eamarais commented Jul 1, 2021

@yantosca would it help if I rewrote get_localtime in the History code?

@yantosca
Copy link
Contributor

yantosca commented Jul 1, 2021

I think it's a bit more complicated than that. The GCClassic get_localtime function assumes a Cartesian grid. But if you go to other models like WRF-GC that might not be the case. Then you'd have to also bring in awareness of where you are in the grid and it gets hairier from there.

Also tagging @sdeastham @LiamBindle who have had some experience w/ 1-D diagnostics.

@sdeastham
Copy link
Contributor

Given that what we want is really a sort of local solar time (as that's what dictates whether or not a satellite is overhead), would it be safer to ask for the UTC time and then just subtract longitude (in deg)/15? That data should be available from every wrapper. Sorry if I'm totally missing the point here!

@yantosca
Copy link
Contributor

yantosca commented Jul 1, 2021

Thanks @sdeastham. I think it goes to whether or not we want to bring time & grid awareness into the HISTORY diagnostics, or to handle that in post. Our philosophy had been to do it in post, I think.

@eamarais
Copy link
Author

eamarais commented Jul 1, 2021

@yantosca I only use get_localtime in GeosCore/diagnostics_mod.F90 in the update. I read the local time values defined my the user in history_mod.F90

@yantosca
Copy link
Contributor

yantosca commented Jul 1, 2021

@eamarais Oh OK. Then that would be OK if its outside of History.

@sdeastham
Copy link
Contributor

sdeastham commented Jul 1, 2021

Will this approach work in GCHP, do you know (by which I mean - will this diagnostic be accessible from GCHP)? We were discussing this exact problem (satellite diagnostics) with the MAPL team just Friday.

@eamarais
Copy link
Author

eamarais commented Jul 1, 2021

@sdeastham, I can't answer that, as I'm not familiar with the different approaches of the two models that might cause issues. I can share the code as a patch if you'd like to see whether it would work.

@sdeastham
Copy link
Contributor

@eamarais that sounds great! Unless @yantosca you happen to know offhand if this would/would not require modification?

@yantosca
Copy link
Contributor

yantosca commented Jul 1, 2021

I would say send us the patch @eamarais and we can take a look. (But probably not at least till next week after the holiday here.)

@eamarais
Copy link
Author

eamarais commented Jul 1, 2021

@sdeastham, @yantosca, I'll share the code as a patch (give me a few minutes to figure out how to do this...). I've uploaded the HISTORY.rc file with the satellite output (SatDiagn) collection. The file is uploaded as a text file to accommodate limited file format options. The SatDiagn diagnostics aren't a complete list of what's in ND51. For example, it doesn't include AOD, but, if my modifications are appropriate for general use across all models, the framework is there for others to add more entries.

HISTORY_rc_with_SatDiagn.txt
.

@eamarais
Copy link
Author

eamarais commented Jul 1, 2021

@yantosca, @sdeastham, patch attached. The parent commit is in the attached file. Again, converted to text file so that it's compatible. Odd oversight that Git doesn't support this file format, unless I'm missing something.

The changes to diag51_mod.F90 aren't relevant. I was playing around with array types to try resolve the differences in the plot I shared earlier.

netcdf-sat-diagn-diff.txt

@eamarais
Copy link
Author

eamarais commented Jul 1, 2021

@yantosca, @sdeastham, I found the cause for the difference between the NetCDF and ND51 diagnostics. It's a minor update. I was subtracting the dynamic timestep from the local time, as in ND51, but this isn't necessary for the NetCDF satellite diagnostic, presumably because of when it's called in the code. Updated patch attached. This merges both my commits.

netcdf-sat-diagn-v2-diff.txt

@yantosca yantosca added the topic: Diagnostics Related to output diagnostic data label Jul 9, 2021
@lizziel lizziel added this to the 13.4.0 milestone Nov 17, 2021
@yantosca yantosca modified the milestones: 13.4.0, 14.0.0 Jan 24, 2022
@msulprizio msulprizio linked a pull request May 26, 2022 that will close this issue
@msulprizio msulprizio modified the milestones: 14.0.0, 14.1.0 May 26, 2022
@msulprizio
Copy link
Contributor

The pull request for this feature has now been merged into dev/14.1.0 and will be included in that version release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: Feature Request New feature or request topic: Diagnostics Related to output diagnostic data
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants