Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PyGMT: A Python interface for the Generic Mapping Tools #43

Closed
3 of 9 tasks
weiji14 opened this issue Jul 23, 2021 · 73 comments
Closed
3 of 9 tasks

PyGMT: A Python interface for the Generic Mapping Tools #43

weiji14 opened this issue Jul 23, 2021 · 73 comments
Assignees

Comments

@weiji14
Copy link

weiji14 commented Jul 23, 2021

Submitting Author: Wei Ji Leong (@weiji14)
All current maintainers: (@weiji14, @seisman, @maxrjones, @michaelgrund, @willschlitzer, @yvonnefroehlich)
Package Name: PyGMT
One-Line Description of Package: A Python interface for the Generic Mapping Tools
Repository Link: https://github.com/GenericMappingTools/pygmt
Version submitted:
Editor: @lwasser
Reviewer 1: @jbusecke
Reviewer 2: @SimonMolinsky
Archive: https://zenodo.org/record/6702566
Version accepted: V 0.7.0
Date accepted (month/day/year): 9/1/2022


Description

  • Include a brief paragraph describing what your package does:

    PyGMT is a library for processing geospatial and geophysical data and making publication quality maps and figures. It provides a Pythonic interface for the Generic Mapping Tools (GMT), a command-line program widely used in the Earth Sciences.

Scope

  • Please indicate which category or categories this package falls under:

    • Data retrieval
    • Data extraction
    • Data munging
    • Data deposition
    • Data visualization
    • Reproducibility
    • Geospatial
    • Education
    • Unsure/Other (explain below)
  • Explain how the and why the package falls under these categories (briefly, 1-2 sentences). Please note any areas you are unsure of:

    • Data munging ⚙️: Several data processing modules are available to work on tabular and grid data (https://www.pygmt.org/v0.4.0/api/index.html#data-processing) including functions like blockmedian, surface, grdtrack, etc
    • Data visualization 🗺️: Plotting publication quality maps and figures is a strong point of PyGMT (https://www.pygmt.org/v0.4.0/api/index.html#plotting). There is a comprehensive set of tools for plotting 2D and 3D data, both geospatial and non-geospatial. It is also possible to add standard map elements like a colorbar or legend, and make multi-panel subplot figures.
    • Geospatial 🌐: PyGMT is inherently a geospatial package that can handle both vector and raster data. Several map projections are supported (https://www.pygmt.org/v0.4.0/projections/index.html), and there is support for plotting shapefiles and GeoTiff/NetCDF files.
  • Who is the target audience and what are scientific applications of this package?

    • The primary target audience of PyGMT is the geoscience community, including anybody working on fields like Earth Observation, Geophysics, Oceanography, Seismology, Planetary Sciences, etc. The package offers a way for users to perform general geoprocessing tasks and create high quality illustrations of their results for posters or publications.
  • Are there other Python packages that accomplish the same thing? If so, how does yours differ?

    Yes, there are several geospatial Python packages such as:

    These tools are typically focused on one thing only (e.g. plotting maps, vector data, raster data). PyGMT allows users to mix vector and raster data easily, so that users can:

    • Produce a map with points/lines/polygons and raster grids in the same plot, e.g. plot weather station points on top of a raster Digital Elevation Model, while including map elements (e.g. scalebar, compass arrow, colorbar, legend, inset overview maps etc)
    • Extract a series of elevation values along a transect line (vector) from a NetCDF/GeoTiff grid (raster) using grdtrack
    • Convert a table of xyz points (vector) to a 2D grid (raster) using surface via a spline-based interpolation method.

    PyGMT integrates with the PyData ecosystem. It allows users to process and plot data stored in NumPy arrays, Pandas DataFrames, Xarray DataArray/Datasets, GeoPandas GeoDataFrames, as well as from standard file formats like CSV and GeoTiff/NetCDF files. There are also plans to integrate with other scientific libraries like ObsPy (ObsPy integration GenericMappingTools/pygmt#967) in the future.

    However, PyGMT can only produce static plots unlike packages like Geoviews which allow for interactive plotting (e.g. panning and zooming). It is also non-trivial to scale out data processing tasks and/or plotting of big datasets (>10GB) that don't fit in RAM to multiple processors/clusters as with libraries like dask-geopandas because of limitations in the GMT C package that PyGMT wraps around.

  • Any other questions or issues we should be aware of?:

    First off, thanks for reading this! We're keen to get some independent reviewers to have a look at the PyGMT package, sort of as a first step before going for a software publication at JOSS/G3 (the team is still deciding). Linking to original discussion at What we need for a first PyGMT paper? GenericMappingTools/pygmt#677 (comment), cc @leouieda @seisman @meghanrjones.

P.S. *Have feedback/comments about our review process? Leave a comment here

@lwasser
Copy link
Member

lwasser commented Sep 1, 2021

hey there @weiji14 Welcome to pyopensci and please excuse the delayed response! We will review this submission and will get back to you with next steps soon. I should be able to find some folks in the geospatial space to review.
I can also walk you through the JOSS piece. I'd encourage it because once our review is done, JOSS simply accepts our review as theirs so the work you have to do in order to get a cross-ref enabled citation is small. but of course the decision is yours. happy to chat more if you have questions.

@weiji14
Copy link
Author

weiji14 commented Sep 3, 2021

Hi @lwasser, no worries on the delay. We've been keen to get some good independent reviews on our package, so take your time with the search!

As for the JOSS piece, I'll need to check with the rest of the team on what they prefer. A few of us have actually edited/reviewed JOSS papers, and I've seen and heard many great things about the review process too. We'll definitely have a good think about this once the review gets underway.

@lwasser
Copy link
Member

lwasser commented Oct 18, 2021

hey there @weiji14 and all involved with this package! i started to go through the editor checks here and notice a disclaimer on your readme that says the package is currently undergoing rapid development. pyOpenSci reviews can be compared to JOSS reviews. Ideally the review happens when the tool has achieved some level of development stability. While we may add a process for a second review if you say added a significant chunk of functionality in the future that wasn't on the radar now, this review can be seen similar to a paper review where the paper is not a draft, it's a fully edited paper. Unlike a paper however software evolves and that is ok. but we wouldn't want version .5 here in pyopensci as an archive and then it moves to JOSS for version 1.0. We'd rather review version 1.x and then push it through JOSS as version 1.x given we collaborate with them closely. Can you kindly let me know what your goals are for this review and what stage this tool is in. I'd love it to go through our process but want to ensure it's ready.
Many thanks.

@lwasser
Copy link
Member

lwasser commented Oct 19, 2021

@weiji14 @leouieda we've been discussing this in an issue. I just want to be transparent and careful about when we review packages. The one specific question i have is related to this statement in your readme:

All of the API (functions/classes/interfaces) is subject to change until we reach v1.0.0 as per the semantic versioning specification. There may be non-backward compatible changes as we experiment with new design ideas and implement new features. This is not a finished product, use with caution.

do you have any planned significant API changes at this point? Or do you feel like the package is relatively stable and ready for a solid review which may help you get to 1.x. many thanks! Once we have this resolved I will kick start the review. I appreciate your patience.

@lwasser lwasser self-assigned this Oct 19, 2021
@weiji14
Copy link
Author

weiji14 commented Oct 19, 2021

Hi @lwasser, thank you opening the discussion at pyOpenSci/software-peer-review#101. The goals for the PyGMT team at this stage are really to see if there are any major issues in terms of our API design, and to ensure that we are following established best practices in software design. Essentially, have a fresh set of eyes look at our documentation and see if there are any glaring issues we may have missed.

All of the API (functions/classes/interfaces) is subject to change until we reach v1.0.0 as per the semantic versioning specification. There may be non-backward compatible changes as we experiment with new design ideas and implement new features. This is not a finished product, use with caution.

do you have any planned significant API changes at this point? Or do you feel like the package is relatively stable and ready for a solid review which may help you get to 1.x. many thanks! Once we have this resolved I will kick start the review. I appreciate your patience.

To be honest, that disclaimer has been sitting there for a year and a half now since Apr 2020 or v0.1.0. We're currently working on a v0.5.0 release for next week (29 Oct 2021), which included a lot of work on API standardization (see e.g. GenericMappingTools/pygmt#1479), so that disclaimer could probably be toned down a little. I'll make a point to update this disclaimer for the next release.

In terms of making backward incompatible changes, we actually have a deprecation policy for a while now since Apr 2021 or v0.4.0 (see GenericMappingTools/pygmt#1160). This ensures that users are given time (typically 2 minor releases or about 6 months) and fair warning when things might break in the future. We've actually just reached 400 stars on GitHub this week, and judging from activity on our forum, we're increasingly aware that there's a significant userbase and things are definitely getting to a stage where we have to be very careful with breaking other people's code.

So in short, I do think PyGMT is stable enough for review, and that going through the pyOpenSci review process will help us get to v1.x.x in a structured manner. Let me know if you have any other concerns, and I can discuss this with the rest of the PyGMT team 😃

@lwasser
Copy link
Member

lwasser commented Oct 19, 2021

hi @weiji14 ! thank you for the thoughtful response. I will move forward with the review! When we get reviews it also allows us to reflect on our process and guidelines and so this review will help us adjust our language around "under development" packages (which all packages are technically always iteratively improving). Do you have a suggestion for a reviewer?

I will also ping the community. We often select one reviewer that is suggested and another will be selected by us. If you don't have any suggestions then I will look for two. Again thank you for your patience!

@lwasser
Copy link
Member

lwasser commented Oct 19, 2021

Editor checks:

  • Fit: The package meets criteria for fit and overlap.
  • Automated tests: Package has a testing suite and is tested using CI (e.g. GitHub Actions, Travis-CI, CircleCI, etc).
    • Github actions
  • Documentation: Package has documentation setup (ReadTheDocs, JupyterBook, website,etc.).
  • License: The package has an OSI accepted license
    • BSD 3-Clause "New" or "Revised" License
  • Repository: The repository link resolves correctly
  • Archive Zenodo. (may be post-review): The repository is archived somewhere outside of GitHub. We suggest using Zenodo and created releases for this purpose.
  • Version (may be post-review) Does the release version given match the GitHub release (v1.0.0)?

Due Date: Dec 27 2021(this is a holiday week in the US)
Reviewers: @SimonMolinsky @jbusecke
More to come when we find reviewers.

@weiji14
Copy link
Author

weiji14 commented Oct 19, 2021

Thanks @lwasser, I don't have a specific reviewer in mind for now, but perhaps someone involved in the Pangeo community would be a good fit? Also if possible, it would be better to start the review process after PyGMT releases v0.5.0 next Friday to capture some of the API standardization changes we've made recently. I suppose it might take time for you to find a reviewer or two, but starting the review sometime early Nov 2021 would be good.

@lwasser
Copy link
Member

lwasser commented Oct 19, 2021

ok @weiji14 let's shoot for early november. i can work on finding reviewers and let them know that we will wait until that release / early november to begin. I also can reach out to the pangeo community. All sounds good!

@jbusecke
Copy link

I would be happy to review this after I am back from vacation (mid-late november). Please let me know if that fits into your timeline.

@lwasser
Copy link
Member

lwasser commented Oct 27, 2021

Thanks @jbusecke !! that is great.
hi there @weiji14 would a slightly later timeline work for you if we started the review in say mid november but provided @jbusecke with a bit of additional time if they are not ready until later in the month? i have one other reviewer that sounds game as well who i will ping shortly.

@SimonMolinsky
Copy link
Collaborator

Hi @weiji14 , Hi @lwasser ,

I'm available as a reviewer and I may start as soon as it will be possible :)

@weiji14
Copy link
Author

weiji14 commented Oct 27, 2021

Thanks @jbusecke !! that is great. hi there @weiji14 would a slightly later timeline work for you if we started the review in say mid november but provided @jbusecke with a bit of additional time if they are not ready until later in the month? i have one other reviewer that sounds game as well who i will ping shortly.

Cool, thanks for offering to review @jbusecke! Mid to late November sounds good, we're not in a hurry 😀

@lwasser
Copy link
Member

lwasser commented Oct 27, 2021

ok wonderful. it looks like we have two reviewers @jbusecke @SimonMolinsky thank you both for volunteering! How does Monday November 15 sound for a start date for this review? The review would then be due on December 6. Does that work for everyone? i can remind everyone in ~two weeks as well about the review period beginning and will add links to our review template as well.

@jbusecke
Copy link

Sounds good to me.

@SimonMolinsky
Copy link
Collaborator

Hi, I'm fine with this timeline :)

@lwasser
Copy link
Member

lwasser commented Oct 28, 2021

Awesome. Thank you, All! i'll check back in on the 15th with specific next steps.

@SimonMolinsky
Copy link
Collaborator

@lwasser, what are the next steps? Now I have a window to review the package. I assume that December will be rather busy so it's better to start now :)

@lwasser
Copy link
Member

lwasser commented Nov 24, 2021

absolutely. @SimonMolinsky thanks for the ping. For some it is a holiday week so I was assuming that may impact the review but let's do this. Simon and @jbusecke thank you for being willing to review. let's set the deadline for 4 weeks which is one week longer than usual. i am doing this because of the holiday week, the christmas holiday for some and AGU for some to ensure everyone has time. Let's set the deadline for December 27th. However if you have time to do it now, please work on it and get it in when you can. that week of the 27th is also a holiday for us so we can be flexible with this review.

feel free to submit feedback as issues (or pr's) to pygmt if you wish and reference this issue here. This has worked for other reviews . @weiji14 are you comfortable with issues from reviewers? When you finish the review, use the review template to submit here . please get in touch here with any questions as you proceed.

I will check back in around the 27th but please know that is a holiday week where I work.

Thank you all!

@weiji14
Copy link
Author

weiji14 commented Nov 24, 2021

Yep, we're ok with people making issues on the repo. The 4 week timing sounds good too.

@jbusecke
Copy link

jbusecke commented Dec 3, 2021

Thanks for the update. I will try to work on this in the coming week, but appreciate the longer deadline just in case 😁.

@lwasser
Copy link
Member

lwasser commented Sep 28, 2022

hey there @weiji14 @jbusecke @SimonMolinsky 👋 Just a friendly reminder to take 5-10 minutes to fill out our survey . We really appreciate it. Thank you in advance for helping us by filling out the survey!! 🙌

Wei, it's really important for us to collect information from our maintainers so that we can both stay in touch with you regarding package maintenance and also support you through time. We really appreciate your time in filling this out. I know that you have a team working on pyGMT. Please have your co-maintainers also fill it out and please list them here as well. 🙌 Many thanks in advance! 🙌

🔗 HERE IS THE SURVEY LINK 🔗

@SimonMolinsky
Copy link
Collaborator

hey there @weiji14 @jbusecke @SimonMolinsky 👋 Just a friendly reminder to take 5-10 minutes to fill out our survey . We really appreciate it. Thank you in advance for helping us by filling out the survey!! 🙌

Wei, it's really important for us to collect information from our maintainers so that we can both stay in touch with you regarding package maintenance and also support you through time. We really appreciate your time in filling this out. I know that you have a team working on pyGMT. Please have your co-maintainers also fill it out and please list them here as well. 🙌 Many thanks in advance! 🙌

🔗 HERE IS THE SURVEY LINK 🔗

Done! 😊

@weiji14
Copy link
Author

weiji14 commented Oct 2, 2022

Cool, thanks for the reminder Leah, just completed the survey!

@michaelgrund
Copy link

Done!

@jbusecke
Copy link

jbusecke commented Oct 6, 2022

Done

@lwasser
Copy link
Member

lwasser commented Oct 6, 2022

Thank you so so much @weiji14 @michaelgrund @jbusecke !!! 🎉 🎉 So appreciative of your time.

@lwasser
Copy link
Member

lwasser commented Jun 12, 2023

hey y'all - i am going to close this issue. i suspect the JOSS submission after our review might now happen at this point. but if you'd like it to please reach out and we can reopen!! Hope you are all well!!

@lwasser lwasser closed this as completed Jun 12, 2023
@juanis2112
Copy link

Hi @weiji14, I’m currently working for @lwasser on community outreach for pyOpenSci, and as part of this I’m doing some social media posts to promote the packages accepted through our peer review process like this one: https://twitter.com/pyOpenSci/status/1666155502877671461?s=20

I wanted to ask you about PyGMT's logo. I didn’t find a logo in the package’s GitHub repo so I was wondering if you have one somewhere or are planning to work on one.

For packages that don’t have a logo I was thinking about writing the name of the package in a nice font/color. I will like to know if you have any ideas of colors/fonts you would like to use since I’ll like to make sure the people involved in the package like the way it it is promoted. I'm happy to send you some ideas and work on this with you while you have your own logo :).

@weiji14
Copy link
Author

weiji14 commented Jun 14, 2023

Hi @juanis2112, thanks for reaching out! A PyGMT logo is something we've been meaning to make for years (see https://forum.generic-mapping-tools.org/t/design-for-pygmt-logo/1346), but there's no official version yet. If you want to start a new thread somewhere, we can discuss this in more detail 😄

@juanis2112
Copy link

juanis2112 commented Jun 16, 2023

I opened an issue on PyGMT's repo GenericMappingTools/pygmt#2579 to start a discussion with the ideas people have posted and maybe do some voting to decide which direction to explore more.

@lwasser lwasser moved this to pyos-accepted in peer-review-status Jul 11, 2023
@lwasser
Copy link
Member

lwasser commented Aug 18, 2023

hi there @weiji14 / colleagues 👋 we are doing a bit of housekeeping. i have the names of all maintainers (i think) for this package but am missing GitHub usernames. Could someone please reply here and provide the GitHub handles for everyone? We want to credit all maintainers on our website! and to do that i need to add the GH handles to this issue metadata at the top! Many thanks in advance for doing this!

@weiji14
Copy link
Author

weiji14 commented Aug 24, 2023

Hi @lwasser, sorry for the late reply! Our list of maintainers are listed at https://www.pygmt.org/v0.9.0/team.html:

  • @seisman
  • @maxrjones
  • @michaelgrund
  • @weiji14
  • @willschlitzer

Could you post the link on the website when this is updated? Thanks!

@lwasser
Copy link
Member

lwasser commented Sep 11, 2023

@weiji14 no worries! thank you for replying! i just wanted to be sure we are recognizing all of the maintainers on our package listing page. i've added everyone to this issue so the data will get updated in a few weeks (i have a bug in our code to fix and then it can be updated!!). i hope all is well!!

once i perform that update (i could do it now if that bug didn't exist) i'll let you know and add a screen shot here! but i did update the issue which the code parses so everyone gets recognized!!

@lwasser
Copy link
Member

lwasser commented Sep 11, 2023

Screen Shot 2023-09-11 at 4 09 46 PM

@weiji14 i just updated the website!! all maintainers are now credited on the pygmt package page!! And they should be listed on our contributor page too!!

in the future you will also be able to filter by maintainer on our contributor page - that is a feature i plan to add!

thank you for helping me flesh out the maintainer list here. we really want to make sure all of the authors / maintainers get credit for the great work done on these tools!

@weiji14
Copy link
Author

weiji14 commented Sep 11, 2023

Thanks @lwasser, looks great! Would it be possible to update the list to add @yvonnefroehlich who recently got added as a maintainer? If you point me to the right page, we can update the list with a PR.

@lwasser
Copy link
Member

lwasser commented Sep 12, 2023

@weiji14 absolutely. you can actually update this issue here if you wish at the top (i think because you opened it you can update it - if you can't let me know).

I have a ci action that parses these reviews and grabs maintainers. we've been updating all of the issues. it runs on a cron job but we can also trigger it to update things at any time.

Screen Shot 2023-09-12 at 4 49 55 PM

@weiji14
Copy link
Author

weiji14 commented Sep 13, 2023

Cool, I've updated the maintainer list above, and see that the cron-job runs on the 1st and 15th of each month, which is tomorrow, so should be ok to wait for a bit. Nice work on automating all of this!

@lwasser
Copy link
Member

lwasser commented Sep 13, 2023

Yes! you got it! so it should open a pr and then everything will get updated.

i forget if we already talked about this - but do you (or anyone else involved in this review including other maintainers!!) have any interest in joining our slack? if so i can send invites to anyone who's interested.

@weiji14
Copy link
Author

weiji14 commented Sep 14, 2023

I'll personally pass for the Slack channel (have more than a dozen channels already...), but maybe someone else would be interested.

@jbusecke
Copy link

I am in the same boat, but please feel free to ping me for future reviews on github!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: pyos-accepted
Development

No branches or pull requests

7 participants