-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rewrite the release workflow and refactor existing GHA workflows #40
Conversation
@sjlongland Feel free to take a look at this if you want. It's theoretically done (or, done enough for now), but given how complex these changes are, I'm going to want to wait a day or two and look over it with fresh eyes before I'm ready for it to be merged. |
I want to force all releases to go through Github for traceability, so I'm removing the release environment from tox.ini to make it difficult for someone to create a release locally. And since the existing release job in Github Actions uses the tox environment, I'm removing that job as well. In an upcoming commit I'm going to add a brand new GHA workflow to produce a release.
In this commit I'm breaking out the parts of the main GHA workflow that build the documentation and run tests. That will allow us to use the reusable docs and test workflows as part of the release process, which I'm going to add in the next commit.
This commit adds a completely new Github Actions workflow for handling a release. Or technically, we have to make a release in the Github web interface (or by using the Github API); the workflow just handles everything that needs to happen after that, namely building and testing the package and uploading it to (test and regular) PyPI. The flow is thus: - create a tag and Github release (manual) - test the code from the main Git branch - test the documentation from the main Git branch - build the package for distribution - publish to Test PyPI - publish to PyPI I'm not sure if that's the best way to sequence the steps involved in making a release, though. For one thing, the tests are not run against the actual distributable package, so there's a slight chance that a checkout of the repo would pass tests while the same tests run against the installable wheel would fail. (This is very unlikely, but it could happen if the package uses a hard-coded relative file or something like that.) This sequencing also carries the risk that the package might fail tests *after* creating the tag, and then we're faced with having a tagged release that doesn't work and can't be posted to PyPI. There are some alternative sequences we could consider, such as: - create a tag and Github release (manual) - build the package for distribution - test the code from the package - test the documentation from the main Git branch - publish to Test PyPI - publish to PyPI This would take care of the risk that tests pass against the checked-out source code but fail against the installed package. But it turns out to be somewhat tricky to set up tox to run tests against the installed package (although doable with some hacky workarounds in our case, at least for now). Or we could do this: - test the code from the main Git branch - test the documentation from the main Git branch - create the tag - build the package for distribution - publish to Test PyPI - publish to PyPI - create the release (semi-manual) That would take care of the risk of the tests failing when we've already created a tag, but it still has the problem where what we're testing is not exactly the same as what gets installed. And it's also awkward to have the semi-manual step of creating the Github release at the end, which basically needs Github Actions to prompt us to create a release but wait for us to actually fill in the release description. (Of course we could probably get away with having the release be created automatically and then just manually editing the release notes afterwards, but it would be nice to have the release notes prepared in advance.) I think the ideal sequence would be this: - draft the release including the release notes and the name of the tag to be created, but don't make it public yet (manual) - check out the code in a local view - create the tag in that local view - build the package for distribution - test the code from the package - test the documentation in the local view - push the tag from the local view to the Github repo - publish to Test PyPI - publish to PyPI - publish the previously drafted release But it's fairly complicated to make all those steps happen in exactly that order, as well as to arrange for tests to be run against the built package. I decided to leave that as a future goal, and for now just commit something that we can use to make a release so that we're not blocked in making this package available to the public.
49e1277
to
3961149
Compare
I think it's ready now 👍 (unless I missed something) We won't be able to test the release workflow without actually creating a release, but it could be a pre-release that never gets pushed to PyPI. |
In this PR I've removed the old release workflow (including the release environment in
tox.ini
) and added a completely new Github Actions workflow for handling a release. As part of that, I refactored the existing workflow into multiple files, including reusable workflowstest.yml
anddocs.yml
which get called in multiple ways from the primary workflows.Release step sequencing
The process I implemented here works as follows:
I'm not sure if that's the best way to sequence the steps involved in making a release, though. (Note I'm using "release" by itself to mean a tagged and published version, in contrast with "Github release") For one thing, the tests are not run against the actual distributable package, so there's a slight chance that a checkout of the repo would pass tests while the same tests run against the installable wheel would fail. (This is very unlikely, but it could happen if the package uses a hard-coded relative file or something like that.) This sequencing also carries the risk that the package might fail tests after creating the tag, and then we're faced with having a tagged release that doesn't work and can't be posted to PyPI, which kind of forces us to skip a version number.
There are some alternative sequences we could consider, such as this:
This would take care of the risk that tests pass against the checked-out source code but fail against the installed package. But it turns out to be somewhat tricky to set up tox to run tests against the installed package (although doable with some hacky workarounds in our case, at least for now).
Or we could do this:
That would take care of the risk of the tests failing when we've already created a tag, but it still has the problem where what we're testing is not exactly the same as what gets installed. And it's also awkward to have the semi-manual step of creating the Github release at the end, which basically needs Github Actions to prompt us to create a release but wait for us to actually fill in the release description. It would be nice to have the release notes prepared in advance.
I think the ideal sequence would be this:
But it's fairly complicated to make all those steps happen in exactly that order, as well as to arrange for tests to be run against the built package. I decided to leave that as a future goal, and for now just commit something that we can use to make a release so that we're not blocked in making this package available to the public.