Release tarball script #4837

dvzrv · 2020-03-22T13:13:43Z

Purpose and Motivation

This script creates a source tarball from a valid tag in this repository by checking out the SuperCollider sources recursively in a temporary working directory. Optionally, this script can also provide a detached PGP signature. The naming scheme of the tarball and the contained directory follows the current naming scheme.
The script automatically removes its working directory on exit or fail and is tested on Linux and macOS.

Eventually I think this should run in CI, but it can also be done manually by whoever is in charge of the release process (to be able to provide a detached PGP signature).

To create a source tarball:

./package/create_source_tarball.sh -v Version-3.11.0

To create a source tarball with detached PGP signature:

./package/create_source_tarball.sh -v Version-3.11.0 -s my-pgp-keyid-or-mail-address

Fixes #4545 #4818.

Types of changes

Bug fix
New feature

To-do list

Code is tested
All tests are passing
Updated documentation
This PR is ready for review

joshpar · 2020-03-22T19:28:27Z

Thanks for this! I’ll test today with the source re-release for 3.11 Josh /* Josh Parmenter www.realizedsound.net/josh */

…

On Mar 22, 2020, at 06:14, David Runge ***@***.***> wrote: Purpose and Motivation This script creates a source tarball from a valid tag in this repository by checking out the SuperCollider sources recursively in a temporary working directory. Optionally, this script can also provide a detached PGP signature. The naming scheme of the tarball and the contained directory follows the current naming scheme. The script automatically removes its working directory on exit or fail and is tested on Linux and macOS. Eventually I think this should run in CI, but it can also be done manually by whoever is in charge of the release process (to be able to provide a detached PGP signature). To create a source tarball: ./package/create_source_tarball.sh -v Version-3.11.0 To create a source tarball with detached PGP signature: ./package/create_source_tarball.sh -v Version-3.11.0 -s my-pgp-keyid-or-mail-address Fixes #4545 #4818. Types of changes Bug fix New feature To-do list Code is tested All tests are passing Updated documentation This PR is ready for review You can view, comment on, or merge this pull request online at: #4837 Commit Summary package/create_source_tarball.sh: Adding script to create a source tarball. package/create_source_tarball.sh: Switching from gpg2 to gpg (default everywhere now). package/create_source_tarball.sh: Add specific gnupg requirement. File Changes A package/create_source_tarball.sh (122) Patch Links: https://github.com/supercollider/supercollider/pull/4837.patch https://github.com/supercollider/supercollider/pull/4837.diff — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

mossheim

thanks! couple things:

let's hook this into the build system proper by adding a target via add_custom_target
the resulting artefact should go in $BUILD_DIR/package (preferably) or $BUILD_DIR
please indent at 4 spaces per our style guidelines

package/create_source_tarball.sh

dvzrv · 2020-03-22T21:59:15Z

* let's hook this into the build system proper by adding a target via `add_custom_target`

Do you mean the cmake setup by that?
Wouldn't it make more sense to directly call this script in the CI upon tagging?

* the resulting artefact should go in `$BUILD_DIR/package` (preferably) or `$BUILD_DIR`

I guess an optional $BUILD_DIR environment variable could be made use of in there. Need to play with this a little.

mossheim · 2020-03-23T01:56:20Z

I guess an optional $BUILD_DIR environment variable could be made use of in there. Need to play with this a little.

oh, all i mean by that is that if you are building in supercollider/build then this artefact should go somewhere in supercollider/build.

Do you mean the cmake setup by that?

yes

Wouldn't it make more sense to directly call this script in the CI upon tagging?

for developers working with this repo i think it's easiest to both explain and use a system where all commands that produce executables, redistributable packages, etc. can be invoked through CMake.

dvzrv · 2020-03-23T10:58:17Z

oh, all i mean by that is that if you are building in supercollider/build then this artefact should go somewhere in supercollider/build.

I understand. However, how exactly would this work? How do we derive the tag that is supposed to be built ( -v flag to the script)? How do we handle potential user input (when potentially signing)?

I guess the tag string could be assembled using Version-SC_VERSION. This however doesn't help in the case where we're not building the exact commit a tag points to (e.g. someone executing cmake on a random commit post the Version-3.11.0 tag will still generate the tarball for Version-3.11.0).
I have the feeling the use-case is not in development of SuperCollider, but its release process.

mossheim · 2020-03-24T00:09:12Z

ah you're right, the design of the script makes it a bit unusual compared to other build targets. the command-line arguments could become interactive user input, ehh.. forget the build system integration.

everything looks fine, can either you or @joshpar document how this fits into the release process in the wiki (https://github.com/supercollider/supercollider/wiki/git-workflow-and-guidelines) please? and then i am good to approve this.

also, it would be very helpful if you rebased this on 3.11 instead of develop.

joshpar

please point to the 3.11 branch. After merge, I'll run this on the 3.11 release to package the new tarballs.

dvzrv · 2020-03-24T08:04:56Z

everything looks fine, can either you or @joshpar document how this fits into the release process in the wiki (https://github.com/supercollider/supercollider/wiki/git-workflow-and-guidelines) please? and then i am good to approve this.

Will do

package/create_source_tarball.sh: This script accepts two parameters: * v: The version (aka. the tag) the source tarball is supposed to be created for (mandatory). * s: The signing entity (an e-mail address or key ID) to sign the source tarball with a PGP private key. The script creates a tarball of the form SuperCollider-X.Y.Z-Source.tar.bz2, which extracts to a directory of the form SuperCollider-X.Y.Z-Source/ (here X.Y.Z denote a version without the conventional Version- prefix, used for the tags). Optionally, this script creates a detached PGP signature (tarball name + .asc). If the BUILD_DIR environment variable is set it will be used as the output_dir. In this case the source tarball (and the optional detached signature) are moved to BUILD_DIR/package and the output_dir is created on-the-fly. If BUILD_DIR is not provided the script defaults back to its own location (the directory the script resides in) as output_dir. The script utilizes a trap to cleanup temporary working directories (even if unsuccessful). .gitignore: Adding ignores for source tarball and detached signature creation. Signed-off-by: David Runge <dave@sleepmap.de>

mossheim

thanks!

dvzrv · 2020-03-24T11:43:46Z

I guess as a follow-up the scripts previously used could be removed.

@joshpar how do you currently generate the binary release assets (e.g. .dmg for macOS and .exe for Windows)? Are you using binaries created in CI in that case? I'm trying to figure out, if any of the scripts in package/* are involved in that (it seems not).

Also: Please add the source tarball for Version-3.11.0 to the assets at your earliest convenience, so that it can be updated downstream.

joshpar · 2020-03-24T15:28:47Z

Mac OS unsigned and Win are generated through CI. I then use the Mac one to generate the notarized version. Thanks! I’ll update the assets soon. Josh

…

On Mar 24, 2020, at 04:44, David Runge ***@***.***> wrote: I guess as a follow-up the scripts previously used could be removed. @joshpar <https://github.com/joshpar> how do you currently generate the binary release assets (e.g. .dmg for macOS and .exe for Windows)? Are you using binaries created in CI in that case? I'm trying to figure out, if any of the scripts in package/* are involved in that (it seems not). Also: Please add the source tarball for Version-3.11.0 <https://github.com/supercollider/supercollider/releases/tag/Version-3.11.0> to the assets at your earliest convenience, so that it can be updated downstream. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4837 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAANHLGDXCUESBHX4WJU7JDRJCMIBANCNFSM4LRJH2LQ>.

dvzrv · 2020-04-01T15:14:10Z

@joshpar can you please generate and upload the tarball? Currently 3.11.0 can not be built by downstreams relying on the tarball and the reproducible build on Arch Linux is broken due to it missing.

joshpar · 2020-04-01T17:51:24Z

@dvzrv - working on this now. If I run this:
./create_source_tarball.sh -v 3.11.0

I get an error about there being no branch on the remote (which is fine - I can change it to 3.11) - but I have a concern with this... 3.11's branch has changes POST 3.11 release. Is this pulling the branch? Would pointing to a tag be better?

dvzrv · 2020-04-01T20:15:05Z

@joshpar you need to reference the tag:

./create_source_tarball.sh -v Version-3.11.0

Otherwise git will probably use the branch. Maybe I can make the wiki more clear about this?

dvzrv · 2020-04-01T20:18:25Z

@joshpar I've updated the documentation to be more explicit:
https://github.com/supercollider/supercollider/wiki/git-workflow-and-guidelines#releasing

joshpar · 2020-04-01T20:20:16Z

@dvzrv - thanks! please see https://github.com/supercollider/supercollider/releases/tag/Version-3.11.0 to see if this looks right to you

dvzrv · 2020-04-03T08:10:30Z

@joshpar looks all fine! I've used it in this package in version 3.11.0-4.

Thanks a lot! :)

_includes/download.md: The source tarball name has been generalized with supercollider/supercollider#4837 and is now not Linux specific anymore. The naming scheme is now 'SuperCollider-<version>-Source.tar.bz2'.

dvzrv requested review from mossheim and joshpar March 22, 2020 13:14

dvzrv added the organizational label Mar 22, 2020

mossheim suggested changes Mar 22, 2020

View reviewed changes

package/create_source_tarball.sh Outdated Show resolved Hide resolved

package/create_source_tarball.sh Outdated Show resolved Hide resolved

joshpar approved these changes Mar 24, 2020

View reviewed changes

dvzrv changed the base branch from develop to 3.11 March 24, 2020 08:00

dvzrv force-pushed the release_tarball_script branch 4 times, most recently from 5e17bc9 to ee8048f Compare March 24, 2020 08:36

dvzrv force-pushed the release_tarball_script branch from ee8048f to 7d7c801 Compare March 24, 2020 08:45

mossheim approved these changes Mar 24, 2020

View reviewed changes

mossheim merged commit f849619 into supercollider:3.11 Mar 24, 2020

dvzrv mentioned this pull request Mar 24, 2020

Improve our source tarball packaging process #4545

Closed

dvzrv deleted the release_tarball_script branch March 24, 2020 11:32

dvzrv mentioned this pull request Mar 25, 2020

Removing legacy package scripts #4840

Merged

4 tasks

dvzrv mentioned this pull request May 7, 2020

Fixing source tarball naming supercollider/supercollider.github.io#217

Merged

dvzrv mentioned this pull request Aug 4, 2021

Latest 3.12.0 release does not have GPG signature #5533

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Release tarball script #4837

Release tarball script #4837

dvzrv commented Mar 22, 2020 •

edited

Loading

joshpar commented Mar 22, 2020 via email

mossheim left a comment

dvzrv commented Mar 22, 2020

mossheim commented Mar 23, 2020

dvzrv commented Mar 23, 2020

mossheim commented Mar 24, 2020

joshpar left a comment

dvzrv commented Mar 24, 2020

mossheim left a comment

dvzrv commented Mar 24, 2020

joshpar commented Mar 24, 2020 via email

dvzrv commented Apr 1, 2020

joshpar commented Apr 1, 2020

dvzrv commented Apr 1, 2020

dvzrv commented Apr 1, 2020

joshpar commented Apr 1, 2020

dvzrv commented Apr 3, 2020

Release tarball script #4837

Release tarball script #4837

Conversation

dvzrv commented Mar 22, 2020 • edited Loading

Purpose and Motivation

Types of changes

To-do list

joshpar commented Mar 22, 2020 via email

mossheim left a comment

Choose a reason for hiding this comment

dvzrv commented Mar 22, 2020

mossheim commented Mar 23, 2020

dvzrv commented Mar 23, 2020

mossheim commented Mar 24, 2020

joshpar left a comment

Choose a reason for hiding this comment

dvzrv commented Mar 24, 2020

mossheim left a comment

Choose a reason for hiding this comment

dvzrv commented Mar 24, 2020

joshpar commented Mar 24, 2020 via email

dvzrv commented Apr 1, 2020

joshpar commented Apr 1, 2020

dvzrv commented Apr 1, 2020

dvzrv commented Apr 1, 2020

joshpar commented Apr 1, 2020

dvzrv commented Apr 3, 2020

dvzrv commented Mar 22, 2020 •

edited

Loading