Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: progress on codesigning macOS builds #620

Merged
merged 29 commits into from
Jan 26, 2022
Merged

Conversation

ErikBjare
Copy link
Member

@ErikBjare ErikBjare commented Jun 23, 2021

Progress on code signing, building on lessons learned in #344.

Co-authored-by: @xylix

What works

  • pyinstaller produces a signed .app, which passes xcnotary precheck and doesn't return 'adhoc' when checked with codesign -cv ActivityWatch.app.

Issues

Related PRs

Bounty recipients

Resources

xylix and others added 6 commits March 3, 2020 15:21
Fix travis warnings, cleanup is now disabled by default. Rename some keys which travis was aliasing anyway

Fixes for macOS signing

Add osx signing keys in before_install step
try fixing macos-signing script and makefile command. Add warning for running it locally

Move import-macos-cert script to scripts/ci

Increase macOS signing tool verbosity
@@ -17,6 +17,9 @@ current_release = subprocess.run(
).stdout.strip()
print("bundling activitywatch version " + current_release)

entitlements_file = Path(".") / "scripts" / "package" / "entitlements.plist"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using . here is probably bad.

@ErikBjare ErikBjare added bounty An issue or PR with a bounty for solving platform: macos labels Aug 29, 2021
@ErikBjare
Copy link
Member Author

This as at the point where we should really give it some keys to try using in CI.

Builds might still not work, but CI failing due to missing keys needs to be resolved sooner or later anyway.

@oreHGA
Copy link

oreHGA commented Dec 31, 2021

Happy holidays! Was trying the latest version on the Mac OS Monterey 12.0.1

Would this be something that this PR fixes?
Screen Shot 2021-12-31 at 12 39 51 AM

Edit:
Got the app running by manually running these:

sudo xattr -cr /Applications/ActivityWatch.app
sudo codesign --force --deep --sign - /Applications/ActivityWatch.app

Happy to help making this work out of the box but not sure where to start..lmk if there's more description on what needs input :)

@kewde
Copy link
Contributor

kewde commented Jan 5, 2022

I managed to get it to build but notarization will not succeed (neither on the app or the dmg).
PyInstaller is not enabling the hardened runtime (flags=0x2(adhoc) should be flags=0x10000(runtime)):

(venv) user@MacOSs-iMac dist % codesign -dvv ActivityWatch.app
Executable=/Users/user/Documents/projects/activitywatch/activitywatch/dist/ActivityWatch.app/Contents/MacOS/aw-qt
Identifier=net.activitywatch.ActivityWatch
Format=app bundle with Mach-O thin (x86_64)
CodeDirectory v=20400 size=20920 flags=0x2(adhoc) hashes=643+7 location=embedded
Signature=adhoc
Info.plist entries=12
TeamIdentifier=not set
Sealed Resources version=2 rules=13 files=992
Internal requirements count=0 size=12

https://wiki.freepascal.org/Hardened_runtime_for_macOS#Verifying_hardening

@ErikBjare
Copy link
Member Author

@kewde Thanks for the reply, noting the same issue. In the CI logs I see a couple lines of Code signing identity: None, which suggests PyInstaller only does ad-hoc signing.

I also found pyinstaller/pyinstaller#4701 which suggests you can turn on hardened runtime at codesign-time with the --options=runtime param.

@kewde
Copy link
Contributor

kewde commented Jan 5, 2022

@ErikBjare Investigating this.

It shouldn't be adhoc signing, instead it should be signing using the certificate as specified identifier.

@ErikBjare
Copy link
Member Author

Might be due to the certificate not being available when PyInstaller does its thing (if that is the case, would be nice if it threw an error instead).

So I guess we need to retrieve the cert somehow, I assume you can't do that with just xcrun notarytool store-credentials ...?

Also found this, which seems pretty comprehensive: https://stackoverflow.com/questions/64652704/how-to-notarize-an-macos-command-line-tool-created-outside-of-xcode

@kewde
Copy link
Contributor

kewde commented Jan 5, 2022

The certificate is available on my machine, so it should work. This is something else, will comment once I know.

@kewde
Copy link
Contributor

kewde commented Jan 5, 2022

@ErikBjare I used the original codesign.sh script which did do a proper job in signing, it enabled the hardened runtime and it set the team identifier to my personal team. So this is a PyInstaller issue - taking a deeper look!

user@MacOSs-iMac dist % codesign --verify --display --verbose ActivityWatch.app
Executable=/Users/user/Documents/projects/activitywatch/activitywatch/dist/ActivityWatch.app/Contents/MacOS/aw-qt
Identifier=net.activitywatch.ActivityWatch
Format=app bundle with Mach-O thin (x86_64)
CodeDirectory v=20500 size=20939 flags=0x10000(runtime) hashes=643+7 location=embedded
Signature size=8980
Timestamp=5 Jan 2022 at 12:26:22
Info.plist entries=12
TeamIdentifier=MYIDENTIFIER
Runtime Version=11.3.0
Sealed Resources version=2 rules=13 files=992
Internal requirements count=1 size=192

@ErikBjare
Copy link
Member Author

Thanks for all the help on this @kewde. I don't have access to a macOS box today, but will investigate further tomorrow or friday.

@kewde
Copy link
Contributor

kewde commented Jan 5, 2022

@ErikBjare codesigning & notarization works, will PR 🎉

PyInstaller codesigning command line is rather buggy: the --codesign-idenity does not work.
The codesign_identity for the BUNDLE(...) is derived from the EXE(...) and COLLECT(...).

Notarization output

bash ./scripts/notarize.sh
Detecting availability of notarization tools
+ Found notarytool
Notarizing: dist/ActivityWatch.app

This process stores your credentials securely in the Keychain. You reference these credentials later using a profile name.

Validating your credentials...
Success. Credentials validated.
Credentials saved to Keychain.
To use them, specify `--keychain-profile "ActivityWatchSigner"`
Notarization: starting for dist/ActivityWatch.app.zip
Notarization: in progress for dist/ActivityWatch.app.zip
Conducting pre-submission checks for ActivityWatch.app.zip and initiating connection to the Apple notary service...
Submission ID received
  id: SNIPPED
Successfully uploaded file52,7 MB of 52,7 MB)   
  id: SNIPPED
  path: /Users/user/Documents/projects/activitywatch/activitywatch/dist/ActivityWatch.app.zip
Waiting for processing to complete.
Current status: Accepted...............
Processing complete
  id: SNIPPED
  status: Accepted
Processing: /Users/user/Documents/projects/activitywatch/activitywatch/dist/ActivityWatch.app
Processing: /Users/user/Documents/projects/activitywatch/activitywatch/dist/ActivityWatch.app
The staple and validate action worked!

@kewde
Copy link
Contributor

kewde commented Jan 7, 2022

@ErikBjare it seems like the CI is not importing the certificate into the keychain, fixed in #703.

Please note: for notarization you must use a developer id (used for distribution), not a development id.

@ErikBjare
Copy link
Member Author

ErikBjare commented Jan 26, 2022

Seems like signing & notarization works now, nice!

Still can't get the app to open when I double-click in Finder, but works with open ActivityWatch.app from the terminal. Probably related to pyinstaller/pyinstaller#5109 (comment) or pyinstaller/pyinstaller#1892.

That can wait for another PR though.

@ErikBjare
Copy link
Member Author

Merging this, fixing the double-click in Finder will have to wait for another PR...

@ErikBjare ErikBjare merged commit 093f630 into master Jan 26, 2022
@ErikBjare ErikBjare deleted the dev/macos-sign-and-notarize branch January 26, 2022 12:19
@ErikBjare
Copy link
Member Author

Huge thanks @kewde for helping with getting this done!

@kewde
Copy link
Contributor

kewde commented Jan 26, 2022

@ErikBjare re: double clicking Finder

We should check out the info.plist file - it's autogenerated by PyInstaller but we can modify it a bit iirc.
#630 (comment)

^It's not the solution but seems like it might be a bit funky

@ErikBjare
Copy link
Member Author

ErikBjare commented Jan 28, 2022

@kewde Unfortunately it does not seem to be that simple. Pretty sure that the cause/fix is pyinstaller/pyinstaller#5109 (comment).

I worked briefly on a minimum viable reproducible example yesterday, and managed to use macOS syslog as a way to get some basic logging/output even when double-clicked from Finder, so we can find the offending calls to open() (or whatever).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bounty An issue or PR with a bounty for solving platform: macos
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants