-
Notifications
You must be signed in to change notification settings - Fork 205
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci/cron: move concurrency from Bash to Haskell #7696
Conversation
One small step further in removing the Bash. CHANGELOG_BEGIN CHANGELOG_END
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice, thank you!
"wait $pid", | ||
"done", | ||
"'"] | ||
Control.Concurrent.Async.forConcurrently_ (map uri $ assets release) (\url -> do |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
forConcurrently_
will spawn an unbounded number of threads. I guess we don’t care since the number of assets is small? If we do care, the easiest solution is usually just to throw a semaphore around the map function you pass to forConcurrently_
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The Bash code this replaces was also unbounded, so at least this is not getting worse. 🙂
I'm not concerned; we're not going to have a very big number of files anyway. If you are concerned, I can add a semaphore with, say, 30 slots, so it won't change anything for the happy path (16 assets atm) but will shield against future issues. I'd prefer to do that in a future PR, though.
"set -euo pipefail", | ||
"eval \"$(dev-env/bin/dade assist)\"", | ||
"cd \"" <> tmp <> "\"", | ||
"wget --quiet \"" <> show url <> "\"", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems like we could just call wget
directly and -o
to avoid the cd
but I’ll leave that up to you.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ì'm planning to remove wget in my next PR (perhaps next-next now, if the next one is the semaphore), so I don't see much point in trying to make the Bash code nicer. I would point though that we are not in the tmpdir in terms of Haskell process cwd as far as I can tell, so the cd
there is not superfluous as it stands.
/azp run |
Azure Pipelines successfully started running 1 pipeline(s). |
As requested in review of #7696. CHANGELOG_BEGIN CHANGELOG_END
As requested in review of #7696. CHANGELOG_BEGIN CHANGELOG_END
As promised in #7696. CHANGELOG_BEGIN CHANGELOG_END
As promised in #7696. CHANGELOG_BEGIN CHANGELOG_END
As promised in #7696. CHANGELOG_BEGIN CHANGELOG_END
One small step further in removing the Bash.
CHANGELOG_BEGIN
CHANGELOG_END