Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Disable http2 with Nix to work around segfaults #4427

Merged
merged 2 commits into from
Feb 6, 2020
Merged

Disable http2 with Nix to work around segfaults #4427

merged 2 commits into from
Feb 6, 2020

Conversation

aherrmann-da
Copy link
Contributor

On CI and locally we observe occasional segmantation faults of nix. A known issue since Nix 2.2.2 is that HTTP2 support can cause such segmentation faults. Since Nix 2.3.2 it is possible to disable HTTP2 via a command-line flag, which reportedly solves the issue. See NixOS/nix#2733 (comment)

Pull Request Checklist

  • Read and understand the contribution guidelines
  • Include appropriate tests
  • Set a descriptive title and thorough description
  • Add a reference to the issue this PR will solve, if appropriate
  • Include changelog additions in one or more commit message bodies between the CHANGELOG_BEGIN and CHANGELOG_END tags
  • Normal production system change, include purpose of change in description

NOTE: CI is not automatically run on non-members pull-requests for security
reasons. The reviewer will have to comment with /AzurePipelines run to
trigger the build.

garyverhaegen-da
garyverhaegen-da approved these changes Feb 6, 2020
View reviewed changes
Copy link
Contributor

@garyverhaegen-da garyverhaegen-da left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does that mean we should be able to remove all the while loops to retry nix-build -A tools -A cached nix?

@aherrmann-da
Copy link
Contributor Author

aherrmann-da commented Feb 6, 2020 •
edited
Loading

Does that mean we should be able to remove all the while loops to retry nix-build -A tools -A cached nix?

Hopefully, I'll add a commit to this PR, so we can test it.

Looking at that code these loops are working around occasional download issues

# Nix cache downloads can sometimes be flaky and end with "unexpected end-of-file" so we
# repeat this a few times. There does not seem to be an option that we can pass to nix
# to make it retry itself. See https://github.com/NixOS/nix/issues/2794 for the issue requesting
# this feature.

So, unrelated.

@cocreature
Copy link
Contributor

You probably want to update https://github.com/digital-asset/daml/blob/master/ci/dev-env-install.sh#L26

@aherrmann-da
Copy link
Contributor Author

You probably want to update https://github.com/digital-asset/daml/blob/master/ci/dev-env-install.sh#L26

Thanks, good point!

Using a fresh Docker container with Ubuntu 19.10 and Nix 2.3.2 I was able to reliably reproduce the segfaults on master in both the dev-env setup and Bazel rules_nixpkgs fetches. Adding --option http2 false in rules_nixpkgs and the dev-env these issues went away and . .envrc and bazel build damlc succeeded without hickups.

cocreature
cocreature approved these changes Feb 6, 2020
View reviewed changes
Copy link
Contributor

@cocreature cocreature left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, let’s see how this goes 👍

@mergify mergify bot merged commit a982856 into master Feb 6, 2020
@mergify mergify bot deleted the nixpkgs-disable-http2 branch February 6, 2020 15:15
@aherrmann-da aherrmann-da mentioned this pull request Feb 7, 2020
Merged
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@garyverhaegen-da garyverhaegen-da garyverhaegen-da approved these changes

@cocreature cocreature cocreature approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@aherrmann-da @cocreature @garyverhaegen-da @aherrmann