infra/hoogle: use nix #9362
Merged
infra/hoogle: use nix #9362
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is a demonstration, commit-by-commit, of how to use the blue/green deployment for Hoogle servers. The actual change (using nix) is stolen from #9352. CHANGELOG_BEGIN CHANGELOG_END
First step is to just copy the bits we want to change to a field in the `h_clusters` local. At this point, `terraform plan` should report nothing to do as we have changed the Terraform code but not what it produces.
In this case, we know the Hoogle service works with just two machines, so we don't need to scale up the green group. Similarly, the blue group is already set to just one machine so we don't need to scale it down before proceeding. This step does require a `terraform apply`.
In this case, this is mostly just looking at the logs of the blue machine. I did it with ``` gcloud beta \ compute instances \ --project "da-dev-gcp-daml-language" \ tail-serial-port-output \ --zone "us-east4-a" \ hoogle-blue-jcw2 ``` Because it was so much faster, I did take a minute to log into the machine and check everything was fine. In order to do that I had to temporaily open the network, and I though I should also leave that code in place to simplify future operations.
Now that we have a working blue group, we can switch all instances to it.
garyverhaegen-da
force-pushed
the
get-hoogle-from-nix
branch
from
eba51fb to
1b227a7
Compare
cocreature
approved these changes
Apr 8, 2021
|
Does it actually accomplish the goal, i.e., is it faster? |
|
Yes, anecdotally, the total time it takes for a machine to boot has gone down from ~35 minutes to ~3 minutes. I've only looked at one machine on each side, but I believe this is still significant. |
|
Nice! that seems like a good improvement for a relatively simple change |
garyverhaegen-da
added a commit
that referenced
this pull request
Apr 8, 2021
After #9362, hoogle stopped returning any result. It was up and running, but with an empty database. Problem was two-fold: 1. In the switch to `nix`, I lost track of the fact that we were previously doing all the work in `~/hoogle` rather than `~/`, which broke the periodic refresh. 2. The initial setup has been broken for months; machines have been initializing with an empty DB and only getting data on the first refresh since #7370. CHANGELOG_BEGIN CHANGELOG_END
Merged
garyverhaegen-da
added a commit
that referenced
this pull request
Apr 8, 2021
After #9362, hoogle stopped returning any result. It was up and running, but with an empty database. Problem was two-fold: 1. In the switch to `nix`, I lost track of the fact that we were previously doing all the work in `~/hoogle` rather than `~/`, which broke the periodic refresh. 2. The initial setup has been broken for months; machines have been initializing with an empty DB and only getting data on the first refresh since #7370. CHANGELOG_BEGIN CHANGELOG_END
azure-pipelines bot
pushed a commit
that referenced
this pull request
Apr 14, 2021
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging. @remyhaemmerle-da is in charge of this release. Commit log: ``` 2dc09ba LF: fix contract ID freshness check (#9370) 1627b70 Pattern matching for RoundingMode (#9381) b41e1ed fix daml-lf/governance.rst (#9399) 8f885f4 MutableCacheBackedContractStore implementation (#9378) 7890381 Publish EE JSON API to artifactory (#9392) 6ab9655 Fix scala version in quickstart-scala (#9401) 87d3f89 offboarding @hurryabit (#9398) 29fcebe Fix recording of cache metrics [KVL-888] (#9382) 67b0e2b Drop hurryabit from release rotation 😭 (#9393) 24c64ce Clean up participant-integration-api command line help page (#9385) 2f4b32f update NOTICES file (#9391) cda2940 KVL-861 Add support for dumping expected and actual updates in the integrity checker (#9379) 7fc86b7 Fix flaky StateCacheSpec test (#9389) 301dcd9 DPP-316: Enable the use of the append only index database (#9368) 9ac74e6 Oracle json-api: websockets and testing (#9278) 104ad06 Streaming contract state events (on the append-only schema) [DPP-304] (#9365) da6a0d6 Extend prometheus metrics with min/mean/max values (#9380) 10edc66 Implement Prometheus metrics back-end (#9373) 89b5dbb LedgerDao and ContractsReader interface updates (#9349) 8480032 daml package: bump timeout for tests (#9377) bca24a9 Make LogEntryId computation strategy injectable to submission validators (#9302) 78dc238 StateCache implementation for mutable contract state (#9299) 91b65e8 Patch hoogle binary to include bugfix (#9366) dc4b9e5 Publish trigger service EE fat JARs (#9363) 11e5dd3 link to postgres docs (#9353) f84b6ab daml build: add a --access-token-file for remote dependencies (#9358) dfe26b9 fix hoogle (#9364) c220e05 infra/hoogle: use nix (#9362) 5a983e3 Deduplicate LfValueTranslation cache (#9354) e84c954 Expose Oracle support in the EE trigger service (#9342) 0303017 Delete "testing with scenarios" section (#9360) 587bff2 Fix comment formatting in state (#9359) 4c231dc Update canton tests to pre 0.23.0 release (#9356) bc4e00b Run Canton tests with JDK11 (#9355) 948d4dd infra: hoogle blue/green tf (#9351) 69ecf57 Removing Git as a prerequisit from the GSG (#9202) 2745bc0 macos: move cache setup to step 2 (#9350) 38c417e bump hoogle ubuntu (#9344) a2ccf1b LF: release LF 1.13 archive snapshot (#9348) 867e625 Add exception handling to Daml Script (#9324) c97db24 fix macOS cache cleaning (#9343) bac3521 Add sqrt to DA.Math (#9346) 06701f7 Expose rounding modes as constructors + add BigNumeric docs. (#9336) b90a9c3 Remove exception message from AnyException (#9328) 35759fc LF: Freeze archive proto for LF 1.13 (#9345) 0251e93 Improve TX normalization. (#9341) 2b1f882 daml-ledger: new list-packages command (#9325) 568a852 LF: release preview of LF 1.13 (#9329) 8b8f736 update compat versions for 1.12.0-snapshot.20210406.6646.0.631db446 (#9332) 960134d Fix typo in profiler docs, file name starts with profile (not profiler) (#9338) 0eac00f Avoid deprecated forHostWithLedgerIdDiscovery in quickstart-java (#9337) 5b11571 Introduce parallel indexer (#9304) e84a94e Update issue number for BigNumeric Divisible instance. (#9334) 8f7ef05 Avoid mention of scenarios in Sandbox docs (#9335) 93de56d Optimize race condition integration tests [DPP-330] (#9320) 74956e2 LF: BigNumeric spec. (#8899) 4d0c6db daml ledger: better error messages for missing host/port args (#9322) 247a1a3 snapshot to test non-repudiation publish (#9326) 7b3b669 LF: add test for bigNumeric operations (#9310) fd63cf0 use {tag, value} format for SQLizing advanced JSON queries (#9321) ``` Changelog: ``` - [Engine] Fix contract ID freshness check when validating transaction - [Integration Kit] new streaming query for contract state events - [Integration Kit] indexing contract keys for consuming exercise events - Add prometheus metrics as a cli option in the participant integration api [daml build] A new flag `--access-token-file` is added for the `daml build` command. It allows the specify the path to an access token to authenticate against the ledger API. This is needed if the project depends on a remote Daml package hosted on such a ledger. Alternatively, the path to the token can also be specified in the `daml.yaml` project file under the `ledger.access-token-file` field. [daml packages] A new `daml packages list` command has been added to list packages deployed on a remote Daml ledger. - Daml: (Early access) add support for BigNumeric - [HTTP JSON API] Range queries within variant data would not return matching data when using the PostgreSQL backend with JSON API; this is fixed. See `issue #9321 <https://github.com/digital-asset/daml/pull/9321>`__. ``` CHANGELOG_BEGIN CHANGELOG_END
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a demonstration, commit-by-commit, of how to use the blue/green deployment for Hoogle servers.
The actual change (using nix) is stolen from #9352.
CHANGELOG_BEGIN
CHANGELOG_END