Skip to content

Commit

Permalink
release: Update with new experimental split cloud/db process
Browse files Browse the repository at this point in the history
  • Loading branch information
quodlibetor committed Dec 15, 2020
1 parent d362e5b commit ab789d2
Showing 1 changed file with 17 additions and 12 deletions.
29 changes: 17 additions & 12 deletions .github/ISSUE_TEMPLATE/release.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,9 @@ a release is published.

### Test the release candidate

Cloud engineers run many of the steps in this section, their steps are prefixed with **cloud
engineer**.

To run the required load tests on the release candidate tag, Materialize employees
can follow [these instructions for running semi-automatic load tests][load-instr]
in the infrastructure repository. All of these tests can be run in parallel.
Expand All @@ -108,21 +111,23 @@ in the infrastructure repository. All of these tests can be run in parallel.

You can continue on to the next step, but remember to verify that this test passes.

- [ ] Start the load tests according to [the same instructions][load-instr], using your release tag as the
`git-ref` value for the release benchmarks. You can use [This commit][] as an example to follow.
- [ ] **cloud engineer** Start the load tests according to [the same instructions][load-instr],
using your release tag as the `git-ref` value for the release benchmarks. You can use [This
commit][] as an example to follow.

[This commit]: https://github.com/MaterializeInc/infra/commit/fd7f594d6f9fb2fda3a604f21b730f8d401fe81c

- [ ] Find the load tests in https://grafana.i.mtrlz.dev/d/materialize-overview, and link
to them in #release, validating that data is present. Note that the default view of
that dashboard is of a full day, so it may look like the test started and aborted
suddenly:
- [ ] **cloud engineer** Find the load tests in
https://grafana.i.mtrlz.dev/d/materialize-overview, and link to them in #release, validating
that data is present. Note that the default view of that dashboard is of a full day, so it may
look like the test started and aborted suddenly:

- [ ] chbench
- [ ] billing-demo
- [ ] perf-kinesis

- [ ] Let the tests run for at least 24 hours, with the following success criteria:
- [ ] **cloud engineer** Let the tests run for at least 24 hours, with the following success
criteria:

- [ ] chbench: The ingest rate should not be slower than the previous release.
- [ ] billing-demo: The container should run and finish without error. You can get the exit code
Expand All @@ -141,12 +146,12 @@ in the infrastructure repository. All of these tests can be run in parallel.
- [ ] perf-kinesis: The "Time behind external source" dashboard panel in Grafana should
have remained at 0ms or similar for the entirety of the run.

- [ ] Let the chaos test run for 24 hours. The test's `chaos_run` container should complete
with a `0` exit code. To check this, SSH into the EC2 instance running the chaos test and
run `docker ps -a`. You can ssh in using our [teleport cluster][], the chaos test has a
`purpose=chaos` label.
- [ ] **cloud engineer** Let the chaos test run for 24 hours. The test's `chaos_run` container
should complete with a `0` exit code. To check this, SSH into the EC2 instance running the chaos
test and run `docker ps -a`. You can ssh in using our [teleport cluster][], the chaos test has a
`purpose=chaos` label.

- [ ] Remove all load test machines, documented on [the same page][load-instr].
- [ ] **cloud engineer** Remove all load test machines, documented on [the same page][load-instr].

[teleport cluster]: https://tsh.i.mtrlz.dev/cluster/tsh/nodes

Expand Down

0 comments on commit ab789d2

Please sign in to comment.