Update conformance test policy #23353
Conversation
Mostly doc updates and cruft removal - describe conformance test policy and howto in e2e-tests.md - rm e2e test info from testing.md in the name of DRY - rm cluster/test-conformance.sh; unusable in release tar, not e2e.go - update e2e test link in write-a-getting-started-guide.md
Totally guessing at the escaping in that string
|
Labelling this PR as size/L |
k8s-github-robot
added
the
size/L
|
GCE e2e build/test passed for commit 3f501f6. |
|
GCE e2e build/test passed for commit 9182fbf. |
|
Big thanks for removing those outdated scripts (#21843) |
| Finally, `[Conformance]` tests are tests we expect to pass on **any** Kubernetes cluster. The `[Conformance]` label does not supersede any other labels. `[Conformance]` test policies are a work-in-progress (see #18162). | ||
| Finally, `[Conformance]` tests represent a subset of the e2e-tests we expect to pass on **any** Kubernetes cluster. The `[Conformance]` label does not supersede any other labels. | ||
|
|
||
| As each new release of Kubernetes providers new functionality, the subset of tests necessary to demonstrate conformance grows with each release. Conformance is thus considered versioned with an eye towards backwards compatibility. Conformance tests for a given version should be run off of the release branch that corresponds to that version. Thus `v1.2` conformance tests would be run from the head of the `release-1.2` branch. eg: |
There was a problem hiding this comment.
do we have a limit on how long we expect conformance tests to work? I think the upgrade policy is trailing 3 versions (or something like that).
There was a problem hiding this comment.
makes sense to me, will link to the versioning doc
|
/cc @karlkfi for the mesos contrib change |
|
GCE e2e build/test passed for commit d8d785f. |
| @@ -31,7 +31,7 @@ TEST_ARGS="$@" | |||
|
|
|||
| KUBE_ROOT=$(cd "$(dirname "${BASH_SOURCE}")/../../.." && pwd) | |||
|
|
|||
| TEST_CMD="KUBECONFIG=~/.kube/config hack/conformance-test.sh" | |||
| TEST_CMD="KUBE_CONFORMANCE_TEST=y KUBECONFIG=~/.kube/config go run hack/e2e.go --test --test_args=\"--ginkgo.focus=\\[Conformance\\]\"" | |||
|
GCE e2e build/test passed for commit d794b86. |
| - Contains clients, namespace and auto resource cleanup | ||
| * See [coding conventions](coding-conventions.md). | ||
|
|
||
| ### e2e test philosophy |
There was a problem hiding this comment.
What is the thinking behind yanking all this?
There was a problem hiding this comment.
Primarily to DRY up the docs, so that future updates around e2e testing happen in one place vs. two places. My read of the e2e-tests doc was that all this info was sufficiently expressed there. Maybe that doc lacks something that's as tl;dr-ish as this?
There was a problem hiding this comment.
Sure we can always add back something similar if someone misses it, and I'm always for less code.
On the other hand I liked having at least one command to run tests to serve as a reminder (I pretty much never do anything without having a reference doc open reminding me how to do it):
Maybe add something like this in a future PR:
cd kubernetes
go run hack/e2e.go -v --build --up --test --down --test_args="--ginkgo.focus=\[Conformance\]"
fejta
added
the
lgtm
|
@k8s-bot test this Tests are more than 48 hours old. Re-running tests. |
|
GCE e2e build/test passed for commit d794b86. |
|
@k8s-bot test this [submit-queue is verifying that this PR is safe to merge] |
|
GCE e2e build/test passed for commit d794b86. |
|
Automatic merge from submit-queue |
Automatic merge from submit-queue Update conformance test policy Addresses kubernetes#14913 /cc @kubernetes/sig-testing
Addresses #14913
/cc @kubernetes/sig-testing